Module 3: Prompt Engineering

The prompt is the DNA of your agent. It defines personality, capabilities, constraints, and decision-making style. For a simple chatbot, a mediocre prompt produces mediocre answers. For an agent, a mediocre prompt produces wrong actions — and wrong actions have real-world consequences.

Consider an agent that manages a database. If the system prompt does not explicitly forbid destructive operations, the agent might decide that DROP TABLE users is a reasonable way to "clean up" data. Prompt engineering for agents is not about getting nicer text — it is about controlling behaviour.

Prompt engineering affects every aspect of agent behaviour:

• Tool selection — which tool to call and when (the wrong tool wastes time and money)
• Multi-step reasoning — errors at step 1 compound through every subsequent step
• Safety boundaries — preventing harmful, irreversible, or costly actions
• Output format — downstream systems expect structured data, not free-form text
• Cost efficiency — a well-crafted prompt reduces unnecessary tool calls and LLM iterations

Every effective agent prompt has four components, each serving a distinct purpose. Missing any one of them leads to predictable failure modes.

Here is a complete example that includes all four components:

system_prompt = """You are a financial research assistant specialising in
public company analysis.

CONTEXT:
- You have access to tools: search_sec_filings, get_stock_price, search_news.
- The user is a portfolio manager at an investment firm.
- Current date: 2026-03-28.

CONSTRAINTS:
- Never provide specific investment advice ("buy" / "sell" recommendations).
- Always cite your data sources with URLs or filing references.
- If data is older than 30 days, explicitly warn the user.
- If you are unsure, say so. Never fabricate financial data.

FORMAT:
- Use markdown for readability.
- Present numerical data in tables.
- End each response with a "Sources" section.
"""

The Role Statement

The role statement does more than set a persona. It activates domain-specific knowledge within the model's training data. "You are a Python developer" produces different code style and library choices than "You are a data scientist." Be specific about expertise level, domain, and working style.

# Weak role statement
system_prompt = "You are a helpful assistant."

# Strong role statement
system_prompt = """You are a senior backend engineer specialising in Python
microservices. You follow PEP 8, write type hints, and prefer composition
over inheritance. You have deep experience with FastAPI, SQLAlchemy, and
PostgreSQL."""

The system prompt is the most powerful lever to control agent behaviour. It is sent with every API call and acts as a persistent set of instructions that the model treats with high priority. For agents, the system prompt must go far beyond personality — it must define decision-making procedures.

A good agent system prompt answers these questions:

• Identity: Who is the agent? What is its expertise?
• Tool guidance: When should each tool be used? In what order?
• Failure handling: What should the agent do when a tool fails or returns unexpected results?
• Tone and style: How should the agent communicate? Formal? Casual? Terse?
• Escalation: When should the agent stop and ask the user for help?

system_prompt = """You are a customer support agent for Acme Software.

IDENTITY:
- You are friendly, professional, and concise.
- You represent Acme Software and always act in the customer's best interest.

TOOLS (use in this priority order):
1. search_docs: Use FIRST for any product question. Always search before answering.
2. lookup_order: Use when the customer mentions an order number or shipping.
3. create_ticket: Use for bugs, feature requests, or issues you cannot resolve.

DECISION RULES:
- If search_docs returns no results, say "I don't have information on that"
  and offer to create a support ticket. NEVER guess at technical specifications.
- If the customer is frustrated, acknowledge their feelings before problem-solving.
- If you need information the customer hasn't provided, ask ONE clear question.

ESCALATION:
- Refund requests over $100: create a ticket tagged "manager-review".
- Security concerns: immediately create a ticket tagged "security-urgent".

FORMAT:
- Keep responses under 150 words unless the customer asks for details.
- Use bullet points for multi-step instructions.
"""

System Prompt in OpenAI vs Anthropic

The two major providers handle system prompts differently in their API calls:

# OpenAI: system prompt is a message in the messages array
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": system_prompt},   # <-- system message
        {"role": "user", "content": "How do I reset my password?"}
    ]
)

# Anthropic: system prompt is a separate parameter
import anthropic
client = anthropic.Anthropic()
message = client.messages.create(
    model="claude-sonnet-4-20250514",
    max_tokens=1024,
    system=system_prompt,                                # <-- separate param
    messages=[
        {"role": "user", "content": "How do I reset my password?"}
    ]
)

Few-shot prompting provides examples of the desired input/output behaviour. Chain-of-thought (CoT) prompting instructs the model to reason step by step before giving a final answer. Both techniques dramatically improve accuracy, especially for agents that need to make complex decisions.

Few-Shot Prompting

Instead of describing what you want, show the model. This is especially effective for controlling output format and decision-making patterns:

system_prompt = """You are a data classification agent. Classify customer
messages into categories and extract key entities.

EXAMPLES:

Input: "My order #12345 hasn't arrived yet, it's been 2 weeks"
Output: {"category": "shipping_delay", "order_id": "12345",
         "urgency": "high", "action": "lookup_order"}

Input: "How do I change my subscription plan?"
Output: {"category": "account_management", "order_id": null,
         "urgency": "low", "action": "search_docs"}

Input: "Your app crashes every time I open the settings page"
Output: {"category": "bug_report", "order_id": null,
         "urgency": "medium", "action": "create_ticket"}
"""

Chain-of-Thought Prompting

CoT prompting asks the model to "think out loud" before acting. This is critical for agents because it makes the reasoning process visible (and debuggable):

system_prompt = """You are a research assistant. When given a question,
reason through it step by step before taking any action.

REASONING PROCESS:
1. Identify what information you need to answer the question.
2. Determine which tools can provide that information.
3. Plan the order of tool calls (some may depend on earlier results).
4. Execute your plan, verifying each result before proceeding.
5. Synthesise the results into a clear answer.

IMPORTANT: Always show your reasoning in a <thinking> block before
taking action. This helps the user understand your approach.

Example reasoning:
<thinking>
The user wants to compare Q4 revenue for Apple and Microsoft.
I need: (1) Apple Q4 revenue, (2) Microsoft Q4 revenue.
I'll search SEC filings for both, then present a comparison table.
</thinking>
"""

The combination of few-shot + CoT is especially powerful. Provide examples that include the reasoning process, and the model will follow the same pattern:

# Combining few-shot and chain-of-thought
messages = [
    {"role": "system", "content": "Solve math word problems step by step."},
    {"role": "user", "content": "A store has 50 apples. They sell 30% on Monday and 20% of the remainder on Tuesday. How many are left?"},
    {"role": "assistant", "content": """Step 1: Start with 50 apples.
Step 2: Monday sales = 50 * 0.30 = 15 apples sold. Remaining = 50 - 15 = 35.
Step 3: Tuesday sales = 35 * 0.20 = 7 apples sold. Remaining = 35 - 7 = 28.
Answer: 28 apples remain."""},
    {"role": "user", "content": "A factory produces 200 widgets per day..."}
]

In production, prompts are never static strings. They are templates populated with dynamic context at runtime — the user's name, today's date, available tools, retrieved documents, and more. Good template design keeps your prompts maintainable and testable.

Using Python f-strings (Simple Cases)

def build_system_prompt(user_name: str, tools: list[str], date: str) -> str:
    tool_list = ", ".join(tools)
    return f"""You are a personal assistant for {user_name}.
Current date: {date}.
Available tools: {tool_list}.

RULES:
- Always greet {user_name} by name.
- Use tools when you need real-time information.
"""

Using string.Template (Safer for User Input)

from string import Template

prompt_template = Template("""You are an assistant for $user_name.
Current date: $current_date
Available tools: $tool_list

RULES:
- Address the user by name.
- If a tool fails, explain the error and suggest an alternative.
""")

system_prompt = prompt_template.substitute(
    user_name="Alice",
    current_date="2026-03-28",
    tool_list="search_web, get_weather, send_email"
)

Using Jinja2 (Advanced / Production)

from jinja2 import Template

prompt_template = Template("""You are a support agent.

AVAILABLE TOOLS:
{% for tool in tools %}
- {{ tool.name }}: {{ tool.description }}
{% endfor %}

{% if user.is_premium %}
NOTE: This is a premium customer. Prioritise their request.
{% endif %}

RULES:
- Always search docs before answering.
- Maximum response length: {{ max_tokens }} tokens.
""")

system_prompt = prompt_template.render(
    tools=[
        {"name": "search_docs", "description": "Search knowledge base"},
        {"name": "create_ticket", "description": "Create support ticket"},
    ],
    user={"name": "Alice", "is_premium": True},
    max_tokens=500
)

Even experienced engineers make prompt engineering mistakes. Here are the most common pitfalls and how to fix them:

Debugging Technique: Prompt Logging

The single most effective debugging technique is logging every prompt and response. When your agent misbehaves, you can inspect exactly what it saw and what it decided:

import json
from datetime import datetime

def log_interaction(system_prompt, messages, response, log_file="agent_log.jsonl"):
    """Log every agent interaction for debugging."""
    entry = {
        "timestamp": datetime.now().isoformat(),
        "system_prompt": system_prompt,
        "messages": messages,
        "response": response,
    }
    with open(log_file, "a") as f:
        f.write(json.dumps(entry) + "\n")

# Usage: call after every API response
log_interaction(system_prompt, messages, response_text)

Debugging Technique: The "Explain Yourself" Test

When your agent makes an unexpected decision, add a temporary instruction to the system prompt asking it to explain its reasoning:

# Temporary debugging addition to system prompt
debug_instruction = """
DEBUG MODE: Before every action, explain:
1. What you understood the user to want
2. Which tools you considered and why you chose this one
3. What you expect the result to be
"""