Prompt Engineering - SustainSys AI Academy

Introduction to Prompt Engineering

Prompt engineering is the art and science of crafting inputs to Large Language Models (LLMs) to elicit desired outputs with high quality and reliability.

A prompt is more than just a question - it's a complete specification including context, examples, instructions, and output format. The difference between mediocre and excellent prompts can mean the difference between useless responses and production-quality results.

Unlike traditional programming with explicit instructions, prompt engineering involves understanding how language models interpret natural language, requiring intuition about model behavior and practical experience with real APIs.

What You'll Learn

Prompt Fundamentals

Understand the anatomy of effective prompts: context, examples, instructions, and output specifications.

Advanced Techniques

Master zero-shot, few-shot, chain-of-thought, tree-of-thought with real code examples.

System Prompts

Build system prompts that define model behavior and role-based AI agents.

Hands-On Code

Write and run real Python code using OpenAI, Anthropic, and LangChain APIs.

Prerequisites

Basic Python knowledge and familiarity with APIs. Understanding how LLMs work is helpful but not required.

Why Prompt Engineering Matters

Prompt engineering bridges the gap between general-purpose AI models and specialized solutions for your specific problem.

The Prompt Determines Everything

Well-engineered

94%

Good prompt

78%

Generic prompt

52%

Why Companies Invest

Cost Efficiency

Well-engineered prompts get better results with fewer tokens.

Reliability

Production systems need consistent, predictable outputs.

Brand Voice

System prompts ensure AI assistants represent your brand.

Speed

Rapid iteration and deployment without training new models.

Reality: Many enterprises now employ dedicated prompt engineers. This role emerged because prompt engineering directly impacts quality, cost, and user satisfaction.

Historical Evolution

Prompt engineering emerged as a distinct discipline driven by rapid LLM advancement.

2018
BERT & GPT

2020
GPT-3

2021
Few-Shot & CoT

2022
ChatGPT

2023
GPT-4

2024+
Multi-modal & Tools

Key Discoveries

In-Context Learning

GPT-3 demonstrated that LMs could learn from prompt examples without weight updates.

Chain-of-Thought

Asking models to reason step-by-step dramatically improves complex task performance.

Instruction Tuning

Fine-tuning on instruction-following revealed that prompting matters enormously.

Core Concepts and Principles

1. The Anatomy of a Prompt

System
Role & behavior

Context
Background info

Examples
Few-shot demos

Instruction
Task description

Format
Output structure

2. Clarity and Specificity

Bad: Write a poem about nature.
Good: Write a 4-line poem about autumn leaves using vivid metaphors.

3. Roles and Personas

Models perform better when given a role. This activates relevant knowledge patterns in the model.

Prompt Structure Deep Dive

The Five-Layer Prompt Structure

Layer 1: System

Sets role and behavior.

Layer 2: Context

Background information and constraints.

Layer 3: Examples

2-3 input-output pairs showing patterns.

Layer 4: Task

Clear description of what you want.

Layer 5: Format

Exact response structure and schema.

System Prompts

The system prompt defines the model's personality, expertise, and constraints. Good components: ROLE, PURPOSE, CONSTRAINTS, STYLE, INSTRUCTIONS.

Examples: Few-Shot Learning

2-3 well-chosen examples often dramatically improve output quality. Choose examples covering common cases, edge cases, and desired style.

Key Components of Effective Prompts

1. Zero-Shot Prompting

Ask the model directly without examples. Works well for tasks the model was extensively trained on.

2. Few-Shot Prompting

Provide 2-3 examples. Usually better than zero-shot because it removes ambiguity.

3. Chain-of-Thought Prompting

Ask the model to reason step-by-step. Dramatically improves performance on reasoning tasks.

The Magic: Adding "Let me think through this step by step:" improves accuracy by 20-50% on reasoning tasks.

4. Structured Output

Request JSON, XML, or Markdown responses. Makes output machine-readable and parseable.

5. Role-Based Prompting

Assign the model an expert role. Activates relevant knowledge patterns.

6. Constraint-Based Prompting

Explicitly state what the model should NOT do. Models respond well to hard constraints.

7. Prompt Chaining

Break complex tasks into multiple prompts. Output of one becomes input to the next.

Implementation Guide

The Iterative Refinement Process

1. Write

2. Test

3. Analyze

4. Hypothesize

5. Refine

Testing Framework

Test Set: 20-50 representative examples
Metric: Define success clearly
Baseline: Record initial score
Iterations: Change one thing at a time
Validation: Test on unseen data

A/B Testing Mindset

Treat prompts like feature experiments. Version them (v1, v2, v3). Keep detailed notes. This scientific approach ensures progress.

Add Examples

If not following pattern, add another example.

Increase Specificity

Replace vague language with precise descriptions.

Change Order

Try different component ordering.

Simplify Language

Remove jargon. Use simple words.

Advanced Prompting Techniques

1. Tree-of-Thought (ToT)

Instead of one reasoning chain, explore multiple paths and select the best one.

Pattern: Ask for 3 different approaches. Evaluate which is best. Execute the best with full detail.

2. Prompt Decomposition

Break complex tasks into multiple simpler subtasks. Each subtask gets its own optimized prompt.

3. Retrieval-Augmented Generation (RAG)

Retrieve relevant context from a knowledge base. Dramatically reduces hallucination.

4. Prompt Ensembling

Run multiple different prompts and aggregate results. Reduces variance and improves robustness.

5. Self-Correction Prompts

Ask the model to generate an answer, review it for errors, then provide a corrected version.

6. Multi-Turn Conversations

Use iterative refinement through conversation. First turn generates draft, second critiques, third improves.

Prompting Technique Comparison

Technique	When to Use	Pros	Cons	Cost
Zero-Shot	Simple tasks	Fast, cheap	Lower accuracy	Lowest
Few-Shot	Most cases	Better accuracy	More tokens	Medium
Chain-of-Thought	Reasoning	Better reasoning	Slower	High
Tree-of-Thought	Complex	Explores paths	Expensive	Very High
RAG	Factual	Grounded	Requires KB	Medium
Self-Correction	High quality	Catches errors	2+ passes	High

Real-World Use Cases

1. Customer Support

System prompt defines brand voice. Few-shot examples show how to handle issues. Chain-of-thought ensures customer context consideration.

2. Content Generation

Marketing teams use prompts to generate posts, email subjects, product descriptions with consistent brand voice.

3. Code Review

System prompt: expert code reviewer. Output: JSON with structured feedback.

4. Data Analysis

Feed data with prompts asking for insights. Chain-of-thought explains reasoning. RAG brings domain knowledge.

5. Research Synthesis

Summarize papers, extract findings, synthesize conclusions with proper citations.

6. Creative Writing

System prompt defines genre, tone, voice. Examples show style. Instructions specify plot.

7. Multi-Step Workflows

Complex tasks decomposed into sequential prompts. Each step's output feeds into the next.

Enterprise-Scale Prompt Engineering

1. Prompt Versioning

Treat prompts like code. Version them, document changes, maintain changelogs.

2. Quality Metrics

Accuracy: % correct
Coverage: % valid outputs
Latency: Response time
Cost: Per request
Safety: Policy compliance

3. Cost Optimization

Token Reduction

Remove unnecessary context. Use abbreviations. Compress examples.

Cheaper Models

Smaller models + great prompts beat large models + mediocre prompts.

Caching

Use prompt caching. Pay once, use many times.

Batch Processing

Send multiple requests in batch.

4. Compliance and Safety

Use system prompts to enforce safety policies. Explicitly forbid harmful content, personal data processing, etc.

5. Multi-Model Orchestration

Route tasks to different models based on complexity and cost constraints. Optimize prompts per model.

Common Mistakes and How to Avoid Them

Mistake 1: No System Prompt

Adding a system prompt dramatically improves quality (usually 10-30%).

Mistake 2: Multiple Questions at Once

Models get confused. Break into sequential prompts.

Mistake 3: Insufficient Examples

Using zero-shot when few-shot would work better. Add 2-3 examples and accuracy often jumps 10-20%.

Mistake 4: Vague Output Format

Saying "give me JSON" is too vague. Specify exact schema, required fields, data types.

Mistake 5: No Error Handling

Tell the model what to do if the task cannot be completed.

Mistake 6: Forgetting Context

Models cannot read minds. If you need specific knowledge, provide it.

Mistake 7: Not Testing on Real Data

Prompts work on artificial examples but fail on messy real data.

Mistake 8: Treating Prompts as Write-Once

Prompts require iteration. Your first draft is rarely optimal.

Prompt Engineering Best Practices

1. Document Your Prompts

Purpose: What is this prompt for?
Performance: Accuracy metrics
Assumptions: Required conditions
Known Issues: Edge case failures
Examples: Real inputs and outputs

2. Create Test Sets

Build a test set of 20-50 diverse examples. Run prompts against it after each change.

3. Version Your Prompts

Use semantic versioning: v1.0.0 → v1.1.0 → v2.0.0.

4. A/B Test in Production

Before full deployment, run new prompts on traffic subset. Compare metrics.

5. Document Failures

Keep a log of failure cases. This is your improvement roadmap.

6. Use Prompt Compression

Remove redundancy. Reduces cost and sometimes improves latency.

7. Implement Monitoring

Track metrics in production. Alert when quality degrades.

8. Share Knowledge

Maintain a prompt library. Document what works and why.

Advanced Insights and Theory

1. Token Probability

Language models generate responses token-by-token. Better prompts increase probability gaps between correct and incorrect answers.

2. In-Context Learning Theory

Models perform implicit gradient descent on examples. This explains why example quality and order matter.

3. Prompt Injection and Security

Malicious users can inject prompts to override instructions. If user input is in the prompt, validate carefully.

4. Model-Specific Quirks

GPT-4

Excellent reasoning, loves detail, responds to complex instructions

Claude

Values safety, prefers natural conversation, great at long-form

Gemini

Multimodal strengths, good at facts, excellent at coding

Llama

Open source, can be fine-tuned, smaller models need simpler prompts

5. The Scaling Law of Prompts

Better prompts help smaller models perform like larger models. A 7B model with excellent prompts can match a 70B model with mediocre prompts.

6. Prompt Robustness

Good prompts work across input variations. Bad prompts are brittle. Robustness comes from clarity and generality.

Code Examples: Production-Ready Patterns

Example 1: Zero-Shot vs Few-Shot

Python — Zero-Shot vs Few-Shot
from anthropic import Anthropic
client = Anthropic()

# Zero-shot
zero = "Classify sentiment: \"I love this!\" Answer:"
r1 = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=50,
    messages=[{"role": "user", "content": zero}])
print("Zero-shot:", r1.content[0].text)

# Few-shot
few = """Classify sentiment.
Examples: \"I love!\" -> pos, \"Bad\" -> neg
Text: \"Best!\" Answer:"""
r2 = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=50,
    messages=[{"role": "user", "content": few}])
print("Few-shot:", r2.content[0].text)

Example 2: Chain-of-Thought

Python — Chain-of-Thought
from anthropic import Anthropic
client = Anthropic()

cot = """Solve step-by-step:
Sarah has 3 boxes with 4 apples each.
She gives 5 away. How many remain?

Step-by-step:"""

response = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=200,
    messages=[{"role": "user", "content": cot}])
print(response.content[0].text)

Example 3: Structured JSON Output

Python — JSON Extraction
from anthropic import Anthropic
import json
client = Anthropic()

prompt = """Extract entities from text:
Text: Apple Inc. was founded by Steve Jobs 
in Cupertino, California in 1976.

Return JSON: companies, locations, year_founded
JSON:"""

response = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=500,
    messages=[{"role": "user", "content": prompt}])

try:
    data = json.loads(response.content[0].text)
    print(json.dumps(data, indent=2))
except:
    print("Response:", response.content[0].text)

Example 4: System Prompts with Role

Python — Role-Based System Prompt
from anthropic import Anthropic
client = Anthropic()

system = """You are an expert Python code reviewer.
Find bugs, security issues, improvements.
Be constructive. Format as JSON."""

user = """Review this code:
def process(data):
    for item in data:
        if item['age'] > 0:
            print(item)

Provide: bugs, security_issues, improvements"""

response = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=500,
    system=system,
    messages=[{"role": "user", "content": user}])
print(response.content[0].text)

Example 5: Prompt Chaining

Python — Multi-Step Chain
from anthropic import Anthropic
client = Anthropic()

text = "ML enables computers to improve without programming. Deep learning uses neural networks."

# Step 1: Summarize
summary = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=200,
    messages=[{"role": "user", "content": f"Summarize in 1-2 sentences: {text}"}]
).content[0].text

# Step 2: Extract keywords  
keywords = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=200,
    messages=[{"role": "user", "content": f"Extract 3 keywords from: {summary}"}]
).content[0].text

print("Summary:", summary)
print("Keywords:", keywords)

Example 6: A/B Testing Prompts

Python — Prompt A/B Testing
from anthropic import Anthropic
import json
client = Anthropic()

tests = ["I love this!", "Has issues", "Worst ever"]

prompt_a = "Classify sentiment: "{text}" Answer:"
prompt_b = """Classify as pos/neg/neutral.
Examples: \"Love!\" -> pos, \"Bad\" -> neg
Text: "{text}" Answer:"""

results = {"A": [], "B": []}

for text in tests:
    r_a = client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=50,
        messages=[{"role": "user", "content": prompt_a.format(text=text)}]
    )
    results["A"].append(r_a.content[0].text.strip())
    
    r_b = client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=50,
        messages=[{"role": "user", "content": prompt_b.format(text=text)}]
    )
    results["B"].append(r_b.content[0].text.strip())

print(json.dumps(results, indent=2))

Hands-On Exercises

Exercise 1: Zero-Shot to Few-Shot

Simple to Sophisticated

Start with zero-shot for a classification task. Test on 5 examples. Then add 2-3 examples for few-shot. Measure improvement.

Exercise 2: Chain-of-Thought

Reasoning

Find a complex task. Write prompt without CoT, note errors. Add step-by-step reasoning. Did accuracy improve?

Exercise 3: Structured Output

JSON Extraction

Extract info from text. Define JSON schema. Write prompt producing valid JSON. Validate programmatically.

Exercise 4: Prompt Optimization

Iterative Refinement

Write v1 prompt. Test on 10 examples. Identify 3 failures. Hypothesize fixes. Write v2-v4. Track improvements.

Exercise 5: System Prompts

Personality

Create two system prompts - formal and casual. Test on same queries. How does it change tone and quality?

Exercise 6: Prompt Chaining

Multi-Step

Design 3-step: extract → analyze → recommend. Each has optimized prompt. Run real data through. Where do errors accumulate?

Interview Questions for Prompt Engineers

Behavioral Questions

Tell us about a time you optimized a prompt. ▼

Good answer demonstrates: Initial assessment and metrics, systematic testing, root cause analysis, specific changes, quantified improvement. Example: Customer service prompt went from 72% to 89% by adding constraint rules, edge case examples, and structured JSON output.

How do you handle a prompt that fails on specific cases? ▼

Good answer shows: Collect failures, find patterns, understand root cause, test hypotheses, measure improvement. When extraction failed on complex sentences, I added complex examples and made schema more explicit.

Describe balancing cost and quality. ▼

Good answer acknowledges trade-offs and shows data-driven decisions. Example: Could save 40% tokens by removing examples, but accuracy dropped 15%. Kept examples but compressed 30%, achieving 12% savings with 1% accuracy loss.

Technical Questions

When would you use few-shot instead of zero-shot? ▼

Few-shot when accuracy is critical, output format is specific, edge cases exist, or you want to establish patterns. Zero-shot for simple tasks. Few-shot adds cost but improves reliability.

How do you debug inconsistent results? ▼

Good approach: Test 20 examples for variance patterns, identify failing input types, add constraints or examples, use temperature=0 for debugging, re-test systematically.

What is the relationship between prompt quality and model size? ▼

Better prompts help smaller models perform like larger ones. A 7B model with excellent prompts beats 70B model with mediocre prompts. This is how you leverage expensive models.

How would you handle prompt injection? ▼

Good answer: Validate user input, use system + user message separation, never concatenate directly, use structured APIs, validate outputs, monitor for patterns.

Frequently Asked Questions

Do I need to know how LLMs work? ▼

Not in depth. Understanding basics helps: models predict next token, they have context windows, they hallucinate, they respond to structure.

What is the best way to learn? ▼

Practice on real tasks. Start simple, measure, iterate. Join communities, read case studies, experiment with models. The field changes fast.

Is prompt engineering a real career? ▼

Yes. Companies hire dedicated prompt engineers. Salaries competitive (150k-250k+ in tech hubs). Often combined with other skills.

Can I use same prompt across models? ▼

Usually not optimally. Each model has quirks. GPT-4 vs Claude vs Llama differ. Well-written structured prompts work across models. Test each.

How long should prompts be? ▼

As long as needed, no longer. Add examples if they improve. Typical: 100-500 tokens. Long prompts (1000+) get expensive. Use RAG instead.

Should I use temperature=0? ▼

Production consistency: use temperature=0 or low (0.2). Creative: higher (0.8-1.0). Most tasks: 0.5-0.7. Test both and measure.

How do I measure quality without manual review? ▼

Define metrics: accuracy, format validity, length constraints, safety compliance, latency. Combine automated metrics with sample manual review.

Can I fine-tune instead? ▼

Fine-tuning is expensive and slow. Prompt engineering is faster and cheaper. Try prompts first. Only fine-tune if you have lots of data.

What if model doesn't follow instructions? ▼

Add explicit constraints, use structured output, provide examples, rephrase for clarity. If nothing works, try more capable model or add validation layers.

How do I handle hallucinations? ▼

Use RAG to ground in real data, add explicit constraints, instruct to cite sources, ask to acknowledge uncertainty, test on known edge cases.

Summary: Key Takeaways

Structure Matters

Every prompt needs 5 layers: system, context, examples, instruction, format. Skip any at your peril.

Test Everything

Write, test on real data, measure, iterate. First drafts are rarely optimal.

Examples Are Powerful

Few-shot beats zero-shot by 20-50%. Choose examples strategically.

Chain-of-Thought Helps

Step-by-step reasoning dramatically improves complex tasks. Token cost is worth it.

Know Your Constraints

Understand context window, cost, SLAs. Design within constraints. Use RAG for facts.

Iterate Systematically

Change one thing at a time. Measure. Keep if better. Document. Build knowledge.

Most Important: Prompt engineering improves with deliberate practice. Start simple, measure everything, iterate based on data. Treat it scientifically, not artistically.

Your Action Items

Pick a task you care about
Write a basic prompt
Create a test set (10-20 examples)
Measure current performance
Improve the prompt
Re-test and measure
Document what worked
Share findings

Resources and Further Learning

Research Papers

Chain-of-Thought Prompting (Wei et al., 2022)
Tree of Thoughts (Yao et al., 2023)
Retrieval-Augmented Generation (Lewis et al., 2021)
Prompt Engineering vs. Learning (OpenAI)

Online Learning

DeepLearning.AI - Free courses
OpenAI Cookbook - Real examples
Anthropic Docs - System prompts
LangChain Documentation - Prompt management

Tools and Platforms

LangSmith - Versioning, testing, monitoring
OpenAI Playground - Free GPT experimentation
Claude.ai - Direct Claude access
Weights & Biases - Experiment tracking

Communities

r/PromptEngineering - Reddit discussions
OpenAI Discord - Official community
Anthropic Discord - Claude community
AI Engineer Summit - Conferences

Stay Current

Follow researchers, subscribe to newsletters, read arxiv papers. Best prompt engineers stay curious and experiment constantly.