Building a Local AI Agent
From Scratch

What is an AI Agent?

Most people's first experience with AI is a chatbot — you ask a question, it answers. That's a single LLM call. An AI agent is something fundamentally different: it's an LLM that can act, not just respond.

The critical difference is the loop. A plain LLM call is one-shot — in, out, done. An agent loops: it thinks, acts, observes what happened, thinks again, acts again — until the task is complete.

A Live Example

Ask a plain LLM: "What's the weather in Tokyo and should I pack a coat?" — it will fabricate an answer or admit it can't access real-time data.

Ask an agent the same question:

What Makes a Good Agent?

• A clear system prompt defining role, tools, and output format
• A reliable decision loop that terminates gracefully
• Well-described tools so the LLM knows when to use them
• Proper memory so it doesn't forget context mid-task
• Solid error handling so one tool failure doesn't crash everything

What is MCP?

MCP stands for Model Context Protocol — an open standard created by Anthropic that defines how AI applications discover and use tools hosted on external servers.

Think of it like USB. Before USB, every device needed its own special cable. USB standardised the connection so any device works with any port. MCP does the same for AI tools and agents.

The Problem MCP Solves

Imagine building 5 different AI applications — each needing weather data, calendar access, and search. Without MCP, you copy tools into every project. With MCP, tools live on one server and every application connects to it.

The Three MCP Endpoints

Every MCP server exposes three standard endpoints — this is the protocol:

# 1. Health check — "Are you alive?"
GET  /health
# → {"status": "ok", "server": "My Tool Server", "version": "2.0"}

# 2. Tool discovery — "What can you do?"
GET  /tools
# → {"tools": [{"name": "calculate", "description": "..."}], "count": 7}

# 3. Tool execution — "Do this thing"
POST /tools/call
# Body:   {"tool": "get_weather", "input": "London"}
# → {"result": "Weather in London: 14°C, Partly cloudy", "success": true}

Client / Server Architecture

  ┌──────────────────────────────────┐
  │         MCP SERVER               │
  │      mcp_server.py               │
  │      Flask on :5001              │
  │                                  │
  │  Tools hosted here:              │
  │  • calculate()                   │
  │  • get_weather()                 │
  │  • search_wikipedia()            │
  │  • save_note() / read_notes()    │
  │  • get_current_time()            │
  │  • summarise_text()              │
  └───────────────┬──────────────────┘
                  │  HTTP / JSON
                  │  POST /tools/call
  ┌───────────────▼──────────────────┐
  │         MCP CLIENT               │
  │      mcp_client.py               │
  │                                  │
  │  connect()       health check    │
  │  discover_tools()  list tools    │
  │  call_tool()     run a tool      │
  │                                  │
  │  Used by agent.py to act         │
  └──────────────────────────────────┘
      

MCP Server — The Core Pattern

from flask import Flask, request, jsonify

app = Flask(__name__)

# 1. Define tools as plain Python functions
def calculate(expression: str) -> str:
    return str(eval(expression))

# 2. Register in a dictionary
TOOLS = {
    "calculate": {"function": calculate, "description": "Evaluate a math expression"}
}

# 3. Expose via HTTP
@app.route("/tools", methods=["GET"])
def list_tools():
    return jsonify({"tools": [{"name": k, "description": v["description"]} for k,v in TOOLS.items()]})

@app.route("/tools/call", methods=["POST"])
def call_tool():
    d = request.get_json()
    result = TOOLS[d["tool"]]["function"](d["input"])
    return jsonify({"result": result, "success": True})

System Architecture

Here is the complete picture of the system and how every file connects:

  ┌──────────────────────────────────────────────────────────┐
  │                  RESEARCH ASSISTANT                      │
  │                                                          │
  │  ┌──────────┐     ┌───────────────────────────────┐     │
  │  │ main.py  │────►│        agent.py               │     │
  │  │ CLI loop │     │   Orchestrator · ReAct loop   │     │
  │  └──────────┘     └──────┬───────────┬────────────┘     │
  │                          │           │                  │
  │               ┌──────────┘           └────────────┐     │
  │               │                                   │     │
  │          ┌────▼────┐  ┌──────────┐  ┌────────────▼─┐   │
  │          │ llm.py  │  │memory.py │  │mcp_client.py │   │
  │          │ Ollama  │  │short +   │  │HTTP client   │   │
  │          │ :11434  │  │long term │  └──────┬───────┘   │
  │          └─────────┘  └──────────┘         │           │
  │                                            │ HTTP       │
  │                                            ▼           │
  │                              ┌─────────────────────┐   │
  │                              │   mcp_server.py     │   │
  │                              │   Flask  :5001      │   │
  │                              ├─────────────────────┤   │
  │                              │ calculate()         │   │
  │                              │ get_weather()       │   │
  │                              │ search_wikipedia()  │   │
  │                              │ save_note()         │   │
  │                              │ read_notes()        │   │
  │                              │ get_current_time()  │   │
  │                              │ summarise_text()    │   │
  │                              └─────────────────────┘   │
  └──────────────────────────────────────────────────────────┘
      

Data Flow for a Single Query

What happens when you type "What's the weather in Paris?":

Tech Stack

The 9 Build Stages

Each stage adds exactly one new concept. Never skip — each builds directly on the last.

The ReAct Pattern

ReAct stands for Reason + Act. It's the most widely-used pattern for AI agents. ChatGPT plugins, Claude tools, and LangChain agents all use it under the hood. The core idea: force the LLM to show its reasoning before every action — making behaviour transparent and debuggable.

The Output Format

# When the agent needs a tool:
Thought: I need current weather data for Tokyo. I'll use get_weather.
Action:  get_weather
Input:   Tokyo

# After seeing the result, if another tool is needed:
Thought: Got the weather. Now I'll save it to notes as requested.
Action:  save_note
Input:   Tokyo weather: 8°C, partly cloudy

# When the agent has a final answer:
Thought: I have all information needed to answer fully.
Answer:  The weather in Tokyo is 8°C and partly cloudy. Pack a coat.

Parsing the Response

Without a strict format, the LLM responds any way it wants — prose, JSON, bullet points. We need to parse it with code, so we enforce a machine-readable structure and extract the action with regex:

import re

def parse_response(response: str) -> dict:
    # Look for "Action: tool_name"
    action_match = re.search(r"Action:\s*(\w+)", response)
    input_match  = re.search(r"Input:\s*(.+?)(?:\n|$)", response)

    if action_match:
        return {
            "type":  "action",
            "tool":  action_match.group(1).strip(),
            "input": input_match.group(1).strip() if input_match else ""
        }

    answer_match = re.search(r"Answer:\s*(.+)", response, re.DOTALL)
    if answer_match:
        return {"type": "answer", "content": answer_match.group(1).strip()}

    return {"type": "answer", "content": response.strip()}

The Full Agent Loop

def run_agent(user_query: str) -> str:
    context = user_query
    step = 0

    while step < MAX_STEPS:     # safety limit
        step += 1

        # Ask the LLM: what should I do next?
        response = chat(context, system_prompt=build_system_prompt())
        parsed   = parse_response(response)

        if parsed["type"] == "answer":
            return parsed["content"]   # ← done!

        elif parsed["type"] == "action":
            # Call tool via MCP, feed result back in
            result  = mcp.call_tool(parsed["tool"], parsed["input"])
            context = f"Observation: {result}\n\nContinue."
            # ↑ loop again with the new context

    return "Reached maximum steps."

Tools & Tool Calling

A tool is a Python function the agent can choose to call. Tools give capabilities beyond what the LLM knows — real-time data, computation, file access, external APIs. The agent doesn't call tools randomly; it reads their descriptions and uses language understanding to match the right tool to the task.

Why Tool Descriptions Matter

The Weather Tool — Deep Dive

The best example of a realistic tool — calls an external API, parses JSON, handles errors, returns clean formatted text:

def get_weather(city: str) -> str:
    try:
        # Step 1: City name → coordinates (free geocoding API)
        geo = requests.get(
            "https://geocoding-api.open-meteo.com/v1/search",
            params={"name": city, "count": 1},
            timeout=10
        ).json()["results"][0]
        lat, lon = geo["latitude"], geo["longitude"]

        # Step 2: Live weather data from Open-Meteo (free, no key)
        data = requests.get(
            "https://api.open-meteo.com/v1/forecast",
            params={
                "latitude": lat, "longitude": lon,
                "current": ["temperature_2m", "weather_code"],
                "timezone": "auto"
            }, timeout=10
        ).json()

        current = data["current"]
        return f"Weather in {city}: {current['temperature_2m']}°C"

    except Exception as e:
        return f"Error: {str(e)}"

Memory Systems

Without memory, every message starts fresh. Say "My name is James" — next message ask "What is my name?" and the agent has no idea. We solve this with two distinct memory types.

Short-Term Memory — Conversation History

Lives in RAM. Holds every message in the current session. Disappears when the program closes.

class ShortTermMemory:
    def __init__(self, max_messages: int = 20):
        self.messages = []
        self.max_messages = max_messages

    def add(self, role: str, content: str):
        self.messages.append({"role": role, "content": content})
        # Trim oldest messages when we hit the limit
        if len(self.messages) > self.max_messages:
            self.messages = self.messages[-self.max_messages:]

    def get_history(self) -> list:
        return [{"role": m["role"], "content": m["content"]} for m in self.messages]

Long-Term Memory — Persistent Facts

Saved to a JSON file on disk. Survives restarts. Injected into the system prompt every session so the agent "knows" it from the start.

class LongTermMemory:
    def __init__(self, filepath: str = "memory.json"):
        self.facts = self._load()  # load from disk on startup

    def remember(self, key: str, value: str):
        self.facts[key] = {"value": value}
        self._save()  # write to disk immediately

    def get_all_facts(self) -> str:
        # Injected into system prompt every session
        return "\n".join(f"- {k}: {v['value']}" for k,v in self.facts.items())

What is a Context Window?

Every LLM has a context window — the maximum text it can see at once. For llama3.1:8b this is ~128,000 tokens. Everything — system prompt, conversation history, tool results — must fit within this limit. When conversations grow long, we trim oldest messages with messages[-max_messages:].

Orchestration

Orchestration is the coordination layer — the code that manages the overall flow. It's not a single feature; it's the glue that holds the whole system together. In this project, agent.py is the orchestrator.

It coordinates all five subsystems in the right order:

• Receives user input from main.py
• Assembles system prompt with dynamic tool list + user facts
• Calls the LLM via llm.py and parses the response
• Calls tools via mcp_client.py when needed
• Updates conversation history in memory.py
• Decides when the task is complete and returns the answer

The System Prompt — Heart of the Orchestrator

Rebuilt fresh for every query — dynamically including the live tool list and current facts:

def build_system_prompt() -> str:
    tool_descriptions = mcp.get_tool_descriptions()  # live from MCP server
    long_term_facts   = long_term.get_all_facts()       # from memory.json

    return f"""You are a helpful research assistant.

WHAT YOU KNOW ABOUT THE USER:
{long_term_facts}

TOOLS AVAILABLE:
{tool_descriptions}

Use this EXACT format when using a tool:
Thought: [reason]
Action:  [tool_name]
Input:   [input]

When done:
Thought: [reason]
Answer:  [complete answer]
"""

Key Terms Glossary

Every term you'll encounter in this project and the broader AI engineering field — explained with examples from the code you'll build.

Step-by-Step Build Guide

Prerequisites

• macOS or Linux (Windows users: use WSL2)
• Python 3.10 or higher — check with python3 --version
• Ollama installed with at least one model pulled
• ~6 GB free disk space for the LLM model

Stage 1 — Environment Setup

# Check Python version (need 3.10+)
python3 --version

# Create project folder and navigate into it
mkdir research-assistant && cd research-assistant

# Create and activate virtual environment
python3 -m venv .venv
source .venv/bin/activate   # you'll see (.venv) in your prompt

# Install all dependencies
pip install requests python-dotenv flask wikipedia-api

# Create folder structure
mkdir src logs
touch src/__init__.py .env .gitignore requirements.txt main.py

Stage 2 — First LLM Call

import requests

OLLAMA_URL = "http://localhost:11434/api/chat"
MODEL      = "llama3.1:8b"

def chat(user_message: str, system_prompt: str = None) -> str:
    messages = []
    if system_prompt:
        messages.append({"role": "system", "content": system_prompt})
    messages.append({"role": "user", "content": user_message})

    response = requests.post(
        OLLAMA_URL,
        json={"model": MODEL, "messages": messages, "stream": False},
        timeout=60
    )
    return response.json()["message"]["content"]

if __name__ == "__main__":
    print(chat("What is the capital of France?"))
    # → "The capital of France is Paris."

Running the Complete Final Project

The final project requires two terminal windows running simultaneously — this is the MCP client/server pattern in action:

cd ~/research-assistant
source .venv/bin/activate
python3 src/mcp_server.py

# Expected output:
# 🔌 MCP Tool Server v2.0
# Tools available : 7
# Health : http://localhost:5001/health
# Tools  : http://localhost:5001/tools

cd ~/research-assistant
source .venv/bin/activate
python3 main.py

# Expected output:
# 🤖 Research Assistant Agent
# ✅ Connected to MCP server: Research Assistant MCP Server
# 🔧 Discovered 7 tools from MCP server
# Ready! Type /help for commands or just ask me anything.

Test Queries — Try These in Order

# 1. Tests tool calling (calculator)
What is 2 to the power of 16?

# 2. Tests external API (Open-Meteo)
What's the weather in Tokyo?

# 3. Tests Wikipedia search tool
Search Wikipedia for the James Webb Space Telescope

# 4. Tests multi-tool chaining
What's the weather in Paris and save a note about it

# 5. Tests short-term memory
My name is [your name]
What is my name?

# 6. Tests long-term memory (survives restarts)
/remember city=London
quit → restart → What is the weather where I live?

# 7. Tests command interface
/tools
/memory
/facts

The .env Configuration File

# LLM
OLLAMA_MODEL=llama3.1:8b
OLLAMA_URL=http://localhost:11434/api/chat

# MCP Server
MCP_SERVER_HOST=localhost
MCP_SERVER_PORT=5001

# Agent
AGENT_MAX_STEPS=5
AGENT_MEMORY_LIMIT=20

# Logging — DEBUG | INFO | WARNING | ERROR
LOG_LEVEL=INFO

Where to Go Next

Immediate Upgrades to This Project

• Add a web search tool using the free DuckDuckGo API
• Add a read_file tool so the agent can read your local documents
• Add streaming output so responses appear word-by-word
• Try swapping to OpenAI or Anthropic — only llm.py changes

Natural Next Steps in Learning

• LangChain / LangGraph — you'll now understand what they abstract, because you built the patterns by hand
• LlamaIndex — connecting agents to large document collections with semantic search
• ChromaDB or Qdrant — vector databases for proper semantic memory at scale
• FastAPI — replace Flask with a faster, modern async server
• Anthropic's official MCP SDK — the production Python library for MCP
• Multi-agent systems — multiple agents collaborating, with one orchestrating others

What You Now Understand