A fully local agent with tools, in about 40 lines: Ollama plus Pydantic AI

Step-by-Step Breakdown

1. Size the model to your memory first

   Dedicated GPU: check VRAM (the demo machine is a 4090 with 24 GB). Mac or unified memory: check total RAM, typically 16–128 GB on modern machines. Then choose the best model whose file size fits. This is the step that determines whether the rest works.

2. Connect to Ollama's local inference server

   Ollama exposes an inference server on your own device at localhost:11434. In Pydantic AI you construct the Ollama model with the name exactly as ollama list reports it, and point the provider at that address. Nothing leaves the machine.

3. Write tools as ordinary Python functions

   Tools are just functions. The demo defines getting the current time, evaluating a calculation, and saving and reading a note — the last two giving the agent real file access on the local machine. Pydantic infers the types from the function signatures, so there is no separate schema to maintain.

4. Assemble the agent

   An agent is the model, the list of tool functions, and a system prompt. That is the whole definition — the tools are passed by name, exactly as defined above.

5. Add a conversation loop

   A main function keeping a message history, a while loop reading user input, a break on quit or exit, and otherwise a request to the agent with the accumulated history. Keeping the history is what makes it a conversation rather than a series of unrelated calls.

Commands & Code Shown

ollama

ollama

Purpose: Run it bare to confirm Ollama installed correctly. Any output means you are ready to pull a model.

When to use: First, after installing Ollama from its website. Skipping this is how you end up debugging Python for a problem that is not in Python.

ollama pull qwen3.5:4b

ollama pull qwen3.5:4b

Purpose: Downloads a local model. The recommended default in the video is Qwen 3.5 at 4B parameters; 0.8B and 2B variants exist for tighter memory, as do the older Qwen 3 models at 0.6B, 1.7B and 4B.

When to use: Once you have checked the model's file size against your available VRAM or unified memory. You can change models later.

ollama list

ollama list

Purpose: Shows every model available locally — which is also the list of names valid in your code.

When to use: After pulling, and whenever your code reports a model it cannot find.

ollama run qwen3.5:2b

ollama run qwen3.5:2b

Purpose: Opens a chat with the model in the terminal so you can judge whether it responds fast enough on your hardware before writing any code.

When to use: Before building. The rule given: if the response takes forever, drop to a smaller model. Loading takes a moment on first run regardless.

pip install pydantic-ai

pip install pydantic-ai

Purpose: Installs the agent framework used to bind the model, the tools and the system prompt together.

When to use: Once per environment. Installing with uv works equally well.

Gotchas & Caveats

• The presenter promotes a free community mid-video and links the code there. The technical walkthrough stands on its own and the tools used are all free and open.
• Local model quality tracks size, and small models handle tool-calling noticeably less reliably than frontier models. This is a working setup to learn on, not a replacement for a hosted agent on hard tasks.
• Giving the agent file read and write on your own machine — as the note tools do — is worth pointing at a scratch directory rather than your home folder.
• Model names and sizes move quickly; check `ollama list` and current model cards rather than copying names from a video.

Key Takeaways

• Everything runs locally. No API key, no per-token cost, and nothing sent off the machine — which also makes it the cheapest possible place to experiment with tool-calling.
• Model choice is a memory constraint, not a preference. Pick the largest model that fits; a smaller model that responds quickly beats a larger one that does not.
• Tools are plain functions with inferred types — the lowest-ceremony tool definition of any framework in common use.
• Test the model in the terminal before writing code. `ollama run` tells you in thirty seconds whether your hardware can carry the model you chose.
• The agent definition is genuinely three things: model, tools, system prompt. Useful as a mental model even if you use a different framework.
• The framework is swappable — the structure shown (provider → model → tools → agent → loop) is the same in any of them.