Lessons · Coding · intermediate · Tech With Tim

How to Build a Local AI Agent With Python (Ollama, LangChain & RAG)

The video “How to Build a Local AI Agent With Python (Ollama, LangChain & RAG)” by Tech With Tim as 7 lessons, with the idea explained, the code from the screen, steps to try in the practice terminal, practice problems with solutions and a quiz that leads back to the moment.

7lessons
8pieces of code
17exercises
21questions
28min of video
Open the lessons →Lesson 1 is free · Beyond Pass opens all
How to Build a Local AI Agent With Python (Ollama, LangChain & RAG), video by Tech With Tim▶ 28:08 · Tech With Tim
1234567
0:007 lessons across the whole video28:08

You will be able to

  • Create an isolated Python virtual environment and install LangChain, Ollama, and Chroma dependencies
  • Run a local LLM through Ollama and invoke it from Python with a LangChain prompt chain
  • Build an interactive question loop that sends user input through a model chain
  • Embed CSV rows as documents into a persistent Chroma vector store using a local embedding model
  • Implement retrieval-augmented generation by feeding retrieved documents into the prompt before the model answers

Good to know first

  • Basic Python syntax, including functions, loops, and dictionaries
  • Comfort using a terminal and command line
  • Familiarity with VS Code or another code editor
  • Basic understanding of CSV files

Syllabus

7 lessons

  1. 1

    Demo of the finished local review assistant Free

    0:00–2:012 min of video

    You get a first look at the finished app: a terminal program that answers questions by reading a CSV of restaurant reviews. It runs entirely on your own computer, with no paid cloud account.

    2 code2 build steps1 exercise3 questions
  2. 2

    Set up the project folder and virtual environment Beyond Pass

    2:01–4:333 min of video

    You will create a project folder, make an isolated Python virtual environment inside it, activate it, and install the libraries the agent needs. Setting this up first keeps your project's packages separate from everything else on your computer.

    1 code6 build steps2 exercises3 questions
  3. 3

    Install Ollama and pull a chat model and an embedding model Beyond Pass

    4:33–7:033 min of video

    You install Ollama, which runs AI models on your own computer, and then download two models: a chat model (llama3.2) and an embedding model (mxbai-embed-large). You confirm both are available before writing any code.

    1 code4 build steps2 exercises3 questions
  4. 4

    Invoke a local model with a LangChain prompt chain Beyond Pass

    7:03–13:357 min of video

    You will connect a Python script to a model running on your own computer through Ollama, using LangChain. You will write a prompt template with placeholders and join it to the model into a chain you can invoke.

    1 code6 build steps3 exercises3 questions
  5. 5

    Add an interactive question loop Beyond Pass

    13:35–14:501 min of video

    You will wrap the chain in a while True loop so the user can keep asking questions until they type q. You will learn how input(), break and chain.invoke() work together to make a simple question-and-answer program.

    1 code6 build steps3 exercises3 questions
  6. 6

    Load the CSV and build a persistent Chroma vector store Beyond Pass

    14:50–23:259 min of video

    You will turn each row of a restaurant-reviews CSV into a Document, convert the text into vectors with a local Ollama embedding model, and store them in a persistent Chroma database that is built only once. You will also see why IDs, metadata and a "does the database already exist?" check matter.

    1 code4 build steps3 exercises3 questions
  7. 7

    Create a retriever and wire RAG into the chain Beyond Pass

    23:25–28:085 min of video

    You turn the Chroma vector store into a retriever that fetches the most relevant reviews for a question. You then pass those reviews into the prompt so the local LLM answers using your own data.

    1 code5 build steps3 exercises3 questions

Free preview · Lesson 1 in full

Demo of the finished local review assistant

You get a first look at the finished app: a terminal program that answers questions by reading a CSV of restaurant reviews. It runs entirely on your own computer, with no paid cloud account.

0:00 to 2:01 in the video

Practice terminal

Type the lesson's commands, break things, and see why.

Scratchpad

Draw and write next to the video.

The idea

This chapter is a preview. Before any code is written, the video shows the finished product so you know what you are building. The app is a small Python program run from the command prompt. It asks you a question, looks through a file of fake pizza restaurant reviews, and answers using what it found.

The data lives in a CSV file called realistic_restaurant_reviews.csv. A CSV is a plain text table where each line is a row and commas separate the columns. Here the columns are Title, Date, Rating and Review. A review might read 'Best pizza in town' with a date, a rating out of five, and a long text describing the pizza.

The idea behind the app is retrieval augmented generation, often shortened to RAG. The program first retrieves the reviews that are relevant to your question from the file, then gives those reviews to a language model, which writes the answer. The model does not have to know anything about this restaurant in advance; it only sees the pieces the search found.

The tools are all free and run locally. The video uses Ollama to run a language model on your machine, LangChain to connect the pieces, and ChromaDB as a vector database that finds relevant reviews. No OpenAI account or cloud service is needed.

In the demo, the user asks 'how is the quality of the pizza?'. The agent pulls matching reviews and writes a conclusion that notes the answer is hard to score from the reviews alone. Then the user asks 'are there vegan options?' and the agent points to a vegan pizza mentioned in a review, quoting the review's metadata. The answer depends on the reviews in the file, so it is only as good as the data.

Key points

  • The finished app answers questions about restaurant reviews from a local CSV file. 0:00
  • RAG means retrieving relevant information from a file and giving it to the model before it answers. 0:12
  • Everything is free and runs on your own computer: no OpenAI or cloud account is needed. 0:24
  • The tools are Ollama (local model), LangChain (connects the pieces) and ChromaDB (vector search database). 0:36
  • The app is run from the command prompt with python main.py and loops asking questions until you type q. 0:59

Code from the screen

realistic_restaurant_reviews.csv (first lines shown on screen)text
Title,Date,Rating,Review
Best pizza in town,2024-03-15,5,"The crust was perfectly crispy on the outside and chewy inside. Their signature pepperoni pizza had the perfect ratio of sauce to cheese, and the pepperoni curled up into little cups of deliciousness. Will definitely be back!"
Disappointed with service,2024-02-20,2,"While the pizza itself was decent, we waited over an hour for delivery despite being told it would be 30 minutes. When it finally arrived, it was barely warm. The flavors were good but the experience ruined it."

Open the file in any text editor to see the rows; no code needs to run for this chapter.

Running the apptext
C:\Users\User\Desktop\Local Ai Agent>python main.py

--------------------
Ask your question here (q to quit):

In a terminal inside your project folder, type python main.py and press Enter.

Build along in the practice terminal

  1. Open a terminal and move into your project folder (the video uses a folder called Local Ai Agent on the Desktop). 
    $ cd Desktop
C:\Users\User>cd Desktop
C:\Users\User\Desktop>
    You should see: The prompt now shows the Desktop folder.
  2. Run the app with python main.py. The video shows this as a preview, so you may not have the file yet; this step shows what the finished program looks like when it starts. 
    $ python main.py
--------------------
Ask your question here (q to quit):
    You should see: The program prints a line of dashes and waits for your question.

Break it on purpose

Practice terminal
$ python main.py
python: can't open file 'main.py': [Errno 2] No such file or directory
Why: Python can only run a file that exists in the folder you are in. Check that you cd-ed into the project folder where main.py was saved.
Practice terminal
$ cd Local Ai Agent
The system cannot find the path specified.
Why: Folder names with spaces must be wrapped in quotes, so use cd "Local Ai Agent".

Watch out

  • The app is not a general chatbot: it answers from the reviews in the file, so questions the reviews do not cover get weak answers. 0:24
  • Local models run on your own hardware, so responses can be slower than a cloud service. 0:59

Practice

Explain Solution checked

In your own words, explain what retrieval augmented generation means, using the pizza reviews as the example.

Solution
  1. Retrieval: find the reviews in the CSV that match the question, e.g. reviews mentioning vegan.
  2. Augmented generation: give those reviews to the language model so it writes an answer based on them.

RAG first searches a document for the pieces relevant to a question (retrieval), then passes those pieces to a language model that writes the answer (generation). For example, when asked about vegan options, the app finds the reviews that mention vegan food and the model answers using them.

On your scratchpad

On your scratchpad, draw three boxes labelled 'CSV reviews', 'Relevant reviews' and 'Model answer', with arrows between them. Write one sentence under each arrow describing what happens.

Quiz

  1. What does the app use to answer questions about the restaurant?

    1. Only its own built-in knowledge of pizza
    2. Relevant reviews retrieved from the CSV file, given to a model
    3. A paid OpenAI cloud account
    4. A web search of the internet
    Show the answer

    B. Relevant reviews retrieved from the CSV file, given to a model · The app retrieves relevant reviews from the CSV and passes them to a local model (RAG).

    0:19 watch the moment

  2. Which tool is the vector search database in this project?

    1. Ollama
    2. LangChain
    3. ChromaDB
    4. Sublime Text
    Show the answer

    C. ChromaDB · ChromaDB is the vector database; Ollama runs the model and LangChain connects the parts.

    0:08 watch the moment

  3. How do you quit the app?

    1. Press Ctrl+Alt+Delete
    2. Type q at the question prompt
    3. Type exit in the CSV file
    4. Close the Sublime Text window
    Show the answer

    B. Type q at the question prompt · The prompt says 'q to quit', so typing q ends the program.

    1:18 watch the moment

Open lesson 1, with the video →

Lessons 2–7

They open with Beyond Pass: every lesson of every course, in full. One payment: $14.99 for 90 days or $29 for a year. No subscription.

Get Beyond Pass →Start lesson 1 free

Made by Beyond from the video “How to Build a Local AI Agent With Python (Ollama, LangChain & RAG)” by Tech With Tim. The video plays in YouTube's own player. Watch it on YouTube · All lessons