Lessons · Coding · intermediate · Keith Galli

Solving real world data science tasks with Python Pandas!

The video “Solving real world data science tasks with Python Pandas!” by Keith Galli as 9 lessons, with the idea explained, the code from the screen, steps to try in the practice terminal, practice problems with solutions and a quiz that leads back to the moment.

9lessons
22pieces of code
27exercises
28questions
86min of video
Open the lessons →Lesson 1 is free · Beyond Pass opens all
Solving real world data science tasks with Python Pandas!, video by Keith Galli▶ 1:26:07 · Keith Galli
123456789
0:009 lessons across the whole video1:26:07

You will be able to

  • Combine multiple monthly CSV files into one pandas DataFrame using os.listdir and pd.concat
  • Clean messy data by finding and dropping NaN rows, removing repeated header rows, and converting strings to numeric types
  • Derive new columns with string methods, apply with a lambda, and datetime accessors such as dt.hour
  • Aggregate and visualize results with groupby, value_counts-style counts, and matplotlib bar and line charts, including a twin-axis overlay
  • Find items bought together by flagging duplicate order IDs, grouping them, and counting pairs with itertools.combinations and collections.Counter

Good to know first

  • Basic Python syntax, including lists, functions, and f-strings
  • Introductory pandas skills such as reading a CSV and inspecting a DataFrame with head()
  • Jupyter Notebook installed and running locally
  • Basic familiarity with matplotlib plotting

Syllabus

9 lessons

  1. 1

    Setup: download the sales data and start a Jupyter notebook Free

    0:00–3:334 min of video

    You will get the 12-month sales dataset from the course GitHub repository and open a fresh Jupyter notebook where the analysis will be written. This setup makes sure every later step runs on the same files.

    1 code4 build steps3 exercises3 questions
  2. 2

    Combine 12 monthly CSV files into one DataFrame Beyond Pass

    3:33–10:307 min of video

    You will learn how to list every file in a folder with os.listdir, read each monthly CSV into a DataFrame, and stack them together with pd.concat. You will then save the combined result to one CSV file with to_csv.

    3 code6 build steps3 exercises3 questions
  3. 3

    Add a Month Column and Clean NaN Rows and Repeated Headers Beyond Pass

    10:30–24:0014 min of video

    You will add a Month column by slicing the first two characters of the Order Date text and converting it to a number. You will then find and drop the empty NaN rows and the repeated header rows that block that conversion.

    4 code6 build steps3 exercises3 questions
  4. 4

    Create a numeric sales column from quantity and price Beyond Pass

    24:00–29:306 min of video

    You will turn text-looking number columns into real numbers with pd.to_numeric, then multiply quantity by price to make a Sales column. The key lesson is that arithmetic only works once the columns hold numeric types.

    1 code4 build steps3 exercises3 questions
  5. 5

    Find the best sales month with groupby and plot it Beyond Pass

    29:30–34:005 min of video

    You will group sales data by month and sum each group to find which month earned the most. Then you will draw the totals as a bar chart with matplotlib, adding labels so the chart is easy to read.

    3 code4 build steps3 exercises3 questions
  6. 6

    Extracting City and State from Addresses, then Plotting Sales by City Beyond Pass

    34:00–52:0018 min of video

    You will learn how to pull a city and state out of a messy address string using split and apply, and why two cities can share a name (Portland, Maine vs Portland, Oregon). You will then group sales by city and draw a bar chart whose labels line up with the values.

    2 code5 build steps2 exercises4 questions
  7. 7

    Analyze sales by hour of day with datetime parsing Beyond Pass

    52:00–1:03:0011 min of video

    You will convert a text column of order dates into real date-time objects, pull out the hour with the .dt accessor, and count orders per hour to find the best times to show ads. The chapter shows why a proper date-time type beats slicing strings by hand.

    2 code5 build steps3 exercises3 questions
  8. 8

    Find which products are bought together Beyond Pass

    1:03:00–1:14:0011 min of video

    You will learn how to pick out the orders that contain more than one product, join those products onto one line per order, and count which pairs of products appear together most often. The method uses pandas duplicated, groupby with transform, and Python's Counter with combinations.

    4 code4 build steps4 exercises3 questions
  9. 9

    Compare quantity ordered against price with a dual-axis chart Beyond Pass

    1:14:00–1:26:0712 min of video

    You group sales by product to total the quantity ordered, then draw a bar chart of those totals. You then add a second y-axis with twinx() to overlay each product's average price, and use the two lines together to test a hypothesis.

    2 code5 build steps3 exercises3 questions

Free preview · Lesson 1 in full

Setup: download the sales data and start a Jupyter notebook

You will get the 12-month sales dataset from the course GitHub repository and open a fresh Jupyter notebook where the analysis will be written. This setup makes sure every later step runs on the same files.

0:00 to 3:33 in the video

Practice terminal

Type the lesson's commands, break things, and see why.

Playground

Run and change the code, with real Python and JavaScript.

Scratchpad

Draw and write next to the video.

The idea

Real-world data science starts with a dataset you can open. Here the data is 12 months of sales records. It sits in a folder called Sales_Data, inside the SalesAnalysis folder of Keith Galli's Pandas-Data-Science-Tasks repository on GitHub.

A repository is a project folder stored online. There are two ways to get it. If you know Git, fork the repository and clone it. If not, click the green Clone or download button and choose Download ZIP.

With the ZIP, extract it into the folder where you want to work. Keep the data, the code and the notebook together. Then your code can find the files with short, simple paths.

Next, open Jupyter Notebook. It is a tool that opens in your web browser and lets you write code and see the results in one document. Go to the SalesAnalysis folder, click New, choose Python 3, and name the notebook analysis. The repository already has a finished notebook, but you start from a blank one so you practise every step yourself.

The video also sets the rule for the series: when a new task appears, try it on your own first, then watch the solution. In the first cell, type import pandas as pd and run it. pandas is the Python library for working with tables of data, and pd is its short name.

Key points

  • The dataset is 12 months of sales data in the Sales_Data folder of the SalesAnalysis project. 0:43
  • You can get the repository either by forking and cloning with Git, or by downloading the ZIP and extracting it. 1:00
  • Jupyter Notebook lets you write code, run it and show the analysis together in one document. 2:17
  • Start with a new blank notebook so you practise each step yourself. 3:03
  • Try each task on your own first, and only then watch the solution. 3:16
  • The first code step is to import pandas. 3:27

Words to know

repository
A project folder stored online (here on GitHub) that holds the data and code.
clone / fork
With Git, to fork is to make your own copy of a repository on GitHub, and to clone is to download that copy to your computer.
ZIP file
One compressed file that holds many files. You extract it to get the folder back.
Jupyter Notebook
A tool in your browser where you write code in cells and see the results below each cell.
pandas
A Python library for working with tables of data.
relative path
A file location written from the notebook's own folder, such as Sales_Data, instead of the full path on your computer.

Code from the screen

Codepython
import pandas as pd

Type this into the first cell of your new Jupyter notebook in the SalesAnalysis folder, then press Shift+Enter to run it. Install pandas first with: pip install pandas

Build along in the practice terminal

  1. Download the repository as a ZIP from the green Clone or download button on GitHub, then click Download ZIP. 1:00
  2. Extract the ZIP to the folder where you want to work on the code. 1:40You should see: A folder named Pandas-Data-Science-Tasks-master appears, containing SalesAnalysis with Sales_Data inside.
  3. Open Jupyter Notebook (it opens in your browser), go to the SalesAnalysis folder, and click New then Python 3. 2:16
    $ jupyter notebook
[I 10:12:01.234 NotebookApp] Jupyter Notebook is running at:
[I 10:12:01.235 NotebookApp] http://localhost:8888/?token=...
    You should see: A blank notebook opens with an empty cell showing In [ ]:.
  4. Name the notebook analysis, then in the first cell import pandas and run it. 3:27
    $ import pandas as pd
    You should see: The cell runs without an error and shows In [1]: next to it.

Watch out

  • Downloading the ZIP but not extracting it first: Jupyter cannot browse inside a ZIP file, so extract it.
  • Creating the notebook in the wrong folder, so the Sales_Data folder is not next to it and later file paths fail.
  • Forgetting to run the cell: typing import pandas as pd does nothing until you press Shift+Enter.

Practice

Do it Solution checked

Create a new folder called my_sales on your computer, and inside it open a new Jupyter notebook named practice. In its first cell, import pandas and run the cell.

Hint

Open a terminal in the folder and type jupyter notebook.

Solution
  1. Make a new folder called my_sales.
  2. Open a terminal in that folder and run jupyter notebook.
  3. Click New, then Python 3 to create a notebook.
  4. Rename it practice by clicking the title.
  5. Type import pandas as pd in the first cell and press Shift+Enter.

The cell runs with no error and shows an In [1]: number.

Code Solution checked

In a second cell of your notebook, check that pandas works by printing its version number.

import pandas as pd
Hint

pandas stores its version in pd.__version__ (two underscores on each side).

Solution
  1. Make sure the first cell with import pandas as pd has been run.
  2. In a new cell, type print(pd.__version__).
  3. Run the cell with Shift+Enter.
import pandas as pd
print(pd.__version__)

A version number such as 1.5.3 is printed below the cell, with no error.

Explain Solution checked

In your own words, explain why the data, the code and the notebook should be kept in the same folder.

Hint

Think about how the notebook finds a file.

Solution
  1. The notebook can then find the data files using simple relative paths such as Sales_Data.
  2. If the files are in different places, the code breaks with a file not found error.
  3. Keeping them together also makes the analysis easy to share.

Keeping them together lets the notebook load data with simple relative paths, so the code still works when the folder is copied elsewhere.

On your scratchpad

Draw a simple folder tree: a top folder, with a SalesAnalysis folder inside it, holding Sales_Data (the CSV files) and your analysis notebook side by side.

Quiz

  1. Which folder holds the 12 months of sales data?

    1. Misc
    2. Sales_Data inside SalesAnalysis
    3. Output
    4. The README.md file
    Show the answer

    B. Sales_Data inside SalesAnalysis · The data is in the Sales_Data folder inside SalesAnalysis.

    0:43 watch the moment

  2. What is the first code you run in the new notebook?

    1. Plot the sales
    2. import pandas as pd
    3. Delete the data
    4. Install Git
    Show the answer

    B. import pandas as pd · The first cell imports pandas so it is ready for the next steps.

    3:27 watch the moment

  3. Why does the video start a new blank notebook instead of opening the finished one?

    1. The finished one is broken
    2. So you can build the analysis yourself from scratch
    3. Jupyter cannot open old notebooks
    4. The finished one has no data
    Show the answer

    B. So you can build the analysis yourself from scratch · The goal is for you to practise and solve each task yourself.

    3:03 watch the moment

Open lesson 1, with the video →

Lessons 2–9

They open with Beyond Pass: every lesson of every course, in full. One payment: $14.99 for 90 days or $29 for a year. No subscription.

Get Beyond Pass →Start lesson 1 free

Made by Beyond from the video “Solving real world data science tasks with Python Pandas!” by Keith Galli. The video plays in YouTube's own player. Watch it on YouTube · All lessons