AI on the Edge LESSON 27: Track Objects of Interest in OpenCV Using Contours

In this lesson we show you how to box your object of interest in openCV.  We do this by finding outline contours around the object we tune in based on color in the HSV color space. Then we create a bounding rectangle around the contour. Then we end up having a rectangular bounding box that tracks the object of interest as it moves. This is the code we developed in the video. Note you will have to tune the LC and UC parameters for your object of interest, as we showed last week.

 

AI on the Edge LESSON 26: Understanding the HSV Color Space in OpenCV

Hey guys, welcome back to the channel. If you’ve been following along, you know we’ve been pushing our hardware absolutely down into the dirt. We’ve been running large language models right on the edge, pushing our boards hot and heavy until the silicon is screaming and the thermal throttling flags are popping up all over the place.

But today, we are stepping away from the heavy-compute server terminals, and we are getting back to our roots: Real-Time Computer Vision and Embedded Control. In our previous lessons, we learned how to hook up our high-speed camera, capture raw frames, and interact with individual pixels using standard RGB/BGR math. But today, we are going to look under the hood of a completely different way of representing color: The HSV Color Space (Hue, Saturation, Value).

If you try to track objects or isolate specific colors in the traditional RGB world, you are going to pull your hair out. The moment a shadow hits your object or the room lighting changes, your Red, Green, and Blue values completely collapse. By shifting our mathematics into the HSV space, we can lock onto a color’s pure identity regardless of whether it is sitting under a bright laboratory spotlight or a dim shadow.

Not only are we going to capture and process these video streams at a smooth-as-silk 60 frames per second, but we are also going to translate that raw visual math directly into the physical world. We are using our trusty SunFounder Fusion HAT+ to dynamically pulse an external RGB LED, matching its brightness and color hue perfectly to whatever pixel your mouse is clicking on in real-time.

Let’s look at the blueprint to make this happen.

The Complete Python Code

Here is the clean, un-guardrailed Python script for today’s lesson. Paste this directly into your local terminal workspace. No bloated libraries, no unnecessary frameworks—just pure, deliberate engineering.

Under the Hood: How the Code Works

1. The Real-Time Telemetry Smooth Filter

Look closely at how we calculate our frames-per-second metric inside the main processing loop:

If you simply print out the raw math of 1 / deltaT, your numbers on the screen are going to jump all over the place like a wild animal. By applying a 95% historical weight and a 5% instant weight, we create a low-pass software filter that smoothly tracks our true hardware operational speed without erratic layout jitter.

2. The Mouse Vector and BGR Array Sequence

When your mouse triggers an event over the window, OpenCV passes us the standard coordinate pairs (x, y). But remember: inside a NumPy data structure, images are structured as Rows first, then Columns. That means when you slice into your image array to read a pixel’s color values, you must pass the parameters as frame[y, x]. If you pass it as [x, y], your program is going to index out of bounds and crash hard.

Furthermore, always remember that OpenCV handles colors in a BGR (Blue, Green, Red) sequence, not RGB. When we extract those elements, they unpack straight into valB, valG, valR.

3. Masking and Bitwise Isolation

To lock onto our target color, we use cv2.inRange() to look at our HSV frame and check it against our lower constraint (LC) and upper constraint (UC). This generates a Mask—a pure black-and-white image where pixels within the target color space are completely white (255), and everything else is completely black (0).

By taking that mask and running a fast bitwise operation

We force the computer to evaluate every single pixel. If the mask is zero, the output is blacked out. If the mask is active, the original, rich color information passes through perfectly, isolating our target object from the background noise instantly.

Get your circuits wired up, get this script running on your machine, and let me know in the comments section below what kind of performance numbers you are pulling on your local workbench. I’ll catch you guys in the next lesson!

Remember we are still setting the LED color to the color that cursor is pointing at. This is the circuit for connecting the RGB LED.

Fusion Hat Circuit Diagram
This is the circuit we will use moving forward in the class

NVIDIA Jetson Orin Nano: How to Create and Use Swap Space for Larger Local LLM Models

In our first LLM class, we learned that we could run several of the smaller LLM models on the Jetson Orin, and then in the last class, we saw that we could make those small models run on the GPU, and we did notice faster performance as we began to move the workload to the GPU. The next challenge we found was that the larger models could give EOF errors, which were End of File errors, which usually means we have crashed due to not enough memory.  So, we need to work through this more methodically, and we need to run real benchmarks. Remember, we are running with the Big Dogs, and we are finding that we face tradeoffs between running on the CPU more slowly, or switching to the GPU and facing unpredictable throttling.

Our approach today is to deal with the memory issue. We will begin by turning off the GPU modifications, and just operate on the CPU. We will address the memory issue by creating swap space, and then we will benchmark our models running on the CPU, and complete a spreadsheet. We will need to begin by removing our configuration file that pointed us to the GPU:

Now lets create some swap space by allowing the system to use the SD card or the NVME for memory. Note, I am booting on a NVME. If you are using a SC card, you will notice more performance degradation by using swap space.

This command will create a swapfile:

This command will activate the swapfile:

This command will turnoff use of the swap space

This command shows you if the swapfile is being used

Now lets activate the swapfile:

Now each of these models should run on our system, as the swapfile will prevent the EOF error. Larger models will take a hit because they are bigger, and hence run slower, and then they bigger models will take a further hit in speed because swap memory will be slower than system RAM,

 

Model

Model Family Size / Parameter Count Best Used For
gemma3:1b Google Gemma 3 1 Billion Ultra-fast responses, light footprint
llama3.2:1b Meta Llama 3.2 1 Billion High-efficiency conversational loops
phi4-mini:3.8b Microsoft Phi-4 3.8 Billion Heavy reasoning and coding logic
qwen3:4b Alibaba Qwen 3 4 Billion Structured data and multilingual logic
qwen3.5:4b Alibaba Qwen 3.5 4 Billion Advanced context processing
gemma3:4b Google Gemma 3 4 Billion Maximum analytical depth on Orin Nano
nemotron-3-nano:4b NVIDIA Nemotron 3 4 Billion Edge-optimized reasoning and tool-use

For this lesson, we will just be using CPU computation. This will allow us to benchmark simply between models.

Benchmarking Local LLM on NVIDIA Jetson Orin on Jetpack 7.2        
Model CPU/GPU Power Prompt Rate Eval. Rate Throttling Correct Answer Swap
gemma3:1b CPU MaxN
gemma3:1b GPU MaxN
gemma3:1b GPU 25 Watts
gemma3:1b GPU 15 Watts
llama3.2:1b CPU MaxN
llama3.2:1b GPU MaxN
llama3.2:1b GPU 25 Watts
llama3.2:1b GPU 15 Watts
phi4-mini:3.8b CPU MaxN
phi4-mini:3.8b GPU MaxN
phi4-mini:3.8b GPU 25 Watts
phi4-mini:3.8b GPU 15 Watts
qwen3:4b CPU MaxN
qwen3:4b GPU MaxN
qwen3:4b GPU 25 Watts
qwen3:4b GPU 15 Watts
qwen3.5:4b CPU MaxN
qwen3.5:4b GPU MaxN
qwen3.5:4b GPU 25 Watts
qwen3.5:4b GPU 15 Watts
gemma3:4b CPU MaxN
gemma3:4b GPU MaxN
gemma3:4b GPU 25 Watts
gemma3:4b GPU 15 Watts
nemotron-3-nano:4b CPU MaxN
nemotron-3-nano:4b GPU MaxN
nemotron-3-nano:4b GPU 25 Watts
nemotron-3-nano:4b GPU 15 Watts

NVIDIA Jetson Orin Nano: Secret to Running Ollama on the GPU

One of the biggest frustrations with the new Jetpack 7.2 release is finding out that a standard installation of Ollama—the gold standard for running local LLMs—completely ignores your powerful NVIDIA GPU and defaults to the CPU.

In this lesson, we aren’t just going to fix that; we are going to measure the “truth” behind the performance. We will use data to see exactly how much gain we get from the GPU and where the hardware starts to hit the thermal throttling wall.

The Problem: The “Canned” Installation

When you run a standard Ollama install on the Jetson Orin Nano, the system doesn’t automatically recognize the integrated GPU (iGPU). If you open your NVIDIA Power GUI (jtop), you will see your CPU cores pegged at 100% while the GPU sits idle. This leads to slow response times and a disappointing experience.

Lets start by the standard ‘Canned’ Installation. The good news is, it is very simple:

To see exactly how your system is performing, run Ollama in verbose mode:

At this point you will have Ollama running a simple LLM locally on your Jetson Orin Nano. This is a huge step forward, but we now want to dig deeper and actually see how well this simple model is performing.  The first thing we do is run the Jetson Power GUI, hidden behind the NVIDIA icon in upper right of the menu bar.

Pay close attention to the Prompt Eval Rate and Eval Rate (tokens per second). These are our baseline numbers.

The “Secret Sauce” Solution

To force Ollama to use the Jetson’s CUDA cores, we have to manually override the system service configuration.

Step 1: Install the Nano Editor

Before we can edit system files, we need a reliable text editor. If you don’t have it yet, run this command:

Step 2: Create the Service Override

We need to tell the Ollama service exactly where to look for the GPU libraries. Use nano to open the following file:

Step 3: Add the Configuration

Copy and paste the following block into that file. This is the “Secret Sauce” that enables the iGPU and points the system to the correct CUDA backend:

Note: Save the file by pressing Ctrl+OEnter, and then Ctrl+X to exit.

Step 4: Reboot

For the changes to take effect, We will do a reboot.

Benchmarking the Results

Once you have the GPU engaged, the real work begins. In the video, we look at a side-by-side comparison of performance across different Jetson Power Modes (10W, 15W, and MaxN).

Power Level Prompt Eval Rate (t/s)  Eval Rate (t/s) Throttling Observed?
CPU [Your Data] [Your Data] Yes/No
10W [Your Data] [Your Data] Yes/No
15W [Your Data] [Your Data] Yes/No
MaxN [Your Data] [Your Data] Yes/No

As we discovered, moving to the GPU provides a boost, but it also increases the heat signature. Watch the full video to see the charts and understand which power level provides the best “sweet spot” for stable, long-term AI performance on your Jetson Orin Nano. This is an important first step . . . getting the heavy lifting down to the GPU. Now in future videos we will explore how to get the work done Well on the GPU.

 

AI on the Edge LESSON 25: Create Region of Interest (ROI) in openCV Using the Mouse

Well, hello there! I’m absolutely delighted you could join me today. If you’ve been following along with our journey into AI on the Edge, you know that we are getting closer and closer to building some truly powerful, real-world computer vision applications. But before we can get to the fancy AI stuff, we have to master the fundamentals. Today, we’re tackling something that is going to make your projects look—and feel—a whole lot more professional: creating a Region of Interest (ROI) using the mouse.

Why Do We Need an ROI?

Think about it. When you’re processing a video feed, you’re usually wasting a ton of compute power looking at things that don’t matter. Maybe you’re tracking a ball on a table, but your camera is seeing the whole room. Why process the walls and the ceiling when you only care about the table? By defining an ROI, we tell our code: “Ignore everything else. Only look here.” It saves processing time, it reduces noise, and it makes your AI much more accurate.

Interacting with OpenCV

In this lesson, we’re going to step beyond simple static code. I’m going to show you how to use OpenCV’s callback functions to make your program “live.” We’ll use the mouse to click and drag a rectangle directly on the video feed to define our ROI in real-time. It’s interactive, it’s intuitive, and it’s a vital skill for anyone building real-world vision systems.

The Code

Now, I’ve put a lot of work into making this code clean and easy to follow. You’ll see exactly how we capture those mouse events—cv2.EVENT_LBUTTONDOWN, cv2.EVENT_MOUSEMOVE, and cv2.EVENT_LBUTTONUP—to create that bounding box dynamically.

Putting It to the Test

I want you to take this code, run it on your Jetson, and play around with it. Try defining different regions. Notice how the frame rate stays steady because we aren’t bogging down the CPU with unnecessary pixels. This is the “Edge” part of “AI on the Edge”—making smart, efficient decisions right where the data is being captured.

I can’t wait to see what you build with this. As always, keep those questions coming, stay curious, and most importantly—don’t get discouraged! We’re doing hard things, and you are doing a great job.

I’ll see you in the next lesson!

What questions do you have about implementing ROI in your own computer vision projects? Post them in comments on the video! Thanks for learning.

We will be using the circuit used in the earlier lessons:

Fusion Hat Circuit Diagram
This is the circuit we will use moving forward in the class

 

Making The World a Better Place One High Tech Project at a Time. Enjoy!