One of the biggest frustrations with the new Jetpack 7.2 release is finding out that a standard installation of Ollama—the gold standard for running local LLMs—completely ignores your powerful NVIDIA GPU and defaults to the CPU.
In this lesson, we aren’t just going to fix that; we are going to measure the “truth” behind the performance. We will use data to see exactly how much gain we get from the GPU and where the hardware starts to hit the thermal throttling wall.
The Problem: The “Canned” Installation
When you run a standard Ollama install on the Jetson Orin Nano, the system doesn’t automatically recognize the integrated GPU (iGPU). If you open your NVIDIA Power GUI (jtop), you will see your CPU cores pegged at 100% while the GPU sits idle. This leads to slow response times and a disappointing experience.
Lets start by the standard ‘Canned’ Installation. The good news is, it is very simple:
Shell
1
curl-fsSL https://ollama.com/install.sh|sh
To see exactly how your system is performing, run Ollama in verbose mode:
1
ollama run gemma3:1b
At this point you will have Ollama running a simple LLM locally on your Jetson Orin Nano. This is a huge step forward, but we now want to dig deeper and actually see how well this simple model is performing. The first thing we do is run the Jetson Power GUI, hidden behind the NVIDIA icon in upper right of the menu bar.
Pay close attention to the Prompt Eval Rate and Eval Rate (tokens per second). These are our baseline numbers.
The “Secret Sauce” Solution
To force Ollama to use the Jetson’s CUDA cores, we have to manually override the system service configuration.
1
sudo apt-get update&&sudo apt-get install nano
Step 1: Install the Nano Editor
Before we can edit system files, we need a reliable text editor. If you don’t have it yet, run this command:
Step 2: Create the Service Override
We need to tell the Ollama service exactly where to look for the GPU libraries. Use nano to open the following file:
Note: Save the file by pressing Ctrl+O, Enter, and then Ctrl+X to exit.
Step 4: Reboot
For the changes to take effect, We will do a reboot.
Benchmarking the Results
Once you have the GPU engaged, the real work begins. In the video, we look at a side-by-side comparison of performance across different Jetson Power Modes (10W, 15W, and MaxN).
Power Level
Prompt Eval Rate (t/s)
Eval Rate (t/s)
Throttling Observed?
CPU
[Your Data]
[Your Data]
Yes/No
10W
[Your Data]
[Your Data]
Yes/No
15W
[Your Data]
[Your Data]
Yes/No
MaxN
[Your Data]
[Your Data]
Yes/No
As we discovered, moving to the GPU provides a boost, but it also increases the heat signature. Watch the full video to see the charts and understand which power level provides the best “sweet spot” for stable, long-term AI performance on your Jetson Orin Nano. This is an important first step . . . getting the heavy lifting down to the GPU. Now in future videos we will explore how to get the work done Well on the GPU.
Well, hello there! I’m absolutely delighted you could join me today. If you’ve been following along with our journey into AI on the Edge, you know that we are getting closer and closer to building some truly powerful, real-world computer vision applications. But before we can get to the fancy AI stuff, we have to master the fundamentals. Today, we’re tackling something that is going to make your projects look—and feel—a whole lot more professional: creating a Region of Interest (ROI) using the mouse.
Why Do We Need an ROI?
Think about it. When you’re processing a video feed, you’re usually wasting a ton of compute power looking at things that don’t matter. Maybe you’re tracking a ball on a table, but your camera is seeing the whole room. Why process the walls and the ceiling when you only care about the table? By defining an ROI, we tell our code: “Ignore everything else. Only look here.” It saves processing time, it reduces noise, and it makes your AI much more accurate.
Interacting with OpenCV
In this lesson, we’re going to step beyond simple static code. I’m going to show you how to use OpenCV’s callback functions to make your program “live.” We’ll use the mouse to click and drag a rectangle directly on the video feed to define our ROI in real-time. It’s interactive, it’s intuitive, and it’s a vital skill for anyone building real-world vision systems.
The Code
Now, I’ve put a lot of work into making this code clean and easy to follow. You’ll see exactly how we capture those mouse events—cv2.EVENT_LBUTTONDOWN, cv2.EVENT_MOUSEMOVE, and cv2.EVENT_LBUTTONUP—to create that bounding box dynamically.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
import cv2
import time
from picamera2 import Picamera2
from fusion_hat.pwm import PWM
piCam=Picamera2()
W=1280
H=720
tStart=time.time()
fps=0
redPin=5
greenPin=6
bluePin=7
redLED=PWM(redPin)
greenLED=PWM(greenPin)
blueLED=PWM(bluePin)
RES=(W,H)
piCam.preview_configuration.main.size=RES
piCam.preview_configuration.main.format="RGB888"
piCam.preview_configuration.controls.FrameRate=60
piCam.preview_configuration.align()
piCam.configure("preview")
piCam.start()
textLowerLeft=(int(W*.01),int(H*.06))
fontFace=cv2.FONT_HERSHEY_SIMPLEX
fontThickness=int(W/425)
fontScale=H*.0015
fontColor=(0,0,255)
textLowerLeft1=(int(W*.01),int(H*.06)*2)
textLowerLeft2=(int(W*.01),int(H*.06)*3)
xPos=0
yPos=0
valR=0
valG=0
valB=0
frame=None
drawing=False
startX=0
endX=0
makeROI=False
boxColor=(0,255,255)
boxThick=3
def mouseAction(event,x,y,flags,param):
globalframe,xPos,yPos,valR,valG,valB
globaldrawing,makeROI,startX,startY,endX,endY
ifevent==0:
xPos=x
yPos=y
ifframe isnotNone:
valB,valG,valR=frame[y,x]
redLED.pulse_width_percent(int(valR/255*100))
greenLED.pulse_width_percent(int(valG/255*100/2))
blueLED.pulse_width_percent(int(valB/255*100/4))
ifdrawing==True:
endX=x
endY=y
ifevent==1:
drawing=True
makeROI=False
startX=x
startY=y
endX=x
endY=y
ifevent==4:
drawing=False
makeROI=True
endX=x
endY=y
cv2.namedWindow('Camera')
cv2.moveWindow('Camera',0,65)
cv2.setMouseCallback('Camera',mouseAction)
print("Click and Drag to Draw a Box")
print("Release Mouse to Finalize box. Press q to Quit")
I want you to take this code, run it on your Jetson, and play around with it. Try defining different regions. Notice how the frame rate stays steady because we aren’t bogging down the CPU with unnecessary pixels. This is the “Edge” part of “AI on the Edge”—making smart, efficient decisions right where the data is being captured.
I can’t wait to see what you build with this. As always, keep those questions coming, stay curious, and most importantly—don’t get discouraged! We’re doing hard things, and you are doing a great job.
I’ll see you in the next lesson!
What questions do you have about implementing ROI in your own computer vision projects? Post them in comments on the video! Thanks for learning.
We will be using the circuit used in the earlier lessons:
This is the circuit we will use moving forward in the class
Welcome back. If you are watching this, you’re ready to stop playing with toys and start building real-world AI. Today, we are looking at the NVIDIA Jetson Orin Nano. Let’s get one thing straight: this is not a Raspberry Pi.
Under the hood, you are working with an Ampere-architecture GPU featuring 1,024 CUDA cores and 32 Tensor cores. You have a 6-core ARM Cortex-A78AE v8.2 64-bit CPU. Depending on how you configure your power mode, you are looking at anywhere from 20 to 40 TOPS of AI performance. This is raw, unadulterated horsepower that can process multi-stream video pipelines in real-time. In the 15W mode, you are managing a delicate balance of thermals and throughput; in the 25W mode, you are pushing the limits of the silicon itself. But this power comes with a price. You have been playing in an amusement park, but now, you’re going skydiving. The guardrails are gone.
The Skydiving Mindset: In the Pi or Arduino world, everything is ‘turn-key.’ You follow the recipe, you get the cake. It’s safe. It’s predictable. But when you are dealing with 40 TOPS of compute, the environment is fundamentally different. There are no guardrails here. If you don’t do the work, if you don’t check your own gear, you hit the ground.
There is a fundamental shift in responsibility when you move from consumer hobbyist boards to professional embedded silicon. You aren’t just a user anymore; you are an architect. If you’re looking for a guaranteed result because you clicked a link, go back to the Pi. If you’re looking to master high-performance silicon, welcome to the deep end. We are ‘Running with the Big Dogs’ now.
The Infrastructure Tax: Let’s start with the cost of entry. If you are trying to develop on an Orin using a Virtual Machine or a dual-boot setup on your Windows gaming laptop, stop. Just stop. You are setting yourself up for a failure that has nothing to do with the board and everything to do with your infrastructure.
I’ll give you a horror story. I tried to dual-boot my main workstation to make it ‘easier’ to access the Ubuntu environment needed for the SDK Manager. I triggered a BitLocker conflict. It didn’t just break the bootloader; it effectively bricked my NVMe drive so thoroughly that I had to dump the drive, buy a replacement, and reload my entire backup image from scratch.
That is the ‘Big Dog’ tax. Professionals don’t risk their primary workstation for a development tool. You build a dedicated, stand-alone Ubuntu machine. That is the cost of entry. If you can’t commit to a clean Linux environment, you aren’t ready for this hardware. The SDK Manager requires low-level USB access and partition control that hypervisors simply cannot handle reliably. You want to play with the big silicon? You bring the right infrastructure.
The Illusion of Instructions: You’ve probably heard people complain that my instructions didn’t work. Or they get angry at NVIDIA because the latest JetPack caused a kernel panic. I want to tell you the truth: You aren’t following instructions; you’re following suggestions.
Look at JetPack 7.2. Thousands of people followed the official documentation to the letter, and for half of them, it failed. The ‘Super Mode’ didn’t show up. And in the frantic attempt to force it to appear, many of them bricked their boards. When you brick an Orin—and you will—you don’t get a ‘reset’ button. You get a terminal, a flashing USB cable, and the SDK Manager.
When you’re flying a jet, you don’t blame the manual when the engine flame-outs. You check the instrumentation. The Jetson is your instrumentation. If it says ‘Over-Current,’ you don’t get mad at the manufacturer—you analyze your power budget. You are pushing hardware to its thermal and electrical limits. You are choosing your destiny with every power-mode configuration you change. This isn’t a software update; it’s a battlefield.
The Oracle of Delphi: Now, let’s talk about the NVIDIA forums. Think of those forums as the Oracle of Delphi. You do not walk into that house and demand service. If you post, ‘I followed the instructions and it broke, what a goat rodeo, you guys released a broken OS,’ you are done. You will be ignored, and you will lose all professional credibility.
Here is the 12-Hour Rule: Before you post, you spend 12 hours of deep-dive, log-file-reading, self-inflicted pain on your own. You read the dmesg output. You check your logs in /var/log/syslog. You look at jtop and you watch the power rails. If you can’t describe exactly what is happening, you aren’t ready for help.
When you do post, you provide a reproduction script. You provide data. You treat those engineers with the respect they deserve. And when they respond? You shut up and listen. They are the pilot; you are the co-pilot. You do not touch the controls. You follow their lead, you execute their tests, and you report the results. Any frustration you express makes you look like a hobbyist who doesn’t understand the complexity of what they are touching. You are a guest in their house. Earn your stay.
Log-Driven Development: If your terminal isn’t covered in log outputs, you aren’t debugging—you’re guessing. Guessing is for hobbyists. Engineers measure. In the Pi world, you just write code and it works. On the Jetson, you have to think like an architect. Is your code saturating the memory bandwidth? Is your model actually hitting the Tensor cores? If you treat the Orin like a general-purpose PC, you are wasting the most powerful tool on your desk. You have to learn the power envelope. You have to learn the thermal limitations. You are driving a Ferrari in first gear if you don’t understand what’s happening under the hood.”
The Verdict: So, here is my promise to you. You will brick it. You will want to throw it against the wall. But the moment you decide to solve the problem instead of blaming the manufacturer, that is the exact moment you stop being a hobbyist and start being an engineer. You want to run with the Big Dogs? Then stop whining about the guardrails and start learning how to read the logs. See you in the next lesson.
So the question for you now is, are you really ready to Run with the Big Dogs? Are you ready to jump into the deep end of the pool, or do you want to return to the wading pond?
Welcome back, everyone! In our last lesson, we learned how to use matrix slicing to hardcode a Region of Interest (ROI) into our frames. That was a great static approach, but today we are taking interactivity to a whole new level.
In this lesson, you are going to learn how to catch Mouse Events inside your OpenCV windows. Instead of guess-and-checking coordinates in your code, you will be able to click anywhere on your live video stream to instantly grab the precise (x, y) pixel coordinates and read the exact color value of the pixel right under your mouse pointer. This is the foundational mechanic you need to build interactive, point-and-click AI applications.
The Core Concept: Mouse Callbacks and Global Frames
To listen for mouse clicks or movement, OpenCV uses what is called a Callback Function. You tell OpenCV: “Hey, keep an eye on this specific window. If the user does anything with the mouse inside it, instantly jump over to my custom function and tell me what happened.”
We set this up using:
cv2.setMouseCallback(‘Camera’, mouseAction)
The [y, x] Matrix Inversion Trap
There is a massive mathematical trap that catches almost every beginner when they start mapping mouse clicks to image matrices:
OpenCV Mouse Coordinates: When you move your mouse, OpenCV tracks position using standard Cartesian geometry: (x, y), where x is the column (horizontal distance from the left) and y is the row (vertical distance from the top).
NumPy Array Coordinates: When you plug those numbers into your image array to inspect a pixel, NumPy expects matrix indexing: [row, column].
Because rows correspond to the height (y) and columns correspond to the width (x), you must always invert the coordinates when accessing the frame array:
If you try to pass frame[x, y], your program will either crash with an “index out of bounds” error or return data from the completely wrong part of the image!
The Python Code Developed in This Lesson
Here is the complete, streamlined script we built during today’s tutorial. Copy this code into your workspace on your Raspberry Pi 5, fire it up, and watch your terminal output as you click around the video window.
We first developed this program as a simple example of processing mouse clicks, and print the detected event:
In order to make the program more useful, we developed this code that monitors the position of the mouse cursor, and reports the color of the pixel the mouse points at. The values are printed as labels on the openCV frame:
We can now take the project to the next level by setting the LED color to the color pointed at by the cursor in the openCV window. We will be using our standard circuit we have used in the earlier lessons.
This is the circuit we will use moving forward in the class
This is the code we developed to set the LED color based on the pixel position of the cursor in the openCV window.
Alright, it’s time to put this knowledge to work. Your homework assignment is to turn this simple reporting tool into an interactive, dynamic ROI selector. The homework is to first create a text display under the FPS on the frame that show RGB value at the pixel position the mouse is pointing at, and the pixel location.
Your homework assignment is to turn this simple reporting tool into an interactive, dynamic ROI selector.
Start with your clean 1280×720 live camera stream.
Modify your mouseAction callback function to look for specific mouse clicks.
The Target Mechanic: When you Left-Click on the video window, store those specific coordinates as your upper-left corner. When you release the click, store those coordinates as your lower-right corner. As you are selecting, draw a live box outline over your ROI
Using those two dynamic coordinate sets, use matrix slicing to pull a clean Region of Interest (ROI) out of the frame and instantly display it in a completely separate, standalone window called “Target ROI”.
Safety Requirement: Make sure your code can handle clicks in any order without crashing (e.g., if a user right-clicks higher or further left than their left-click, write the conditional logic to sort the indices properly before slicing).
Get your black coffee ready, write your logic step-by-step from scratch, and do not copy code you can’t explain. Post your homework solution video on YouTube and drop a link in the comments section below so I can see who is running with the big dogs!
Welcome back, everyone! In this lesson, we are stepping into a foundational aspect of computer vision: manipulation of specific regions within a video frame.
Up to this point, we have been grabbing the full frame from our camera and performing operations on the entire image. But in real-world edge AI and robotics applications, processing every single pixel of a high-resolution frame is an absolute waste of compute power. If you want to detect a license plate, track a face, or monitor a specific sensor layout on a machine, you don’t need to look at the sky or the floor. You need to isolate a Region of Interest (ROI).
In this lesson, you will learn how to use Python’s powerful matrix slicing capabilities to chop up a frame, isolate specific quadrants, manipulate pixels inside an ROI, and display multiple synchronized windows across your desktop without crashing your system footprint.
The Core Concept: Image Slicing and ROIs
In OpenCV, an image frame isn’t just a visual picture—it is a standard NumPy array. A color frame is a 3D matrix structured by rows, columns, and color channels: [Rows, Columns, Channels] or [Height, Width, Color].
Because it is a standard array, we can use standard Python slicing notation to isolate any rectangular box we want:
ROI = frame[rowstart : rowend, colstart : colend]
The .copy() Trap
When you slice a piece of an array in Python like ROI = frame[0:100, 0:100], Python does not create a new image in your RAM. It creates a view or a pointer back to the original frame. If you modify pixels inside that ROI, you will accidentally alter your original main camera frame!
To isolate a region and modify it independently without bleeding back into your primary frame, you must explicitly use the .copy() method:
Below is the complete code script we built during the video tutorial. Copy this code exactly into your Python environment, verify your geometry setups, and run it.
Alright, it is time to earn your stripes and see if you can fly with the big dogs. Your homework assignment is to take this foundation and build a dynamic tracking target box using the array geometry principles we just learned.
Create a single main camera window (640 x 360).
Draw an independent rectangular ROI box that starts directly in the dead center of the screen.
Using your keyboard parameters (cv2.waitKey), program the system so that using the Arrow Keys (or ‘i’, ‘j’, ‘k’, ‘l’) smoothly updates variables to move the ROI box dynamically around the screen in real-time.
Crucial Constraint: Do not let your boundary indices drift off the array! You must write conditional boundaries so that if your moving target hits the edge of your $640 \times 360$ boundary layout, it locks at the frame border and prevents an out-of-bounds index crash.
In a separate output window, display only the contents of the moving target box in real-time grayscaled format.
Grab your morning coffee, fire up your code editor, write the script from scratch, and do not copy-paste code you don’t understand. Leave a link to your homework solution video in the YouTube comments section so I can see your progress!
Making The World a Better Place One High Tech Project at a Time. Enjoy!
We use cookies to ensure that we give you the best experience on our website. If you continue to use this site we will assume that you are happy with it.
