AI On the Edge, Raspberry Pi

AI on the Edge LESSON 56: Identifying Object of Interest in Target Zone with Mediapipe

Welcome back, everyone! Paul McWhorter here from toptechboy.com, and welcome to LESSON 56 of our AI on the Edge tutorial series! Grab yourself a nice hot cup of iced coffee (no sugar needed!), sit back, relax, and let’s jump right into today’s project.

AI on the Edge LESSON 56: Identifying Object of Interest in Target Zone with MediaPipe

In our previous lessons, we learned how to use MediaPipe’s object detection task to scan our live camera feed and pick out objects using pre-trained edge AI models like efficientdet_lite0.tflite. We got our bounding boxes drawn, displayed class labels, and showed confidence scores—all running smoothly right on our Raspberry Pi!

Today, we are taking our computer vision setup to the next level by building a Target Zone Detector.

What We Are Doing Today

Instead of just blindly detecting everything in the frame, we want our edge device to actively monitor a specific region of interest—a “Target Zone”—and trigger an alert when a specific Target Class enters that boundary.

Here is what we accomplish in this lesson:

  1. Define a Fixed Target Zone: We calculate a customizable bounding box right in the center of our 1280×720 video feed (e.g., 150×250 pixels).

  2. Filter by Object Class: We set a target object (in this example, a ‘bottle’). The system ignores all other object classes detected in the scene.

  3. Calculate the Bounding Box Center: For the target object, we compute its exact center point (X_center, Y_center) on the fly and draw a tracking dot on it.

  4. Boundary & Collision Logic: We test whether the center coordinates of our target fall inside our designated target zone using the condition: zoneX1 <= X_center <= zoneX2 and zoneY1 <= Y_center <= zoneY2.

  5. Dynamic Visual Feedback:

    • Idle State: When no target is inside the zone, the box remains Green.

    • Active State: The moment the target’s center point steps into the target zone, the boundary box turns Red, and an alert banner—”Target Has Entered the Zone”—is dynamically rendered on the camera feed.

  6. Real-Time Performance: We track frame rate (FPS) using an exponential moving average calculation to keep an eye on edge processing speed.

Homework / Challenge Assignment

Now that you have seen how to set up logic for a single target object in a single zone, here is your homework assignment to solidify what you’ve learned!

  1. We want to now actively track the selected Object of Interest. Turn your servos on on the pan tilt camera, and keep the Object of Interest in the center of the camera view.

Give it a try on your own before watching the solution in the next video. Remember: You don’t learn by watching; you learn by doing!

Discussion & Community

Did your target zone detection work smoothly on your Raspberry Pi setup? How many FPS are you getting with your Pi Camera? Leave a comment below on YouTube to share your results, questions, or modifications!

If you enjoyed this tutorial, hit that Like button, Subscribe to the channel, and share it with anyone who wants to learn real-world AI on the Edge.

As always, I’ll see you guys in the next lesson!

General Knowledge / Context

  • MediaPipe Tasks API: MediaPipe Python Tasks API provides high-level Python interfaces for object detection, landmasking, and classification tasks utilizing lightweight edge models such as TensorFlow Lite (.tflite).

  • PiCamera2 & OpenCV: In modern Raspberry Pi OS (Bullseye/Bookworm), Picamera2 handles native camera hardware capture in raw formats like RGB888, which can then be converted to MediaPipe Image objects (mp.Image) and rendered on OpenCV (cv2) GUI windows seamlessly.

 

AI On the Edge, Raspberry Pi

AI on the Edge LESSON 55: Object Detection on the Raspberry Pi 5 With MediaPipe

Welcome back, everyone! This is Paul McWhorter from TopTechBoy.com, and welcome to Lesson 55 in our AI on the Edge tutorial series!

If you’ve been following along, grab yourself a nice, hot cup of black coffee, fire up your Raspberry Pi 5, and let’s get right into it!

What We Are Doing in Lesson 55

Up to this point in the series, we have spent a lot of time working with land-marking models—tracking human hands, pose points, and facial features. But today, we step into full-blown computer vision object detection!

In this lesson, we learn how to implement MediaPipe’s lightweight, standalone object detection task API (mediapipe.tasks.python.vision.ObjectDetector) running on the Raspberry Pi 5. Using the pre-trained COCO dataset model (efficientdet_lite0.tflite), our Pi 5 can instantly identify up to 80 different real-world objects—from people, cups, and bottles to chairs, laptops, and cell phones—and draw real-time bounding boxes around them on our video feed.

Key Takeaways & Concepts Covered

  1. Setting Up the MediaPipe Object Detector Task API:

    Unlike older mp.solutions wrappers, the modern MediaPipe Tasks API requires configuring BaseOptions with the explicit local path to your .tflite model asset (modelPath = "/home/pi/ai-lab-kit/mediapipe/efficientdet_lite0.tflite").

  2. Configuring Detector Parameters:

    • score_threshold (0.35): Sets the minimum confidence score required to accept a detected object. Lowering this value captures weaker detections but risks false positives; raising it ensures high-confidence target lock.

    • max_results (3): Limits the maximum number of bounding boxes predicted per frame to keep our processing loop lean and fast.

  3. Decoding Detection Bounding Boxes:

    MediaPipe returns detected objects with bounding box geometry relative to the frame origin (origin_x, origin_y, width, height). We parse these dimensions directly into OpenCV integers (x, y, w, h) to draw bounding boxes with cv2.rectangle().

  4. Category Labeling & Confidence Rendering:

    We pull category names (category_name) and classification scores (category.score) from detection.categories[0], format the confidence as a percentage, and render clean text labels anchored directly above each target box.

  5. Optimal Frame Alignment & FPS Monitoring:

    We capture 60 FPS video directly from picamera2 in RGB888 format, flip it properly to match our physical camera mounting, and track real-time frames-per-second using our low-pass smoothing formula.

What You Will Need

To follow along with this lesson, ensure you have your standard gear ready from the kit:

  • Raspberry Pi 5 (with Active Cooler/Heat Sink)

  • Raspberry Pi Camera Module 3 (or compatible Camera Module driven via picamera2)

  • SunFounder Fusion AI Lab Kit (with pre-downloaded MediaPipe .tflite model weights)

Homework Assignment

Alright, class, you know I never let you off the hook without some homework!

Your assignment for this lesson is to take this foundational object detection script and upgrade its intelligence:

  1. Target Filter: Modify the loop so that it only draws bounding boxes around a specific target object of your choice (for instance, only detect "person" or only detect "bottle"), ignoring all other detected COCO classes.

  2. Target Tracking Dot: Calculate the exact center pixel coordinate $(x + \frac{w}{2}, y + \frac{h}{2})$ of your targeted object, and draw a solid red target circle (cv2.circle) right on the object’s center point.

  3. Dynamic On-Screen Target Status: If your target object is visible in the frame, display a green text banner in the upper corner reading "TARGET ACQUIRED". If it leaves the frame, change the status banner to a red "SEARCHING...".

Give it a shot, take your time, and write your solution from scratch without copying and pasting! Drop a comment on the YouTube video to let me know if you got it working or if you ran into any snags.

Until next time, get your coffee, keep coding, and I’ll see you guys in Lesson 56!

 

AI On the Edge, Raspberry Pi

AI on the Edge LESSON 54: Custom Hand Gesture Recognition & Dynamic NeoPixel Control

Welcome back, everyone! In this lesson, we take edge AI interactivity to the next level. Up until now, we have learned how to track hand landmarks, extract pixel coordinates, and calculate distances using MediaPipe. Today, we put all those building blocks together into a self-contained, real-time hand gesture recognition system running on the Raspberry Pi 5.

We are not relying on a black-box pre-trained gesture model. Instead, we write our own mathematical feature extractor, train our custom hand poses on the fly, save those gesture profiles directly to disk, and map each gesture to a non-blocking lighting animation on a NeoPixel LED ring driven over SPI.

System Architecture Overview

The system operates in three distinct phases running simultaneously inside our main loop:

  1. Normalized Feature Extraction: MediaPipe extracts 3D hand landmark coordinates. To make gesture detection scale-invariant and distance-invariant, we normalize the distances between the fingertips and the wrist against a reference palm dimension.

  2. Nearest-Neighbor Gesture Matching: Live distance vectors are compared against stored gesture profiles using a mean absolute error algorithm. If the error falls below our maximum threshold, the gesture is identified; otherwise, it defaults to “Unknown.”

  3. Non-Blocking Hardware Animation Dispatch: The recognized gesture string triggers corresponding pattern functions on the 12-LED NeoPixel ring via neopixel_spi. Animations use non-blocking time delta checks to preserve high camera preview frame rates.

Gesture-to-Hardware Animation Mapping

Our script contains predefined animation routines mapped to recognized gesture strings. Here is how each pose controls the LED ring:

Recognized Gesture LED Animation Pattern Visual Effect
fist Red Pulse Sinusoidal breathing effect fading in and out
palm Solid Beacon Full brightness white ring
peace Dual Spinner Dual green and blue dots rotating over a dim red background
thumbsup Solid Success Constant green feedback
point Radar Sweep A green trailing dot chasing around a red ring
rock Cyan Flame Sinusoidal green-cyan intensity modulation
ok Rainbow Wheel Rotating HSV spectrum cycle
Unknown / Unmatched Clear LEDs turn completely off

Interactive Keyboard Controls

While the camera loop is active, you can interact with the system using your keyboard in the preview window:

  • Spacebar: Captures the current hand pose’s normalized distance vector, assigns it to the active target gesture name, and saves the updated dataset to gestureNEOPIXEL.pkl.

  • t: Prompts the terminal to add a new custom gesture name to your training queue dynamically.

  • d: Prompts the terminal to delete a specific gesture profile from the saved dataset.

  • q: Clears the NeoPixel ring, closes the camera pipeline, saves training data, and terminates the program cleanly.

Hardware Wiring Reference

Ensure your Raspberry Pi 5 SPI interface is enabled before running the code (sudo raspi-config -> Interface Options -> SPI).

  • NeoPixel Data Input (DIN): Connect to GPIO 10 (SPI0 MOSI / Pin 19).

  • NeoPixel Ground: Connect to any GND pin on the Pi expansion header.

  • NeoPixel Power: Connect to an external 5V power supply (ensure a shared ground with the Raspberry Pi).

Homework Challenge

Now that you have a working gesture recognition system linked to hardware animations, try adding your own custom gestures! Train a brand-new gesture pose (like a “three-finger count”), write a corresponding non-blocking animation function in updateNeoPixels(), and share your video results.

Here is the code we developed in the video lesson:

Homework & Next Steps

Try setting up the circuit and running the code on your Raspberry Pi 5. Observe how the frame rate behaves when driving both the camera preview, the hand-tracking model, and the I2C OLED display simultaneously. For your homework, extend this script to include a dynamic color slider or gesture-based brightness control!

AI On the Edge, Raspberry Pi

AI on the Edge LESSON 54: Custom Hand Gesture Recognition and Dynamic NeoPixel Control

Welcome back, everyone! This is Paul McWhorter from TopTechBoy.com, and welcome to Lesson 54 in our AI on the Edge series.

In this lesson, we are taking our hand gesture recognition system to the next level by connecting it directly to real physical hardware! We’re going to take the normalized vector distance calculations and MediaPipe hand-tracking models we’ve been building over the past few lessons, and use them to drive a 12-LED NeoPixel Ring via SPI on the Raspberry Pi in real time.

If you’re following along in the series, pull up a chair, get yourself a nice hot cup of iced coffee, and let’s get right into it!

What We Are Building Today

In previous lessons, we learned how to extract 3D landmark coordinates from MediaPipe, scale them by the palm length to achieve scale-invariant gesture tracking, and store those relative Euclidean distances into a pickle database (gestureNEOPIXEL.pkl).

In Lesson 54, we map those live gesture detections directly to non-blocking LED animation patterns:

  1. Fist: Pulses a dynamic red “charge” sine wave.

  2. Open Palm: Drives the NeoPixel ring at full intensity white beacon.

  3. Peace Sign (✌️): Runs a dual-spinner pattern with opposing green and blue tracer LEDs over a low background red glow.

  4. Thumbs Up: Illumination of a solid, bright green “success” status.

  5. Point: Sweeps a fast green radar chaser dot across a red background array.

  6. Rock / Metal (🤘): Generates a smooth, sinusoidal cyan-to-green energy pulse.

  7. OK Sign: Cycles through a full spectrum RGB rainbow wheel.

  8. Unknown / Default: Gracefully turns off all LEDs.

Hardware Setup & Wiring (SPI Interface)

To ensure high frame rates without flickering, we drive our NeoPixels over the SPI bus using the neopixel_spi library.

Wiring Diagram Guide

  • NeoPixel VCC: Connect to 5V power supply (ensure adequate power if driving high brightness).

  • NeoPixel GND: Common ground with the Raspberry Pi GND pin.

  • NeoPixel DIN (Data In): Connect to GPIO 10 (SPI0 MOSI) on the Raspberry Pi pinout (Physical Pin 19).

Note: Ensure SPI is enabled in your Raspberry Pi system configuration (sudo raspi-config -> Interfaces -> SPI).

⌨️ Interactive Keyboard Controls

While running the main video processing loop, you can dynamically manage your trained gestures directly from your keyboard without interrupting the camera feed:

  • SPACE: Capture and train/overwrite the current gesture showing on screen.

  • T: Prompt the terminal to type in and add a brand-new gesture name on the fly.

  • D: Prompt the terminal to remove/delete an existing gesture from the active dataset.

  • Q: Cleanly turn off all NeoPixel LEDs, save the updated gesture data, and terminate the program.

Complete Python Source Code

Below is the complete, fully commented Python script for Lesson 54.

Homework Assignment

Your homework for this lesson is to modify the updateNeoPixels() function to add two new custom hand gestures of your own creation! Try implementing custom animations like a warm candle flicker or a pulsating dual-color pattern. Leave a link to your solution videos in the comments below!

AI On the Edge, Raspberry Pi

AI on the Edge LESSON 53: Gesture Recognition on the Pi 5 with Training File

 Welcome back, everyone! In this lesson, we are taking our custom MediaPipe hand gesture recognition system to the next level. Up until now, every time you stopped your Python script, all of your hard-earned gesture training data vanished into thin air. If you wanted to run your project again, you had to re-train every single hand sign from scratch.

That ends today. We are going to make our AI remember what it learned by saving our gesture feature vectors to a local file using Python’s pickle library.

What You Will Learn in This Lesson

  • Data Persistence: How to serialize and save Python dictionaries containing landmark distance vectors directly to disk (gesture_data.pkl).

  • Automated Setup: How to check for existing training files at boot so your program automatically skips the initial training prompt if data already exists.

  • Dynamic Training Management: How to interactively train new gestures on the fly ('t' key) or delete outdated gestures ('d' key) during runtime while automatically updating the saved file.

  • Robust Error Handling: Ensuring index tracking (trainIdx) safely syncs with live gesture arrays so pressing the spacebar never crashes your video feed.

Key Concepts Explained

1. Pickling Data Objects In MediaPipe gesture recognition, a trained gesture isn’t a complex neural network weight file; it’s simply a dictionary mapping a gesture name (string) to a list of normalized keypoint ratios (floating-point numbers). Python’s pickle module allows us to convert this dictionary into a byte stream and write it directly to a file (.pkl). When the program launches, pickle.load() reconstructs the dictionary in milliseconds.

2. Managing Execution Flow on Startup When gesture_data.pkl exists, our program automatically populates gestureNames with the keys from trainedData and sets trainIdx past the end of the array. This keeps the camera pipeline running in Inference Mode immediately. If no file exists, the script seamlessly drops into Training Setup Mode to prompt the user for initial inputs.

This is the code we develop in this video lesson:

Homework Assignment

Now that your Pi 5 can remember hand gestures across reboots, it’s time to bridge software recognition with real-world hardware control!

The Challenge: Integrate an SPI NeoPixel Ring (WS2812B/SK6812) with your gesture recognition code.

  • Map distinct hand signs to dynamic LED patterns (for example: Fist = Red Pulse, Open Palm = Bright White Beacon, Peace Sign = Dual Spinner, OK Sign = Rainbow Wheel).

  • Make sure your animation updates are smooth and non-blocking so your OpenCV camera frame rate doesn’t drop!

We want to see who in the community can build the most impressive physical light demonstration controlled entirely by hand gestures. Share your video results and code modifications in the comments below!

Making The World a Better Place One High Tech Project at a Time. Enjoy!