How to Build a Real-Time Object Detection and Tracking Pipeline with ROS 2 and YOLOv11

                     How to Build a Real-Time Object Detection and Tracking Pipeline with ROS 2 and YOLOv11
               



Tópico:
                     How to Build a Real-Time Object Detection and Tracking Pipeline with ROS 2 and YOLOv11
               
Categoria: Tutoriais | FreeCodeCamp Premium
Idioma Principal: Português (Conteúdo de Tecnologia)

Conteúdo do Tutorial / Guia Passo a Passo:
-------------------------------------------------------------------------
If you've ever tried to build a robotics system that can actually see, track, and respond to the world around it, you know that the hard part isn't training a detection model. The hard part is making that model run reliably inside a real robotic software stack, in real time, without falling apart the moment hardware constraints or timing issues enter the picture.

In this tutorial, you'll build a complete real-time object detection and tracking pipeline using ROS 2 and YOLOv11. You'll learn how to publish camera frames from a simulator into ROS 2, run YOLO inference in a dedicated thread, integrate ByteTrack for multi-object tracking across frames, and export your model to ONNX for faster inference on constrained hardware.

By the end of this article, you'll understand not just how to wire these tools together, but why each architectural decision matters for a perception system that's meant to work in production, not just in a notebook.

Here's what we will cover:

Table of Contents

• Prerequisites

• What We Are Building and Why

• Project Structure

• How to Set Up Your ROS 2 Workspace

• How to Install Dependencies

• How to Publish Camera Frames into ROS 2

• How to Build the Perception Node with Threaded Inference

• How to Integrate ByteTrack for Multi-Object Tracking

• How to Add a Confidence Validation Layer

• How to Export Your Model to ONNX for Edge Deployment

• How to Test the Full Pipeline

• Conclusion

Prerequisites

Before following along, you should be comfortable with the following:

• Python 3.10 or later: All code in this tutorial is written in Python.

• Basic ROS 2 knowledge: You should understand what nodes, topics, publishers, and subscribers are. If you're new to ROS 2, the official ROS 2 documentation is a good starting point.

• Familiarity with PyTorch and object detection concepts: You don't need to have trained a YOLO model yourself, but you should understand what inference means and what a bounding box detection output looks like.

• A working ROS 2 Humble installation on Ubuntu 22.04.

• A GPU is recommended for real-time inference, though the pipeline will still run on CPU at reduced frame rates.

• CARLA Simulator (optional): The camera publisher section uses CARLA. If you don't have CARLA installed, you can substitute any ROS 2-compatible camera source, such as a webcam node or a bag file playback.

What We Are Building and Why

A perception pipeline is the part of a robotic system responsible for understanding what's in the environment around the robot. It takes raw sensor data, usually camera frames, and converts them into structured information: where objects are, what they are, and how they're moving.

This tutorial builds a perception pipeline with four distinct layers:

• Camera ingestion captures raw image frames from a simulator and publishes them as ROS 2 messages so the rest of the robotics stack can consume them.

• Object detection runs YOLOv11 on each incoming frame to identify objects and their locations with confidence scores.

• Multi-object tracking uses ByteTrack to associate detections across frames, giving each object a stable identity over time rather than treating every frame as a fresh scene.

• Validation and optimisation adds a confidence gating layer that prevents low-quality detections from reaching downstream navigation logic, and exports the model to ONNX for faster inference on edge hardware.

We

... [O tutorial continua no link abaixo] ...