EcoVision: Reviving an AI-Powered Mobile App for Visually Impaired Accessibility with YOLOv8

EcoVision: Reviving an AI-Powered Mobile App for Visually Impaired Accessibility with YOLOv8



Tópico: EcoVision: Reviving an AI-Powered Mobile App for Visually Impaired Accessibility with YOLOv8
Categoria: Tutoriais | Programação & Tecnologia
Idioma Principal: Português (Conteúdo de Tecnologia)

Descrição do Conteúdo / Informações:
-------------------------------------------------------------------------
This is a submission for the GitHub Finish-Up-A-Thon Challenge



What I Built


EcoVision is an AI-powered mobile application designed to enhance autonomy for individuals with visual impairments. By combining modern computer vision with an accessible user interface, the app performs real-time edge inference directly on the smartphone to identify everyday household objects.

Initially started as a prototype during my university studies, the project combined Flutter, an older AI model, and an Arduino ultrasonic sensor via Bluetooth to calculate distances. This challenge provided the ultimate motivation to revive the repository, dust off the code, and transform an unstable codebase into a highly stable, production-ready application.



Demo


You can explore the entire codebase, development history, and configuration files directly on my GitHub repository here:

GitHub Repository - EcoVision

You can also view the application in action, including screenshots and a video walkthrough demonstrating the real-time object detection and audio feedback features, in the following public folder:

Google Drive Demo Folder



The Comeback Story


The main objective of reviving EcoVision was to bridge the gap between a fragile university prototype and a robust, production-ready assistive technology solution. Here is how the project underwent a massive evolution during this hackathon:

•
Upgrading the Core Environment: The legacy prototype suffered from outdated package constraints. I completely overhauled the pubspec.yaml file, upgrading the SDK constraints to modern environment standards (>=3.3.4 <4.0.0).

•
Massive Dataset Expansion (COCO): The previous version relied on a limited custom model that only recognized 4 objects (bed, stairs, table, and door) with distance output strictly in centimeters. I migrated the engine to a high-performance YOLOv8 model optimized with the COCO dataset, expanding the app's real-time detection capabilities to dozens of everyday household items using float32 TensorFlow Lite (.tflite).

•
Voice-Activated Commands: I engineered an interactive voice command system. Users can now activate the microphone and say "buscar [object]" (search for [object]) to target specific items, a feature completely absent in the prototype.

•
Intelligent Onboarding & Interactive Instructions: Developed a dedicated home screen before the camera interface. It triggers a step-by-step audio usage guide exclusively on the first launch so it isn't intrusive. If users ever need to hear it again, they can simply use the voice command "instrucciones" (instructions) to replay it.

•
Advanced Settings & Categorization: Added a comprehensive Settings Panel where users can manually filter objects by household categories or other sections provided by the COCO dataset.

•
Hardware Diagnostics & Bluetooth Sync: Integrated live status indicators to verify if the app is connected via Bluetooth to the Arduino module, alongside a sensor diagnostic readout to monitor real-time incoming distance data.

•
Comprehensive Maker Guide & FAQ: Embedded an interactive hardware assembly manual detailing all required electronic components, circuit layouts, and the exact production-ready Arduino source code, alongside a dedicated Frequently Asked Questions (FAQ) section.

•
TalkBack Screen Reader Optimization: Most importantly, the entire application interface, buttons, and settings have been rigorously optimized to work flawlessly with Android's TalkBack screen reader, ensuring a true, production-grade barrier-free accessibility experience that the original prototype completely lacked.



My Experience with GitHub Copilot


GitHub Copilot acted as an invaluable pair programmer throughout this intense revival process.

Migrating an older Flutter project with broken dependencies can be an architectural nightmare. Copilot helped me rapidly debug dependency conflicts in the pubspec.yaml file, suggesting compatible versions for tflite_v2 and permission_handler.

Furthermore, it accelerated the integration of the float32 model inference pipeline by automatically generating boilerplate code for handling camera streams and managing asynchronous text-to-speech cues. It allowed me to focus on optimization and user experience rather than getting stuck on environment configuration.