Multimodal AI – Honors Academies 2027

← Honors Academies 2027 | Course offered at: Brazil

Course Available in Sao Paulo

Multimodal AI: Build an Acoustic Camera

In just one week, students will build a wearable device that shows where sounds come from on live video and uses artificial intelligence to recognize what is making them.

Smart glasses are becoming the next generation of computers, and researchers are teaching them to hear the way humans do: detecting a sound, identifying its direction, and understanding its source. In this hands-on course, students will explore this technology by building their own acoustic camera from scratch.

Working in small teams, students will combine a camera and four microphones with a pocket-sized computer. Using Python, they will transform differences in sound arrival times into a visual map displayed over live video. They will then train an AI model to identify sounds and connect them to the objects producing them. This combination of hearing and vision is known as multimodal artificial intelligence.

Sao Paulo, Brazil January 18 – 22, 2027
Schedule 9:00 am – 3:30 pm, Mon–Fri

What You'll Learn

Real-World Applications

Medicine & Health: Hearing aids and smart glasses that help people hear or understand conversations.

Biology & Environment: Detecting and tracking wildlife through their sounds.

Cars & Robotics: Vehicles that detect sirens and robots that respond to voices.

Music, Film & Games: Spatial audio for immersive experiences.

Engineering: Acoustic cameras that detect gas leaks and machinery problems.

Technology & Business: Exploring the emerging field of smart glasses.

What Students Take Away

Real Coding: Practical Python programming with live audio and video.

Applied Mathematics: Using geometry and trigonometry to locate sounds.

Understanding AI: Training models and evaluating their performance.

Engineering Skills: Building, testing, troubleshooting and collaborating.

A Working Project: A completed device and personalized instructor evaluation.

Tools to Continue: Their code and a parts list to recreate the project at home.

Iran R. Roman, Multimodal AI instructor

Iran R. Roman

PhD Stanford University and Assistant Professor of Artificial Intelligence, Queen Mary University of London

Iran R. Roman is an Assistant Professor of Artificial Intelligence at Queen Mary University of London, where he develops technology for smart glasses through a research partnership with Meta. He holds a PhD from Stanford University and has worked on artificial intelligence at Apple and Tesla. He has also taught AI to high-school students at Stanford and with Eduexplora. Originally from Uriangato, Mexico, he brings academic research and industry experience into the classroom.

Ready to Join This Course?

Apply