Media Engineering project
Droneverse explored whether small drones, helium balloons and interactive control modes could become a playful Media Engineering experience: a drone course that visitors could navigate without a classic controller.
The project brief started with “interactive drones with balloons”. My focus became the software architecture behind a tangible drone experience: gesture control, phone motion input and a safe way to test commands before sending them to hardware.
Early experiments with C++ in Ubuntu and React/TensorFlow were useful, but not practical enough for the presentation setup. Python, OpenCV and MediaPipe became the strongest base, connected through modular detection and controller components.
The final prototype used a small UI to select input and controller modes, filtered noisy direction signals, checked for blocked movement and then routed commands to print output, a Tello drone or the quadcopter controller.
The prototype connects detection and control as interchangeable modules. Gesture, OSC and keyboard modules all pass directions back to the central application through callbacks. There, the most frequent recent direction is selected, checked against collision data and sent to the chosen controller.
Print was the debugging controller: instead of flying, the software wrote the selected direction to the console. That made it possible to test the interaction logic safely when no drone was connected.
OSC used live gyroscope data from the Data OSC iPhone app. After discovering the exact message paths, the software averaged recent X, Y and Z values and translated tilting and rotating the phone into flight directions.
Gesture control used webcam input, OpenCV and MediaPipe hands. Finger regions controlled up and down, thumb-to-index distance controlled forward and backward, and horizontal finger position controlled left and right.
During the final presentation, gesture control became the audience favorite. It was intuitive enough for visitors to overcome at least one obstacle after a short practice phase, even though forward/backward thresholds still depended on hand size and camera distance.
