Billaboop is a real-time beat boxing interface, developed by Amaury Hazan at Music Technology Group.
The system lets the user control drum synths, samplers, and effects with their own voice. The project mainly uses techniques of Audio Signal Processing and Machine Learning. Descriptor generation, onset/offset detection, vocal hit segregation, mapping and synthesis are achieved in real-time.
The system uses two methods of machine learning to detect and classify the incoming percussive sounds; supervised and unsupervised modes, when in supervised mode the system refers to pre-recorded samples and attempts to match the sounds of the instrument they represent. When in unsupervised mode the system sorts incoming sound into 3 groups and simulates a form of short-term memory. Although the classification results are less reliable than what is achieved in supervised mode, unsupervised mode gives more robustness when processing noisy data
Example-samples, you can hear the original percussive signals on the right channel and the BillaBoop output stream on the left channel:
Supervised Mode [mp3]
Unsupervised Mode [mp3]
