Programming Projects
These are some of the projects that Mr. Carballo has been developing during his career. Most part of them are own initiative and are Open Source Code.
FiMAN: A computer system for Finger Motion Analysis
A computer system that allows capturing, recording, analyzing and visualizing finger and hand motion in 3D Polygon Mesh in real-time using a Leap Motion sensor and Java SDK.
FiMAN was extended as a tool for musical training, namely to evaluate certain aspects of piano technique through a virtual piano keyboard with a sound module. The real life usage of FiMAN, demonstrated its modularity and extensibility for its applicability in different areas related to motion analysis.
SpectraUPB App: Smartphone Spectrometer
Developed by CINTI, CIOE and CIPI reasearch centers, the SpectraUPB App allows to calibrate the spectrometer in function of the wavelength (using a flourescent lamp) and to obtain spectra in real time, using the camera of an Android Smarphone. The spectra are obtained through a DIY 3D printed mini-spectrometer, that is clipped to the Smartphone.
A Prima Vista: VR Rhythm Game
A VR Educational/Rhythm Single Player game for Oculus Quest 2 headset that teaches how to sight-read music. The development of this game involves the conversion of MIDI files to a VR Music Score and the creation of a functional VR Keyboard using Unity3D.
MUSIC TO COLORS: VR Synesthesia Experience
“Music to Colors” is an app developed for Oculust Quest 2 that converts music MIDI files into colors in a VR environment. The user is able to listen to MIDI music and at the same time visualize the representation of the sound converted into 3D objects/effects with colors related to the notes played.
Music Reading Training Web Page Using Minimax
Simple web page developed as a tool that helps in the basic reading music training process using the decision rule Minimax and it was programmed/scripted in JavaScript following the concept of music notation flashcards.
Handwritten Music Symbol Recognition - Convolutional Neural Network
convolutional Neural Network (in Python) that generates models to recognize 32 isolated different handwritten music symbols with good level of accuracy. Once the models were produced, they were evaluated with self generated images, giving promising results that might be used in future research.