Guitar Finguitar ?

Finguitar – This is an online instrument which you can play using just your fingers. CodeOdyssey Hack-a-thon || Team NeoStar Aaghadi
This web-app is called Finguitar (Finger-Guitar). It uses Computer Vision to build an online instrument which you can play using just your fingers. You can hold up a gesture with your hand and the web-app will record it using your webcam and play a music note for every gesture! The idea is to then give the user the ability to combine these gestures to make a tune out of it. So, it’s like playing a guitar… but using Computer Vision! Currently we recognize 10 gestures to play different sounds:

  • Palm
  • Fist
  • Thumb
  • Index
  • Middle 2 fingers
  • Pinky
  • Call
  • Rock
  • LShape(finger+thumb)
  • Ok

?Tech-Stack

Front-end:

  • ReactJS

Back-end:

  • Flask

Computer vision:

  • OpenCV

?Communication b/w the web-app and the model:

The communication is carried out on a WebSocket endpoint run by the server. The web-app uses the browser’s Stream API to emit the current frame encoded as a base64 string via the WebSocket. The server receives the string, decodes it and runs the OpenCV model on it. The gesture recognized is then relayed back on the same endpoint as a response.

WebSockets

?Further additions:

  • Host the web-app and the model
  • Improve the animations on the web-app
  • Add a gamified experience for the user, and provide him with a score as a feedback
  • Add a tutorial run, where the user can first train on the set of gestures and get familiar with the sounds.

?‍?Team Members:

GitHub

View Github