
As a piano player, I often find reading traditional sheet music time-consuming and prefer visual piano rolls. Since not every song has a piano roll tutorial available online, I wanted to build a tool that could instantly convert any sheet music into an interactive piano roll. Adding built-in speed controls allows me to slow down complex sections, making it much easier to learn new music directly at my piano. Additionally, for quite a while, I wanted to revisit my original "Genies" project from 2024 to build a "v2". I decided to start a new series of standalone applications under the "Genie" brand, where all my AI and computer vision projects will be categorized. That is why the project name starts with "Genie:", while "Optiroll" is a blend of "Optical" (from Optical Music Recognition) and "Roll" (from Piano Roll).
I decided to take on this project entirely on my own. As the sole developer, I handled the full development lifecycle, from designing the UI/UX and coding the React frontend to building the FastAPI backend and integrating the complex computer vision pipeline.
Project Gallery

Drawing from my past computer vision experience, my initial plan was to build a custom note-detection algorithm from scratch using OpenCV. However, I quickly realized the immense complexity, edge cases, and strict formatting rules required to accurately parse standard music notation. I pivoted my strategy and researched Optical Music Recognition (OMR), ultimately opting for oemer, a state-of-the-art pre-trained model. I built a FastAPI backend that runs oemer on uploaded images to generate a MusicXML file, then uses muspy to extract precise MIDI note events (pitch, time, duration). On the frontend, I developed a custom HTML Canvas to render the falling bars and integrated the smplr Web Audio library to provide realistic, controllable instrument playback directly in the browser.
More Projects