Project: Creative Apps - AI Score Transcriber

Track

Creative Apps (GitHub Copilot)

Project Name

AI Score Transcriber

GitHub Username

@Villamediana

Repository URL

https://github.com/Villamediana/AIScoreTranscriber

Project Description

AI Score Transcriber

1. What does this project do?

AI Score Transcriber is an open-source web application that converts audio into MIDI files and sheet music. Users can upload an audio file, and the system automatically transcribes the music and displays a playable score in the browser. Both the generated MIDI file and the sheet music can be downloaded for practice, editing, or sharing.

2. What problem does this project solve?

As a pianist with over 16 years of practice, there were many times when learning classical pieces or Venezuelan waltzes was difficult because sheet music didn’t exist, was hard to find, or required payment. The only option was often to listen second by second and manually figure out each note on the piano. That challenge still exists today, not only for existing songs, but also for original compositions musicians create that have no written score. While some tools attempt to generate sheet music from audio, most are paid. This project removes those barriers by providing a free and open-source way to generate MIDI and readable notation from any audio.

3. What are the key features?

• Audio input via file upload.
• Automatic polyphonic transcription using the open-source Basic Pitch AI model.
• Sheet music and piano-roll visualizations.
• Playback of original audio and MIDI preview.
• Adjustable BPM and time signature.
• Downloadable MIDI file and sheet music.

Demo Video or Screenshots

Demo video: https://www.youtube.com/watch?v=LQq88k3vV4Y
Screenshots: https://github.com/Villamediana/AIScoreTranscriber/tree/main/screenshots
Live Demo: https://aiscoretranscriber.com/

Primary Programming Language

Python

Key Technologies Used

Backend

• Python – Core programming language
• Flask – Web framework and routing
• Basic Pitch (open-source) – Polyphonic audio-to-MIDI transcription model
• ONNX Runtime – Model execution (no TensorFlow required on Windows)
• pretty_midi – MIDI processing and synthesis
• soundfile – Audio rendering
• imageio-ffmpeg (FFmpeg) – Audio conversion to WAV
• yt-dlp – YouTube audio extraction

Frontend

• HTML / CSS / JavaScript – Web interface
• VexFlow – Sheet music rendering in the browser
• Tone.js (MIDI) – MIDI playback and synthesis

Other

• Session-based result management
• Temporary file storage with automatic cleanup

Submission Type

Individual

Team Members

No response

Submission Requirements

• My project meets the track-specific challenge requirements
• My repository includes a comprehensive README.md with setup instructions
• My code does not contain hardcoded API keys or secrets
• I have included demo materials (video or screenshots)
• My project is my own work with proper attribution for any third-party code
• I agree to the Code of Conduct
• I have read and agree to the Disclaimer
• My submission does NOT contain any confidential, proprietary, or sensitive information
• I confirm I have the rights to submit this content and grant the necessary licenses

Quick Setup Summary

1. Clone the repo
2. Create a Python 3.10 virtual environment and activate it
3. Install dependencies with pip install -r requirements.txt
4. Run the app with python app.py
5. Open http://127.0.0.1:5000 in your browser

Upload an audio file, click Transcribe, then view, play, and download the generated MIDI or sheet music.

Technical Highlights

I’m really proud of this project because it’s something I always wanted to build as a musician, but never took the initiative to start. For years I didn’t even know tools like this existed or that, with the right open-source libraries, it would be possible to assemble something solid in a relatively short time. With research, persistence, and GitHub Copilot for sure hehe.

• One of the most interesting parts was learning how audio can be converted into musical notes. I discovered Spotify’s open-source Basic Pitch, which supports polyphonic transcription for any instrument or voice. I also explored Google’s MAESTRO project, which provides a large open dataset of piano recordings and MIDI (around 150GB) for training models from scratch. I didn’t train my own model due to the time we had, so I made the technical decision to use Basic Pitch for faster implementation, but training a custom model is something I’d love to explore in the future.

• Another part I’m especially proud of is the visualization layer. I didn’t want to just output a MIDI file, I wanted users to see the music. Since Basic Pitch returns note timings and pitches, I used that data to build a piano-roll visualization step by step.

Challenges & Learnings

• One of these was The sheet music rendering was even more challenging. I initially looked into libraries like music21, but ran into Python version compatibility issues because Basic Pitch requires Python 3.10. So instead, I implemented my own logic to calculate note durations (quarter notes, eighth notes, etc.), measures, clef selection based on pitch range, and render everything in the browser using VexFlow. Designing that normalization layer between raw transcription data and readable notation was technically one of the most interesting parts of the project.

What I love most is that this solves a real pain I personally experienced, and that many musician friends share. Not just for existing songs that lack sheet music, but also for original music people compose or improvise. Now it’s as simple as recording an idea and instantly generating a score.

There are still features I’d like to expand, such as enabling stable YouTube processing in production and adding direct score editing so users can correct minor AI transcription errors and export a refined version. I’m also proud of the UI and overall user experience. There are small improvements I would still make, but overall I’m genuinely satisfied with how it turned out.

Contact Information

https://www.linkedin.com/in/miguel-villamediana-3012281a3/

Country/Region

Brazil

This issue is a project submission for AI Score Transcriber rather than a scoped code task. Review the linked repository, README.md, demo materials, and the submission requirements first. Done means assessing the project against the listed challenge requirements; the issue names no specific change, file, or test for a newcomer to implement.