When Skiddosaurus Rex said she wanted to her original song on a music box someday, I thought this would be a perfect community Christmas project.'
My first idea was to see if I could make a real music box play the song, so I picked up a couple from Amazon. I then used a pitch meter to see what notes were available on it's comb. It wasn't a good mix. Never the less, I had a musician friend of mine translate the songs notes into the scale that was available on the combs I had, and then 3d printed a new cylinder. It actually sorta worked, but sounded pretty scuffed. Turns out, every music box has a unique comb specially made for the song it plays. Further, the teeth on the comb often have duplicate notes if the song needs to play one note in quick succession. It's not really feasible make custom music box drums without also making a custom drum.
I decided a new idea was needed, and thought about making it as a web experience using my ThreeJS skills.
The plan was to model her mascot, Scout, as a wooden music box, and then make a programmable 3d music box mechanism that can dynamically play MIDIs when opened. As the box plays, messages from community members would slowly appear on the table, written on snow flake post-it notes.
I after I got the models sorted out, I needed a way to get it playing music. I found the Tone.js library and it was a perfect fit - you can set up virtual sample-based instruments with it, so I found a C3 music box wave online and used it to make an instrument that sounds just like a music box.
I didn't want to deal with reading MIDI directly, so I asked ChatGPT to give me a script that could covert MIDI Files to a custom JSON format. This worked really well. I was able to write code to both play the music nodes via the Tone.js library, as well as dynamically spawn pegs on the music box 3d model that match the notes.
To make the prongs work, I created a blend-shape (shape key / morph target) for each of the notes, where 0 = fully down flat and 1 = bent up backwards.
I was able to use the same code timer that played the music notes to also spin the drum and push the pins up from 0.0 to 1.0 right before they "snap" back down to 0 and the note is fired.
The effect is really convincing - I can convert any MIDI file to my custom format and the music box will not only render the correct pins on the drum, but also rotate, push the pins back, and play the correct notes!
One thing to note, however, is that this experience is not 100% realistic - while my conversion script does limit the total number of unique notes available for a song, it doesn't limit the frequency that they can fire or the length of the song. So, while this experience could play 100% realistic music box scores, it can also play songs that would be too long or too complex to play on a real music box.
This was the unfortunate reality of her main original song - the main recognizable part of the hook was pretty complex, and could not easily play on a real music box. My site, however, provides a very convincing experience however.
The last step was to add some polish. I added some interaction so the first thing the user does is turn a mysterious crank before the reveal that it's a music box. I added a cozy window scene with dynamic snow outside. When I had a prototype working, I started messaging all the community members with a Google form to fill out their thoughts and wishes. Once I had plenty gathered, I stuck them in a JSON file and had them start appearing around the table, written on snowflake post-it notes.
You can check out the live interactive experience here:
https://orokro.github.io/skiddo-xmas/
Or check out the code on github here:
https://github.com/orokro/skiddo-xmas/tree/master
Or just check out the screenshots below!
