Assignment

Software timekeeper. Implement a working timekeeper in code. This can be in the language of your choice. You might attempt to pay homage to a timekeeper you’ve seen elsewhere, or from your design analysis; or create a novel one. Consider how you create something that changes over time – do you use the current frame, a time function such as hours(), a time object, or some other method?

Screenshot of the p5.js sketch showing the particles that are drawn with each finger movement across the screen and instructions for how to play the instrument.

🖍️ Sketch: I recommend it working best for now using a browser on a desktop or laptop device vs mobile, currently I’ve been testing using Google Chrome so there might be bugs on other browsers as well.

What I Did

For this assignment I built off of previous web synth to add a toggle drum beat.

• 🕰️ The time keeper itself is handled using Tone.js’ Transport system which handles beats per minute, controls the start and stop of loops, and is recommended to use for accurate time keeping and performance vs JavaScript callbacks or setInterval.
  • Tone.js is built on top of the Web Audio API allowing audio to be played directly in the browser and is what Transport is built off for timing.
• 🥁 I set the BPM for now as a constant at 80 as that is on the slower side of an average resting human heart rate from what I can tell but would love to do more research here and allow the user to control the BPM using a gesture and using the face detection model so it can be controlled at the same time as playing.
  • If I were to want to use this as more of a timer I could set the beats per minute to 60 which would be equal to one beat every second that passes
• 🔉The synth oscillators used are provided by Tone.js I used different ones for the first hand, the second hand, and the toggleable drum loop.
• 📷 The sketch uses a hand pose detection model through ml5.js to analyze image data captured by your devices camera.
  • The finger key points estimations are used to trigger different notes based off of position across the screen.
  • Small white dots are also displayed using p5.js to the canvas and constantly added to the canvas that tracks the fingers along with text instructions.

More details about the process below as well as some thoughts from my inspiration break in the middle of coding!

Process

I knew I wanted to experiment with the following topics: randomness, physical movement, perception of time, music, visuals and cycles, meditation. So I set a 5 minute timer and did a brain dump/ mind map around those topics and what I might want to make.

My 5 minute software timekeeper mind map on paper

I decided to make a musical time keeper that uses body movement to help anyone make a beat. This gave me enough direction to decide I wanted to build a web metronome and drum machine that you can control with body motions.

Technology and References

This summer I started working as a part of the team at ITP working on the ml5.js library and have been experimenting with it a lot for other projects ITP as well so decided to stick with that direction for this software project. I figured a good first working MVP to work on would be a metronome and then I could work on adding the controls for it through ml5.js.