In May, Levin Leesemann graduated with a BS in Computer Science along with minors in Music and Mathematics. Since he is pursuing a MCS degree as part of our U2G program, he will be at IOWA this coming academic year as well.

During his senior year, he endeavored to meld his tenure on Hawkeye Marching Band Drumline¹ and academic domains of predilection; advised by CS Chair, Prof. Alberto Segre.

The result? A paper entitled: "Automated Drumline Rhythm and Instrument Recognition" summarized in this abstract:

Drumlines typically rely on sheet music or some type of non-audio record, however acquiring or producing such a resource can be an expensive and time consuming endeavor. Previous research and projects have shown that instrument and rhythm recognition can be achieved using a variety of approaches, ranging from very complex and sophisticated systems, to machine learning based approaches. This paper aims to demonstrate that the capabilities of neural networks and readily available audio analysis libraries can be used within a system to help transcribe drumline specific audio recordings. Instrument recognition and rhythm analysis are the two main problems when transcribing drumline music, which are approached by using machine learning and a logical approach, respectively. The system shows that a neural network is capable of recognizing drumline instruments from short audio excerpts. It also shows that accurately identifying common rhythms is attainable using a simple system that can be expanded to interpret more complex rhythms.

In conjunction with his paper, Levin has shared spectrogram images corresponding to roughly 2 seconds worth of snare drum and tenor drum recordings, along with a few pointers:

• The lighter in color a point is, the more volume there is for that frequency at that point in time
• The tenor hits show more distinct frequencies for each hit, which can be attributed to the different over/undertones that the tenor drums inherently produce
• The lower frequencies (below 128 hertz) in the tenor recording are likely background noise and aren't that useful for evaluation
• The program only takes 50 milliseconds of each hit into consideration, meaning that a lot of the visible information isn't ultimately used
• The spectrograms were generated using Librosa (the same library used for the project)

Leesemann was also kind-enough to answer a few questions about this project and more:

What – personally or academically – prompted you to investigate using neural networks to help transcribe drumline audio?

Going all the way back to my early drumming career, when I had just picked up marching percussion and spent most of my time playing drum set, having an automatic drum recording transcription tool seemed like a very exciting and useful idea to me. While brainstorming ideas for my undergraduate computer science honors project, the concept of using neural networks to transcribe drum recordings seemed to be the perfect opportunity to explore this. Having gained much more musical experience as a consequence being part of the Hawkeye Drumline for multiple years and pursuing my music minor as an undergraduate, I felt confident that I could make a program demonstrating the capabilities needed for a piece of software I had always imagined.

How did this “for credit” independent research differ from other academic or professional endeavors of yours?

Being able to personally explore a project without a strict rubric or rough outline set by an instructor or manager was both liberating and a bit overwhelming. There are many libraries, research papers, and small scale examples that helped me develop my project, but to end up with the results I did require much more research than I am accustomed to in academic/professional contexts. Because of this however, I was able to expand my knowledge in a lot of different areas, resulting in it being one of the most insightful and helpful learning experiences I've had to date. It showed me how to apply many skills I had acquired prior to this project into one coherent project that I could take wherever I'd like.

Do you see yourself following up on “Future Work” mentioned and/or applying lessons learned towards other projects (e.g., internship, project “Part 2”)?

Transforming this project into a fully-fledged application with features outlined in the "Future Work" section of the paper is definitely a goal of mine, although it will likely take some time to achieve such a goal. Seeing how the project can be expanded to other types of instruments is also interesting to me, however it will require a different kind of approach than the one used in this project. Applying what I've learned as a whole during this project to other work is something I can see myself doing in the near future though, particularly with regards to professional endeavors.

Congrats on your recent Bachelor’s degree! As a U2G student, you have one more year towards an MCS: any advice for current and prospective students, as informed by your own path at Iowa?

Don't be afraid to jump in at the deep end. Admittedly, prior to this project I had no experience with neural networks. To me this demonstrates that there is always a way to achieve the goals you set for yourself, even if the goals have to change along the way as you become more familiar with the things you are learning. Although it can seem very intimidating and overwhelming at first when exploring something unfamiliar, taking advantage of resources and not being afraid to ask can be of great help when pursuing projects of any kind.

Undergraduate students interested in signing up for their own Independent Research Study should talk to their faculty of choice (based on past interactions or research interests), submit this Independent Study Contract then head to MyUI to register, when authorized.

Other options for undergraduates to delve deeper into a topic include the Iowa Center for Research by Undergraduates and pursuing Honors in the major. Labs also hire undergrads from time to time.

¹ 9/10/21 Iowa vs. Penn State - Pregame - Center Tenor Cam - Hawkeye Drumline [Leesemann POV]