Musical chord identifier

ECE 4760 Final Project: Chord Identifier
¶
For our final project, we created a chord identifier using the Raspberry Pi Pico RP2040 microcontroller that uses frequency analysis of a microphone's audio signal to determine the chord played from an instrument and displays the result on a VGA screen and PuTTY serial interface.
Demo Video
¶
Introduction
¶
The aim of our final project was to design a chord identifier. Our project combined our knowledge from previous ECE courses such as circuits, signal analysis, and digital logic with our experience from past labs in this course. Throughout the four weeks we worked on this project, we were able to use an iterative design process and improve upon the initial design week by week.
First, we implemented a note identifier. We set up the microphone the same way as in Lab 1 and played pure tone frequencies. Using the FFT code provided from Lab 1, we performed an FFT on the audio sample and determined the maximum frequency from the FFT. From the maximum frequency, we used a formula to identify the note name and scale (e.g. A4, D3, etc.). We output the maximum frequency and note name on the serial interface.
Building upon the note identifier, we used the same microphone and serial interface setup, and rather than looking for the single maximum frequency, we identified the three highest frequencies from the FFT. These frequencies use the note identifier to label the three most powerful notes. Applying the foundations of music theory, we determine which chord is played; we distinguish major, minor, augmented, and diminished chords as well as triads.
Once the basic chord identification functionality was implemented, the next step was to improve performance by implementing a low pass filter to limit the range of the frequencies picked up by the microphone. This filter allowed us to change the parameters of the sampling for our FFT algorithm, which increased the efficiency and accuracy of our chord identification. Finally, a Protothreads C program was implemented to make use of the two cores on the RP2040. The serial interface and VGA were implemented on one core, and the FFT was implemented on the other.
High Level Design
¶
Rationale for Project Idea
¶
Originally we wanted to build an instrument identifier, due to our shared interest in music. If you play an A4 on a piano and an A4 on a guitar, you are playing the same note, but you can tell which note comes from a piano and which note comes from a guitar. This distinction is from a musical property called timbre, akin to tone color or tone quality. Words like bright, dull, brassy, reedy, harsh, and soft are used to describe timbre; while this property seems largely abstract and hard to quantify, these unique timbre characteristics can be measured by looking at the frequency spectra. When a piano and guitar play an A4, the highest frequency term is the A4 note; however, due to the shape of the instrument, there are other resonant frequencies. These other harmonics define an instrument's timbre. We wanted to build an instrument identifier that analyzed each instruments’ frequency spectra (collected by an FFT) to predict which instrument was being played, a task which requires some amount of machine learning. There are many data sets available online that match an audio file to an instrument, however to train our system, we needed data sets with FFT data matched to an instrument, and processing each audio file through the Pico's FFT is unfeasible.
Ultimately, we decided to change projects, but keep the musical and signal processing flavors. Instead, we opted to implement a chord identifier to keep the musical theme.
Background Math
¶
Implementing and understanding our chord identifier requires understanding some background math including frequency-to-note conversion, an FFT, and a Sallen-Key low pass filter.
Frequency-to-note Conversion
When an A4 is tuned to 440 Hz, the equation to convert a frequency (in Hz) to a note number is as follows:
From the note number, a lookup table is used to convert note number to note name.
Fast Fourier Transform
We used a Fast Fourier Transform (FFT) to quickly collect frequency information. One of the main concepts in signal processing is that any periodic signal can be represented as a sum of sinusoids by the Fourier Series Approximation. By Euler’s Formula, we can represent any sinusoid as a complex exponential: both of which are defined by a magnitude and a direction.
By the Shannon-Nyquist Sampling Theorem, any analog signal can be digitally sampled and reconstructed as long as the sampling rate is greater than twice the highest frequency term. In Signal Processing, analog signals are typically thought of being continuous, whereas digital signals are thought of being discrete. Using both of these facts together we can take an analog signal, sample it ‘fast enough,’ and get a sum of digital sinusoids. Note that the Fourier Series is an infinite sum of sinusoids.
Rather than give a computer an infinite sum (and infinite number of computations), we instead find a finite number of computations that would be a ‘good enough’ approximation, namely 2N-1. So rather than computing the Discrete-Time Fourier Transform (DTFT), we can compute the Discrete Fourier Transform (DFT). The DFT algorithm is very useful, but also very slow; so the Fast Fourier Transform (FFT) was created to optimize it. The FFT recursively splits the sum of discrete sinusoids into smaller and smaller components. If the number of samples is a power of two, then we can simplify the DFT all the way to summations of length one. In the case where the number of samples isn’t a power of 2, we can zero-extend any signal to make its samples a power of 2. For example if we have a signal that has 253 terms in it, two terms with values of zero can be added to the signal. This zero-padding does not change the signal in any way, but allows us to use the optimization algorithm. In summary, the FFT algorithm is used to represent a discretely-sampled analog signal; it is accurate enough to represent any signal, but uses significantly less computational power than the DTFT or the DFT.
Low Pass Filter Using Sallen-Key Topology
In general, a low pass filter will allow signals with frequencies below some cut off to pass through the system and attenuates signals with frequencies that are above some cutoff. This cutoff frequency (f0) is defined by the poles of the transfer function. When designing an amplifier, the transfer function is used to relate the output signal of the system to the input signal. For the Sallen-Key filter design, the corner frequency is defined as follows:
where R1 and R2 are resistor values in Ohms and C1 and C2 are capacitor values in Farads.
Hardware and Software Tradeoffs
¶
Originally, the hardware for this design was fairly simple: the output of the microphone was directly connected to an ADC pin on the Pico. However, due to the broad spectrum of frequencies detected, the initial design resulted in a large FFT computation which was very slow. To improve the speed of the FFT computation, a complex hardware design was added to the design. We chose to implement a Sallen-Key low pass filter through a unity gain non-inverting amplifier, which eliminated the higher frequency terms, which in turn reduced the number of frequency samples inputted into the Pico. By eliminating the higher frequency terms, we could also decrease our sampling rate from 10kHz to 5kHz. This change allowed the FFT to have increased precision because there were less possible frequencies per bin, allowing notes to be more accurately found. If the sampling rate was decreased without implementing a low pass filter, it would have resulted in an aliasing of the higher frequency terms, which would have severely impacted the accuracy of the chord identifier. Ultimately, this tradeoff was necessary to improve our chord identifier by making it faster and more accurate.
Existing Patents
¶
There exists a
patented Chord Identifier
; however, it is quite different and much more robust than our design. Their patent is for a “computer implemented method, a computer system, and a User Experience (UX) Interface.” Their computer system has a touch screen device, which we do not have. Their patent identifies chords using digital audio files which is quite different from our design which takes audio samples in real time from a microphone. Also, their UX is highly interactive and configurable whereas our UX consists of displaying information on a VGA screen and a PuTTY Serial Interface. They designed a software application which determines which key the chords are played in and stores the chord data it collects so the user can later access it. Our VGA display only displays the chord identified. Our PuTTY Serial Interface displays the three frequencies with the highest power and their corresponding note numbers and notes names. The design described in the patent has far more features than our design and is implemented completely differently.
Hardware Design
¶
The hardware design of the chord identifier is twofold; first, there is the detection of audio from the environment, and secondly, there is the display of the chord identified. To detect audio, we use the Electret Microphone Amplifier, and pass the signal through a low pass filter, which is then connected to an ADC (analog-to-digital) GPIO pin on the RP2040. To display the chord identified, we provide two displays, a VGA screen, which shows the chord identified, and a UART serial interface that displays the three maximum amplitude frequencies detected by the FFT, the note names, and the chord identified. A diagram of the final hardware design is shown below.
Audio Detection and Filtering
¶
Audio input is gathered from a microphone connected to a GPIO pin assigned to analog-to-digital conversion. This pin is controlled by a DMA channel, which samples the converted