West slope landscapes

Toggle navigation
West Slope
Introduction
High level design
Program Design
Hardware Design
Results
Conclusions
Appendix
West Slope landscapes
Simulation of procedually generated tree growth on RP2040
ECE 5730 Final Project, Fall 2022
By Zechen Wang, Ziqing Liang, Xinyue Li
Demonstration Video
Introduction
Our project, west slope, is a combination of visual and audio applications, simulating beautiful landscapes of Conrell west slope on the VGA screen and taking music as inputs to generate trees with different colors and shapes.
The idea of our project came from the impact of music on plants, which is playing music helps the plants grow faster. Though scientists are still arguing about whether this theory is true or not, we were inspired to play different music to plants and generate according features on screen, combining visual and auditory senses to achieve a beautiful and immersive experience. Also, driven by the affection for our gorgeous campus, we integrated the trees with characteristics of the west slope to promote the beautiful sceneries of our Ithaca campus.
Our project has modified the Fast Fourier Transform and Barnsley fern codes from course demos and imitated the code structure of the Lindenmayer system from Github. The music we played was from Youtube. Reference details are listed in the appendix section.
High Level Design
Logical Structure
The west slope project uses a RP2040 microcontroller as the main host, a VGA screen to display and a microphone to receive audio signals. Our project consists of three major functional blocks:
(1) the Lindenmayer system tree generation (2) the Barnsley fern tree generation (3) the Fast Fourier Transform calculation.
We generated trees based on the Barnsley fern and Lindenmayer system algorithm. We did modifications to the generation procedure of barnsley ferns: first generate a template for all leaves and perform transformations for multiple iterations to complete a tree.
Instead of appearing on the screen all at once, the modification realized beautiful-looking leaf-by-leaf animation. Moreover, in order to increase diversity and elegance, we added randomness to the matrix parameters to create diverse shapes of the Barnsley fern,
and implemented three different rules to generate multiple types of L-system trees.
We processed the audio input based on the FFT algorithm that we implemented in lab 1 to obtain frequency of the music and changed the tree growth speed according to it.
We implemented signal processing to reduce noise in the collected signal, using a moving average buffer and intensity thresholding. In that way environmental noises would not be counted as valid inputs,
and sudden sound impulses are also smoothed to avoid drastic changes in tree growth speeds. We utilized frequent context switching between the threads to update the growth speed in real time.
Background Math
Fast Fourier Transform
The Fast Fourier Transform algorithm converts signals from the time domain into the frequency domain. The audio signal is sampled over a period of time and divided into its frequency components that are single sinusoidal oscillations at distinct frequencies with certain amplitude and phase.
There are two important parameters related to the FFT algorithm: sampling rate and FFT size. Sample rate is the amount of time between each value in the input. According to Nyquist's sampling theorem, for a given frequency, the sample rate is required to be twice as large as the given frequency.
We define the sampling rate as 10kHz, as the upper limit of the music frequency we played was 2kHz. FFT size is the number of output frequency bins of the FFT, indicating the number of input samples needed to calculate the FFT. We set the number of samples per FFT as 1024.
Barnsley fern
The Barnsley fern is an iterated function system, which is generated using an iterations function system. The system consists of several sets of equations, each of which represents a rotation, a translation, and a scaling. The equations are as follows.
The parameters A, B, C, D represent a rotation and the magnitudes of them represent the scaling. The parameters E and F represent the linear translation of the point. Also, fewer points are necessary on some parts than others.
For example, the stem does not need as many points as the leaf and we want the majority of the points to generate the leaf. Therefore, each function is also given a probability of being chosen. The parameters of a classic barnsley fern are as follows.
Lindenmayer system
The Lindenmayer system exploits self-similarity and recursive procedure, applying a set of rules iteratively to model the growth processes of plant development. On each iteration, a string of characters is rewritten according to the replacement rules.
Each symbol is given a graphical meaning. ‘F’ means move forward and draw a line. ‘-’ means turn left at a certain angle. ‘+’ means turn right at a certain angle. ‘[’means push state. ‘]’ means pop state. The rules that we adopted in the project are as follows.
Program Design
A high-level description of the program
The figure above shows the high-level organization of our software design. Since we want to draw trees that are generated from the Barnsley Fern algorithm and L-system algorithm simultaneously,
we divide the implementation of these two algorithms into two threads. And we put the FFT part into another independent thread. We run the FFT and Barnsley Fern thread on core 1 and the L-system thread on core 0.
One of the tricky parts that we need to deal with when combining different threads is that we want to synchronize drawing the trees of different algorithms as much as possible. It is not necessary to force the trees from different threads to end together,
but it is important that trees from different threads start simultaneously when a new scene starts. The blue color descriptions in the figure show how we achieve this by setting up two signal variables. Each thread waits for the “finish signal” from the other thread in order to start together for the next scene.
Another tricky part is the bouncing between the FFT thread and the Barnsley Fern thread on core1. During the growth of the tree, we want the speed of growth to dynamically adjust to the max frequency of the music, which needs an appropriate design on when to yield the thread to the other.
The figure above shows our design using red color descriptions and red and black arrows. The arrow loop in red shows we implement FFT after each pair of leaves is drawn.
The arrow loop in black shows that we still need to implement FFT if the Barnsley fern trees finish before the L-system trees so that the L-system trees growing speed can still match the change of frequency of the music. All the variables determining the position and shape of the Barnsley Fern trees have to be static so that the value will not be lost when the thread yields.
In the following part, our program design for FFT, Barnsley Fern, and L-system will be described.
FFT Thread
This thread mainly carries out the following processes:
DMA channel 2, paced by the data request signal from ADC, reads data from the ADC FIFO when it is full and writes to the 1024 bytes ADC sample array;
When the sample array is filled, move the data to an FFT array where the FFT is implemented;
After finishing moving the data to the FFT array, start DMA channel 3 to reconfigure and restart DMA channel 2 to implement step 1), and meanwhile, implementing FFT using the FFT array;
Calculate the max frequency among these samples.
We notice that because noise occurs, the max frequency sometimes changes rapidly, which affects the growth of the trees. We implement signal processing to reduce noise in the collected signal by using a moving average buffer and intensity thresholding. In this way, environmental noises would not be counted as valid inputs and sudden sound impulses are also smoothed to avoid drastic changes in tree growth speeds.
Program related to L-system
In this part of the program, we implement the algorithm of the L-system to generate trees. We refer to the code from https://github.com/telephil9/lsystem/blob/master/lsystem.c.
Based on the l-system algorithm, trees are generated following the command of a string. Each character in that command string determines the action of the tree growth, for example, either rotate an angle or go forward. We need to generate the string before drawing the trees.
There are different types of trees, and each type is generated by one or more rules which determine what character needs to be substituted by what kind of string. We follow the rules to get the next generation of the string from the current string.
We use Struct in C to store a single rule, and one rule will point to another rule if that type of tree needs multiple rules to generate. We write the rules for all types of trees and at the end of every scene, we use a random number to decide which
type of tree will be drawn in the next scene. After the type of tree is decided, the rules will be updated accordingly.
Also, we implement a push-pop data structure in our program. Whenever we need to store the current state (the position or the angle), we need to malloc a space in the heap to push the state. Also, the heap space needs to be freed whenever we pop the top state.
In the while 1 loop in the thread, we:
Generate the command string based on the rules and iteration numbers.
Draw the tree based on the command string. The sleep time between drawing the lines will change according to the max frequency.
Update parameters (color, length of the line, and rules) for the next tree
Wait for the signal from the Barnsley Fern thread
The tricky parts in implementing the L-system contain deal