ECE5725 Project Sand Flow

Introduction

Short description of what was done for the project. The work team has done for the sand flow system is to simulate the motion of sand to realize the sand matrix game on PiTFT screen. The piTFT screen is hand held by users so that users are able to control the flowing direction of the sand by turning around the PiTFT screen. The final project can be divided into two parts: hardware and software. The hardware part is to realize gravity sensitivity by using the accelerometer BNO055 from Adafruit. It is used to detect the angles how much the piTFT screen has been turned and rotated. The platform used is the Raspberry pi and piTFT to display the game. For the software part, team implements all three modes which include sand flowing with barrier, nozzle and swipe to flow based on the algorithm of cellular automation. Also, the color of the sand and the specific nozzle chosen for releasing sand are controlled by the GPIO buttons on the board.

Project Objective:

  • For the final project, team aims to build a sand flow system that allows users to interact with the game with excellent visual experience which helps players become mentally relaxed and comfortable. The different modes of the flowing sand enables players to have different visual experience and create their own beautiful sand.

Design and Testing

      Hardware:

For the hardware part, the first step was to choose an accelerometer, which was used to realize the gravity effect of sand flowing, since the accelerometer could read the data in x, y, z direction. The final type of accelerometer team chose was BNO055 because there are enough resources and reference about BNO055 online. Also the price of the BNO055 is not reasonable, so team finally decided to use BNO55 as the accelerometer. A picture of BNO055 accelerometer is shown below.

Figure 1. BNO055 accelerometer

After reading the reference in detail, team began to wire the accelerometer with the raspberry pi following the circuit diagram as shown below.

Figure 2. sketch of the circuit diagram

After wiring the circuit correctly, team welded it on the breadboard to fix it. Then team needed to connect BNO055 with Raspberry pi for the software part including install the necessary dependencies and the latest version of the BN055 Python module code from GitHub. The problem team had met here was that team was not able to use the code to read the data from accelerometer. This was because when disabling the kernel’s use of the Pi’s serial port, team navigated the menus to to the Advanced Options -> A8 Serial option and select ‘No’ to login shell over the serial port. Then team selected the ‘Finish’ menu option to exit raspi-config. At this point, it was ready to reboot the Pi to make the change take effect, however, team ran a wrong command: sudo systemctl disable [email protected] which caused some errors of accelerometer. After revising the handout, team find that team shouldn’t disable the ttyAMA0, since for the next step, the Raspberry Pi used the serial port and GPIO 18 as the reset pin. After solving that problem, team used another SD card to set up the accelerometer. Finally it worked, it really took team for long to solve this problem. The accelerometer worked fine after solving that problem.

Figure 3. The circuit of accelerometer

To test the BN055 Python module, team ran the simpletest.py and observed that every second the orientation data for the sensor is printed as Euler angles that represent the heading, roll, and pitch of the sensor in degrees. Team turned around the accelerometer to record the values of heading and roll for each direction for further use.

Software:

For the software part, the main challenge was how to realize the sand piles and sand flow motion using python. Professor helped us and gave us some very useful reference about realizing sand pile flow motions using cellular automaton in MATLAB [3]. Team also borrowed a book about cellular automaton from professor Bruce Land and read one of the chapters about sand pile games in detail [1]. A lot of matrix calculations were required for this game so team decided to use the numpy array to do the matrix calculation in python. There were four very important equations used in the code of the final project to simulate sand piles with ground objects (t+1 stands for the next time step):
sul(t+1) = gul*sul + (1-gul)*sul*sll*(slr+(1-slr)sur)
sur(t+1) = gur*sur + (1-gur)*sur*slr*(sll+(1-sll)*sul)
sll(t+1) = sll + (1-sll)*[sul*(1-gul)+(1-sul)*sur*(1-gur)*slr]
slr(t+1) = slr + (1-slr)*[sur*(1-gur)+(1-sur)*sul*(1-gul)*sll]

Equa1.Equations for simulation

For the four equations above, sul stands for the upper left corner of the sand block, sur stands for the upper right of the sand block, sll stands for lower left corner of the sand block and the slr stands for the lower right corner of the sand block. The ground matrix is also included in the equations.
For the algorithm cellular automaton, two matrices were mainly used: sand matrix and ground matrix. Sand matrix recorded the locations of all the sand particles and the ground matrix recorded the positions of the objects. It was necessary to first divide these two matrices into small 2×2 blocks. And for each block, some calculations were needed to determine where the sand particle of each cell should be based on the given ground position and the positions of other sand particles in the block. After the calculations of the matrices, team could get the updated version of sand matrix and the new sand matrix was used to derive the next updated sand matrix. Blit array function of pygame library was useful in displaying every new sand matrix which formed the flow motion of the sand. The basic rules of cellular automaton are shown below.

Now team already had the algorithm to make the sand fall. At the same time, it was really important to derive the new equations so that the sand could fall toward other directions including to the left, to the right and going up. For these three directions, team should change the positions of original sul, sur, sll and slr in each block which would change the direction of sand motion.
The more difficult part of the project was to implement the cellular automaton method to make sand fall in a slope angle. For this part, team members first wanted to increase the block size to 3×3 which allowed more possible directions of sand flow. However, the method would cause various problems to overall calculation between matrices. To fix the issue, team used a method based on the algorithm of previous motion of sand. For example, if team wanted to make the sand fall in upper left direction, team could just first implement the algorithm of making sand go up and them implement the sand going left algorithm. It was a lot easier to use this method to realize the sand flow in the other four slope angle directions.
A big issue team met when implementing cellular automaton demo code was the sand just could not fall at first. Instead the sand pile just kept getting smaller. Team was really confused at first because team double checked the equations which were correct. As a result, it was the problem of using numpy array of python. When we wanted to set a sand matrix to a new sand matrix, the content of sand matrix would also change if the content of new sand matrix changed. That was the reason why the sand initially didn’t fall. It was necessary to use the copy function of numpy array which only got a copy of the matrix and two matrices wouldn’t affect each other. Then the problem was fixed (shown in figure below).

Figure 6. Sand falling test

Another issue was that the performance speed was very low for each matrix display because team members initially used double loop to set the values of matrices. To improve the performance, team found a very useful function in pygame of copying the numpy array to the screen surface which greatly increase the speed [2].
After that, team created a homepage of the game using pygame library, there were three modes can be selected from the homepage. A quit button was also created to exit the game.

Figure 7. Main menu of the sand flow game

For all three modes, team organized the code in a different way. Team put all the cellular automaton code into 8 different python functions. It became easier to implement the rest of code. Team members could just directly use these functions to move the sand.
For mode one, team aimed to display a hourglass, where the sand would stay when it met barriers, and the rest of the sand would flow through the hole of the barrier until it touched the ground, the sand would accumulate on the ground. When the players turned the piTFT screen, the sand would keep being affected by the gravity. The ground objects at middle of the screen were created to test whether they were able to block the sand. Numpy array allowed users to set a range of cells to certain value which was really convenient [5]. The reset button could be pressed to change back to original state. The back button helped players go back to the main menu. The figure of mode one has been shown below. The mode one is shown in figure 8.

Figure 8. Mode one of sand flow game

For mode two, team built a new scene which includes three nozzles distributed on the right, left and top of the screen. Team used three buttons to control each nozzle. The sand would flow from the nozzle when the related button is being pressed. Once the button was pressed again, the sand would be released from certain nozzle. GPIO library was used to detect whether the GPIO buttons were pressed. If the button was pressed, the nozzle part of sand matrix would be set to value 1. The sand would keep coming out when the part of code was put in a while loop. This was basically how the nozzles control worked. The mode two is shown in figure 9.

For mode three, it was necessary for team to develop resistive touch detection of the piTFT screen because the mode three allowed players to swipe on the screen to create their own sand with different colors. At the same time, the sand created by players would look like being affected by the gravity when the players turned the piTFT screen. For this mode, team used get_pressed() function of pygame to check whether the touch screen has been touched. If the screen was touched, the get_pressed function would return true. Otherwise, it would return false. When the screen was touched, the coordinates of the touch positions on the screen should be recorded in x, y variables. Then team just need to set the values of cells in certain square region with center x,y to 1. Then a small size sand would be created on the screen. The created sand would be affected by players turning the piTFT screen.

Team met some issues when team tried to combine the finished cellular automaton code with the hardware and these design interface. For the mode three, it was easy to get lost because there were total 8 directions of sand flow and there were too many lines of code. It was much better to write each direction code step by step instead of copying and pasting a big chunk of code.
Team finally designed and made a case for the whole system so that the players could easily hold the screen and turn the screen. It is shown in figure 11.

Figure 11. New design for the sand flow project

Result

Team did meet several problems in the whole process. Fortunately, team solved all of the problem and reached the goal outlined in the description. Referring to the description, team completed almost all of them.The outlined goal team implemented is the mode two, which is a nozzle case, the sand is controlled by the GPIO input, which relates to different position of the nozzle to spout the sand.
Team even improves the game by building the mode three. The mode three allows users to decide where the sand appear. Users can draw whatever shape and position as they wish, the sand will also flow according to the rotation direction of the piTFT screen.
All three modes have been perfectly realized and everything performs as planned.

Conclusion

All in all, team has completed three modes of the game. Team realizes this game by first working on the accelerometer. After that, most of the time has been spent on the software code. Finally three modes are created.
The first mode is a basic mode, the sand flows just like the hourglass, the sand will stay still when it meets barrier, and the rest of the sand will continue to flow. Then user can turn around the screen of piTFT for different angels, the sand flows in the direction of gravity. This action is controlled by the accelerometer, which returns the heading and rolling value and the sand will interact with the value loaded into the system.
Also, users can change the color of the sand on the screen to yellow, green, blue, and purple. This is controlled by the GPIO button, each of the four button on the Raspberry pi related to one color.
Mode two is like a nozzle, there are three nozzle distributed in the left, right and top of the screen separately, the sand will spout from one nozzle dependent on which button is being pressed. Also, users can turn around the piTFT screen to decide which direction the sand flow.
Mode three is the creative mode of the game, which the origin of sand is not fixed to the screen, users can slide the screen to determine where the sand appears, the trajectory of the sand will be kept on the screen and once the finger is moved, the sand begin to flow. Also, user can change the flow direction by turning around the piTFT.
There are two buttons for the three modes, the first one is “reset” button, when the button is pressed, the sand will be back to the original position. And when “back” is pressed, the game will return to the home page for mode selection.

Future work

First of all, the future work of the project should be changing the color of the certain part of the sand and blend them together, there can be different colors of sand on the same screen, which will bring more amazing visual experience. To do this, it is necessary to create multiple sand matrices to record the motion of sand of each color. It is also important to check whether the sand particles in different matrices contact with each other.
Next, team can also improve the game by changing the flowing speed of sand based on the degree detected by accelerometer, which will be more realistic for user visual experience.
Finally, it is possible to add more directions for sand which can make the sand flow more realistic. To do this, it is necessary to improve the current algorithm of falling in a slope angle direction.

Source: ECE5725 Project Sand Flow

Scroll to Top