Samur-Pi

Introduction
Design
Results
Conclusions
Appendices
ECE4760 Final Project by Mark Franklin and Mandy
Chen
Introduction
Samurpi is a sword-training system that uses IMU sensing and real-time haptic feedback to teach proper sword
technique. An IMU embedded in a custom wristband tracks the user’s motion, while distributed vibration motors
provide intuitive feedback as the user holds the sword. Through structured training and fast-paced game modes,
users can practice authentic techniques and build reaction speed in an engaging, interactive environment. We
created this project to explore how sensing, haptics, and game design can make learning complex physical skills
more accessible, engaging, and effective.
High Level Design
Rationale/sources of idea
We were motivated to take on this project by a shared interest in learning sword fighting and a passion for
designing interactive gameplay experiences that promote active learning. The project was also inspired by Fruit
Ninja, a game we both enjoyed growing up. Rather than slicing virtual objects on a screen, we sought to elevate
the experience by introducing a handheld sword with vibrating motors, creating an engaging and immersive way to
develop coordination, reflexes, and motor skills through interactive sword training. While our work focused on
gameplay, the system could also be adapted to support self-defense training.
Background math
State estimation
Adapted from Lab 3, a complementary filter was used to synthesize the MPU-6050’s gyro and accelerometer
measurements. This technique combines low-passed accelerometer data with high-passed gyroscope data to take
advantage of the zero-bias accelerometer data and the minimal-noise gyroscope data. Without a magnetometer, we
were not able to synthesize an absolute position for the z-axis (yaw) measurement, as in our design this axis
was aligned with the gravity vector. As such, we could not use the accelerometer data in the complementary angle
calculation. We instead integrated the gyro measurements for an approximate z-axis state estimation.
\[
\theta_{\text{est}} = 0.99 \, (\theta_{\text{prev}} - \Delta \theta_{\text{gyro}}) + 0.01 \,
\theta_{\text{accel}}
\]
To measure the rate of change of each angle, we chose to dynamically measure the angle delta as per the below:
\[
\omega = \frac{\theta - \theta_{\text{prev}}}{t_{\text{elapsed}}}
\]
t_elapsed could not be reduced to a constant as our issues with haptic motor noise coupling to the I2C IMU
communication (described more below) could produce random ~25 ms delays between measurements. As such, we used
time_us_64() to track the time deltas and dynamically adjust t_elapsed. While we could have used the raw gyro
measurement for angular velocity, we opted for a sensor fused velocity using the complementary angles as we
hoped to minimize the gyro drift’s contamination of the data. Vibration from the haptic feedback motors had a
profound effect on the accelerometer accuracy, acting to accelerate the damped proof mass system and distort the
sensor’s measurement of gravity. To fix our absolute position estimation (complementary angles) we moved the IMU
off the sword, producing an accurate angular velocity. In retrospect, gyro velocities would have likely been
sufficient for our project. Still, our approach was an interesting learning experience in the ultra-broad field
of sensor fusion. We also ran into a bug wherein large angular velocity spikes occurred as the complementary
angles “wrapped around” axes, eg from +180° to -180° due to the atan2 usage. We were able to fix this, and cap
any other excessive rate by clamping the angle deltas to 1500 °/s - a realistic human maxima in our setting.
Haptic Feedback
Early in the project design, we debated whether the haptic feedback should be used as a reward or penalty
function, i.e. would it activate as the user exemplified correct or incorrect behavior? After setting up a
simple demonstration of each, it became clear that it was much more intuitive to receive vibration as a reward
function, which mirrors the practices of teachers, coaches, and mentors everywhere - beginners need positive
feedback to establish good practices.
We then designed a proportional controller (with configurable coefficients) against the desired ωx, ωy, ωz and
θy as per the below:
Parameter
Condition
Coefficient
Purpose
\(\omega_x\)
\(\omega_x > 200^\circ/\text{s}\)
max_vibe/200
"Cutting power"
\(\omega_y\)
\(\omega_y < (1/3) \omega_x\)
max_vibe/200
Min roll in wrists
\(\omega_z\)
\(\omega_z < (1/4) \omega_x\)
max_vibe/200
Min off-axis velocity
\(\theta_y\)
\(|\theta_y - \theta_d| < 10\)
max_vibe/45
Vertical vs Horizontal vs Diagonal
ωy and ωz conditions were given relative to ωx,as hardcoding them left our reward function vulnerable to axis
leakage - wherein the user’s wrist strap position caused the IMU’s axes to lose some alignment with the sword.
At first, we tried manually tuning the coefficients, but this quickly became time consuming. We instead use each
coefficient in terms of max_vibe (pwm output to the haptic motors) such that vibration would reach zero at a
desired “max_error”. For example, ωx,penalization will produce zero vibration if the user swings the sword at ~=
0 °/s, and ωy penalization will do the same if ωy = ωx-desired. This produced a nice objective function that
required strong performance from the user without being overly harsh. Finally, the θd is set by the training
state machine: 180° for vertical slice, 90°|| 270° for horizontal slice (handles left vs. right-handed users),
or 135°|| 225°for the diagonal slice.
Intuitively, it would make sense to sum the errors to balance your correction of each factor. The downside of
this approach is that it allows strong conditions to “cover up” weak conditions. For example, if the user is
holding the sword in the right place, but not moving it at all, they would receive nearly full vibration!
Instead, we use the max correction. Just as a youth coach teaching a team member, the algorithm corrects for
only the most important teaching at any one time.
Real-Time Slice Classification
As mentioned below, we weren’t able to fully integrate this functionality to our Fruit Ninja game mode. We did
get this algorithm fully functional (in isolation) and it will be submitted in our final code.
To bring the learning process full-circle, we scoped in a game mode that would use the slice types and objective
function described above in a fast-paced test of muscle memory and reaction speed. Effectively, the pattern of
falling fruits (horizontal, vertical, or diagonal combination) would inform the user of the required slice type.
We would then detect and classify the user’s action as per the reward structure described above, and cut the
fruits if it met a minimum score. In doing so, we were training the user to maintain proper form in a high-speed
environment.
Called within ISR at 1000 Hz:
The grading function is executed at a high frequency to
ensure real-time responsiveness.
Score Sword State:
The sword state is scored against our objective function for each of the
following slice types:
Vertical
Horizontal (L)
Horizontal (R)
Diagonal (L)
Diagonal (R)
(The only difference between these scores is the \(\theta_y\) component for absolute orientation.)
Select and Record Result:
Select the lowest score as the most likely sword action.
Compare it to a minimum threshold (configurable by the coach) and record the result.
Classification (Persistence Check):
Classify the action as type \(t\) if you have 100
consecutive successes without 10 consecutive failures.
This approach proved fairly robust, and we could tune response time vs. instability by increasing or decreasing
the consecutive success/failure thresholds. There were some adjacent mis-classifications (vertical vs. diagonal,
diagonal vs. horizontal) that could be fixed by a rigid IMU mounting, but otherwise this classification
performed well.
Fruit Rotation
Our drawRotatedPixel function rotated a pixel offset (dx, dy) about a center point (cx, cy) by an angle before
drawing it. Mathematically, we applied the standard 2D rotation matrix
\begin{bmatrix}
\cos \theta & -\sin \theta \\
\sin \theta & \cos \theta
\end{bmatrix}
to the original offset, computing the rotated coordinates:
\begin{aligned}
rdx &= dx \cos \theta - dy \sin \theta \\
rdy &= dx \sin \theta + dy \cos \theta
\end{aligned}
The rotated pixel was then translated back to screen coordinates by adding the center position:
\begin{aligned}
px &= cx + (dx \cos \theta - dy \sin \theta) \\
py &= cy + (dx \sin \theta + dy \cos \theta)
\end{aligned}
In our code, we implemented these equations using fixed-point math and then converted back to integer pixel
coordinates before calling drawPixel(px,py,color).
Logical structure
As shown below, our software architecture leveraged both cores and multiple threads to balance high-speed
graphics with responsive hardware I/O. Core 0 functioned as the system interface layer, handling debounced
keypad matrix scanning and a serial debugging console used to trigger game events. Core 1 was dedicated to the
application’s computational workload, executing the main VGA game loop, game state machine, and IMU sensor
processing.
The gameplay flow was structured as a hierarchical state machine that transit