3D Orientation Mouse

Project Introduction
We created a handheld 3D orientation mouse that uses IMU motion and orientation quaternions for two practical computer interfaces: a “laser pointer mouse” where pointing the device controls the on-screen cursor, and a “3D mouse” for SolidWorks, where rotating the device live-rotates a CAD model view.
We used an MPU6050 6-axis IMU (3-axis accelerometer + 3-axis gyroscope) connected to an RP2040 microcontroller to record 3D orientation of our device. On the RP2040, we estimate the device’s full 3D orientation using a quaternion-based pipeline (gyro integration with accelerometer-based tilt correction). This quaternion orientation is then used for two functions:
Pointer Mode:
We compute the device’s pointing direction from the quaternion and map that direction to cursor motion, similarly to products like TV “air remotes.”
CAD Mode:
We stream orientation data over serial to a PC program, which converts our device’s orientation into SolidWorks or AutoCAD view rotations, allowing for continuous 3D view manipulation.
To make the device easily usable, we designed and fabricated a 3D-printed enclosure with dedicated mounting for the RP2040 and IMU. The device includes three debounced buttons for user input: one for mouse left-click, one for manual drift correction, and one for entering the RP2040’s BOOTSEL mode for easy reprogramming.
High Level Design
Motivation For Project
Our motivation for this project was to build a handheld device that fully utilizes the IMU capabilities of the MPU6050, explores the use of quaternions to track 3D rotations, and expands the basic functions of USB mouse control. Inspired by consumer devices such as the LG Magic Remote and the Wii remote, which let users control an on-screen cursor by pointing and rotating a handheld controller. We wanted to recreate a similar user experience for a computer by creating a 3D mouse that maps its pointing direction to an (x,y) cursor position on the screen, enabling remote mouse control for tasks like navigating a desktop, using their computer like a TV, or for better interaction with slides during presentations.
We were also inspired by 3D CAD devices such as the SpaceMouse, which make it much easier to manipulate a model’s view. However, traditional CAD controllers are designed for use on a desk. We wanted to combine concept of the SpaceMouse with our 3D controller, allowing a user to rotate the device in their hand (in the air) to live rotate the CAD view directly.
Relavent Math
Quaternions vs. Euler Angles
While Euler angles (Roll, Pitch, and Yaw) are the most intuitive way to visualize rotation, they have a few limitations which make them unsuitable for 3D orientation and rotation tracking:
Gimbal Lock:
Euler angles have a mathematical singularity (at 0 or 90 degrees) where two of the three axes align, causing the system to "lock," losing a degree of freedom and making it impossible to track rotations around the third axis independently.
Multiple Interpolations:
The other issue with Euler angles is that there are multiple (up to 12) possible orders for applying Euler rotations to get to the same final orientation, leading to computational issues with tracking. This also causes issues with smoothly transitioning between two orientations, leading to jerky or unnatural movements.
Here is a funny example of an interpolation issue in graphics caused by using Euler angles:
Because of these issues, we represent orientation using a
unit quaternion
, a 4-dimensional mathematical object $q=(w,x,y,z)$ where w is the real component and (x, y, z) are the imaginary components. For our calculations we use unit quaternions which are normalized, meaning $\sqrt{w^{2}+x^{2}+y^{2}+z^{2}}=1$.
This mathematical representation avoids the Euler angle issues of gimbal lock and rotational interpolation. Additionally, quaternions allow for Spherical Linear Interpolation (Slerp), which creates smooth, constant transitions between two orientations, making them a smoother representation of rotational motion.
Axis-Angle Representation
Quaternions use the idea that every rotation in space has an axis of rotation. We can use this idea to compute a rotation quaternion $q$. Any rotation can be represented by a single rotation, around a specific unit axis in space $(u=u_{x},u_{y},u_{z})$ given by the equation:
$$q=(cos\frac{\theta}{2},u_{x}sin\frac{\theta}{2},u_{y}sin\frac{\theta}{2},u_{z}sin\frac{\theta}{2})$$
Computing Rotations
We can represent 3D vectors with quaternions by setting the real part, $w$, to 0:
$q_{u}=0+u_{x}i+u_{y}j+u_{z}k$
We can apply a 3D rotation to a vector in $u$ in quaternion form $q_{u}$ with the equation:
${q^{\prime}}_{u}=qq_{u}q^{-1}$
where ${q^{\prime}}_{u}$ is the rotated vector, $q$ is the rotation quaternion defined above, and $q^{-1}$ is its inverse defined as:
$q^{-1}=(w,-x,-y,-z)$
Quaternion Multiplication
The rotation equation uses quaternion multiplication where the product of two quaternions \(q_1\) and \(q_2\) is defined as:
\[
\begin{aligned}
q_1 q_2 = ( & w_1 w_2 - x_1 x_2 - y_1 y_2 - z_1 z_2, \\
& w_1 x_2 + x_1 w_2 + y_1 z_2 - z_1 y_2, \\
& w_1 y_2 - x_1 z_2 + y_1 w_2 + z_1 x_2, \\
& w_1 z_2 + x_1 y_2 - y_1 x_2 + z_1 w_2 )
\end{aligned}
\]
It is important to note that quaternion multiplication is non-commutative, meaning the order of multiplication matters.
Filtering
In addition, we used complementary filtering to fuse gyroscope and accelerometer data:
$$q_{est} = \alpha \cdot q_{gyro} + (1 - \alpha) \cdot q_{accel\_correction}$$
We also used low-pass filtering to smooth cursor motion with the equation:
$$x_{smoothed} = s \cdot x_{previous} + (1 - s) \cdot x_{current}$$
where $s$ is the smoothing factor.
Logical Structure
The general structure of the code can be broken down as follows:
1. Interrupt Service Routine
We use a PWM wrap interrupt as a periodic timer to run the interrupt service routine code at around 1.2kHz. At each interrupt we read raw accelerometer and gyroscope data, store them in shared variables, and signal an IMU thread to update the attitude estimation.
3. IMU Thread
The IMU thread is signaled each time we read new IMU data. It primarily runs the imu_update() function to compute the new orientation quaternion from the accelerometer and gyroscope readings.
4. Cursor and CAD Control
The device can switch between two modes. The first is
CAD Mode
, where the device controls the orientation of a CAD model on the user's PC. The second is
Pointer Mode
, where the device controls the cursor location on the user's PC.
4.1 Pointer Mode
In Pointer Mode, we make the RP2040 act as a USB device to the user's PC. We map the orientation of our device to a delta x and y position on the PC screen and output that to the computer as a mouse signal. We also check if the user pressed the calibration or the mouse left click button and output the corresponding signal.
4.2 CAD Mode
In CAD Mode, the RP2040 acts as a serial device where we output the most recent orientation quaternion over serial to the PC. The PC then runs a Python program to manipulate the camera view of an open CAD model to match the orientation of our device.
Hardware/software tradeoffs
The main tradeoff we ran into was system complexity versus long-term orientation accuracy. Our current design uses a 6-axis IMU (accelerometer + gyroscope). The accelerometer provides a stable long-term reference because it can measure the direction of the gravity vector, so it does not accumulate drift. However, the gyroscope, which measures angular velocit,y has an issue of dead reckoning, meaning its orientation estimate is always relative to its initial value and will drift as small bias errors accumulate.
To get the best of both sensors, we use a complementary filter: the gyroscope provides smooth short-term motion, and the accelerometer slowly corrects long-term tilt. This works for pitch and roll, because the gravity vector provides a reference direction to correct drift on those axes. However, yaw rotates perpendicularly to the gravity axis, so gravity does not change during yaw motion. Therefore, yaw cannot be corrected using the accelerometer alone, and it will drift over time if we rely only on the gyroscope.
One common solution is to add a magnetometer (using a 9-axis IMU) to provide an absolute heading reference and correct yaw drift (either by using the earth’s magnetic field or a magnet placed near the user). However, one of the issues with magnetometers is that they require heavy calibration, due to environmental magnetic interference, and would be too complex for the scope and timeline of this project. Instead, we adopted a simple user-friendly approach with manual re-centering calibration during use to manage yaw drift.
From a computational standpoint, we had no major constraints. The project is primarily user experience oriented, so our target update rate was around 100 Hz, which is well within the RP2040’s capabilities for quaternion calculations, filtering, and calculations for the different modes of the device.
Intellectual Property Concerns: Patents, Copyrights, and Trademarks
There are no patents, copyrights, or trademarks associated with the methods we used. Any of the quaternion math and orientation calculation methods were obtained from public-domain sources. We use products such as SOLIDWORKS, AutoCAD, MPU-6050, and VS Code (with the TinyUSB library) for our project.
Hardware Design
Program Details
1. Main
The main function simply initializes the IMU, GPIO, I2C communication, threads, etc.
2. PWM ISR
The PWM ISR runs at a set frequency of around 1.2kHz. It first reads raw gyro