In this project, I build a pen device which can be used to recognize handwritten numerals. As its input, it takes multidimensional accelerometer and gyroscope sensor data. Its output will be a simple classification that notifies us if one of several classes of movements, in this case 0 to 9 digit, has recently occurred.
- Sparkfun Artemis ATP
- SparkFun 9DoF IMU Breakout - ICM-20948 (Qwiic)
- SparkFun Qwiic Cable - 500mm
- Seeed Grove - Mech Keycap
- Gameduino 3
- Micro SD card
- Arduino IDE 1.8.12
- TensorFlow 2.1
- SparkFun_ICM-20948_ArduinoLibrary (A forked repo of the original library with FIFO implementation)
- Jupyter
- Gameduino 3 library
Please see the links above for downloading and installation instructions. Please see the instructions here to setup Artemis ATP board with Arduino IDE.
The Gameduino 3 does not work out of the box with Artemis ATP. We need to do few adjustments in the Gameduino library. Please download Gameduino2.zip from https://github.com/metanav/gd2-lib. Also, we need to add pin definition for A32, please see the pull request sparkfun/Arduino_Apollo3#129. To connect the Gameduino 3 shield, align the longer header pins on the shield to the Artemis ATP RX1->TX1 ... GND->GND as shown in the image below.
We need to connect Grove Mech Keycap (mechenical switch) to the Artemis ATP. It is connected to the 3V, GND and digital pin 36. Since Gameduino 3 shield covers all the power pins of Artemis ATP, we need to solder wires to the secondary rail of plated through-holes as shown below.
The SparkFun 9DoF IMU Breakout-ICM-20948 (attached to a pen) is connected to the Artemis ATP Qwiic connector using a long 500mm SparkFun Qwiic cable.
Use data_collection.ino in the sketches directory to collect data from the IMU sensor.
Use train.ipynb to train the model. You can use a GPU or CPU to train.
Use handwriting_recognizer.ino to recognize handwritten numerals from the data coming from IMU sensor motion.