THP Semifinalist: NoteOn Smartpen

There are a ton of apps out there for taking notes and recording ideas, but sometimes the humble pen is best. However, if you have the tendency to lose, crumple, or spill caffeinated beverages on your pen and paper notes, having a digital copy is quite nice.

The NoteOn Smartpen by [Nick] aims to digitize your writing on the fly while behaving like a normal pen. It does this by using the ST LSM9DS0TR: a 9-axis inertial measurement unit (IMU). These inertial measurements are processed by a STM32 Cortex M4F processor and stored on the internal flash memory.

To retrieve your notes, the Nordic nRF8001 Bluetooth Low Energy radio pairs the MCU with a phone or computer. The USB port is only used to charge the device, and the user interface is a single button and LED.

The major hardware challenge of this device is packaging it in something as small as a pen. Impressively, the board is a cheap 2 layer PCB from OSHPark. The assembled device has a 10 mm diameter, which is similar to that of ‘dumb’ pens.

The NoteOn doesn’t require special paper, and relies only on inertial measurements to reconstruct writing. With the hardware working, [Nick] is now tackling the firmware that will make the device usable.

SpaceWrencherThe project featured in this post is a quarterfinalist in The Hackaday Prize.

28 thoughts on “THP Semifinalist: NoteOn Smartpen

    1. more data means less guesswork for your inertial reference algorithms. you have two accelerometers it makes it easier to compensate for gravity, also it provides a virtual 2 axis gyro. i hear aviation grade units have multiple everything.

  1. Hope there is space for the actual ball pen insert. May be offset a bit above the components. This would give you hard copy back up as well as feel back on paper.
    May be it would be nice to have wireless charging. Something like those electric tooth stands would be great. (The regular wireless pads are too big.)

    1. I’d imagine it won’t be looking at actual position so much as direction of travel (and, more importantly, changes in direction). That gives you a nice intersection that the thing you want to know about is the thing you can directly measure.

      1. Drift is a huge problem with Accelerometers when you’re using them for position tracking. If they are going to us them for position tracking… it’s not going to work. You can imagine if you accelerate by x amount for x time, you have have a fixed velocity, and a increasing position. You then decelerate by x amount for x time, you then should have a zero velocity… (ideally) but if you deceleration rate is of by… anything.. you have a constant velocity and a growing displacement… this will lead to an error that literally grows with time

  2. This reminds me of an idea I had a long time ago.
    I thought of recording the movements of the ball point of a pen, like a the way the
    ball of a mouse was tracked.
    (Oh please, let us not go on with the “mouseballs” jokes!)

  3. very interesting, you can go several ways about software:

    -rebuild 3d scene, translate it into 2d plane and reconstruct actual written symbols, OCR that
    or
    -screw all of the above, throw raw accelerometers/gyros/compass data (no kalman filter) into deep neural network and let every user teach his pen his own handwriting style

  4. This design could be much smaller and more power efficient.

    That MCU is bad for power numbers, if you’re an ST fan a STM32F401 would have been better. Also why didn’t they get an battery charger integrated into the LDO, like the STNS01? Additionally BOSCH makes better/lower power gyros. The gyro they used draws like 3mA when turned on.

    1. Also Nordic isn’t a particularly low power BLE radio either. Dialog has radios that are also M0 cores and have a peak transmission current of 10mA (which is about half of nordic’s)

    1. Not so huge, actually, about a size of a big fountain pen. But it is 10mm on the inside, so should be about 12mm on the outside. And a place for the battery and something to actually write with.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.