The Beauty Of Dance, Seen Through The Power Of Touch

A haptic device that lets visually-impaired people experience the flow and details of dance.

It’s nothing short of amazing what trained dancers can do with their bodies, and a real shame that visually-impaired people can’t enjoy the experience of, say, ballet. For this year’s Hackaday Prize, [Shi Yun] is working on a way for visually-impaired people to experience dance performances via haptic feedback on a special device.

This platform, which is called Kinetic Soul, uses Posenet computer vision to track a dancer’s movements. Posenet detects the dancer’s joints and creates a point map to determine what body parts are moving where, and at what speed. Then the system translates and transmits the movements to the 32 pins on the surface, creating a touchable picture of what’s going on. Each 3D-printed pin is controlled with a solenoid, all of which are driven by a single Arduino.

We think it’s interesting that Kinetic Soul can speak to the user in two different languages. The first is more about the overall flow of a dance, and the second delves into the deconstructed details. Both methods allow for dances to be enjoyed in real time, or via video recording. So how does one deconstruct dance? [Shi Yun] turned to Laban Movement Analysis, which breaks up human locomotion into four broad categories: the body in relation to itself, the effort expended to move, the shapes assumed, and the space used.

[Shi Yun] has been user-testing their ideas at dance workshops for the visually impaired throughout the entire process — this is how they arrived at having two haptic languages instead of one. They plan to continue getting input as they work to fortify the prototype, improve the touch experience, and refine the haptic languages. Check out the brief demonstration video after the break.

Yes indeed, dance is a majestic way of expressing all kinds of things. Think you have no use for interpretive dance? Think again — it can help you understand protein synthesis in an amusing way.

3 thoughts on “The Beauty Of Dance, Seen Through The Power Of Touch

  1. It’s a beautiful idea, but I am skeptical that the nuance of dance can be compressed into 32 bits in a way that makes for a meaningful experience. Small but significant differences would be obscured by this display. Things like whether a hand is supinated or pronated, or if a foot is extended or flat.

    1. I mean, baby steps? Right now, the blind can’t tell the difference what direction the foot is pointed without someone telling them or feeling it in action. Nuisance or not, this is a heck of lot better than nothing.

    2. I had similar thoughts.

      I was wondering if it would be feasible (/safe) to match a small scale humanoid character’s movements to those of the dancer. Then the visually impaired person could feel the movement. Not sure that it would work in real time, but foot pedals controlling speed, frame-by-frame, forward/reverse, etc. might be workable.

      Then, for whatever weird-brain reason, I visualized the character being a Muppet, like Kermit. It might be that I’d find it more entertaining to watch the Muppet do the dance.

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.