This grid of letters is a puzzle game for tablet devices called Ruzzle. The contraption attached is an automated solver which uses LEGO Mindstorm parts to input the solutions on the screen. [Alberto Sarullo] is the mastermind behind the project. As you can seen in his demo video after the break he has a flair for the cinematic. But he makes you work a little bit to discover the details of his project.
His post gives a general overview of how this works. A Linux box takes a screenshot of the Ruzzle board. After processing the graphics with Imagemagick he uses Tesseract — an Optical Character Recognition program — to figure out which letter is on each square of the playing area. From there NodeJS is used to discover all possible words with the help of a dictionary file. The final solutions are pushed to the LEGO parts to be traced out on the touch screen with a stylus. The nice thing is that he published all of his code, so you can drill much deeper into the project by pawing through his repository.
