Did you ever feel the urge to turn the power of image processing and OCR into music? Maybe you wanted to use motion capture to illustrate the dynamic movement of a kung-fu master in stunning images like the one above?  Both projects were created with the same software.

vvvv -pronounced ‘four vee’, ‘vee four’ and sometimes even ‘veeveeveevee’- calls itself ‘a multi purpose framework’, which is as vague and correct as calling a computer ‘a device that performs calculations’. What can it do, and what does the framework look like? I’d like to show you.

Since its first release in 1998 the project has never officially left beta stage. This doesn’t mean the recent beta releases are unstable, it’s just that the people behind vvvv refrain from declaring their software ‘finished’. It also provides an excuse for some quirks, such as requiring 7-zip to unpack the binaries and the UI that takes some getting used to. vvvv requires DirectX and as such is limited to Windows.

With the bad stuff out of the way, let’s take a look what vvvv can do. First, as implied by the close relationship with DirectX, it’s really good at producing graphics. An example for interactive video is embedded below the break. With its data flow/ visual programming approach it also lends itself to rapid prototyping or live coding. Modifications to a patch, as programs are called in this context, immediately affect the output.

The name ‘patch’ harkens back to the times of analog synthesizers and working with vvvv has indeed some similarities with signal processing that will make the DSP nerds among you feel right at home.

What’s in the box?

vvvv comes with a versatile standard library which can be expanded with a wide selection of ‘packs’ that add more functionality. You get nodes to interface with Kinect or Leap Motion controllers, ones that are wrappers for OpenCV and others that let you connect to an I2C bus. XML parsing is available as well as database access and a rather complete audio toolbox.

For applications where the available nodes are insufficient or clumsy to use, vvvv offers several ways to add custom functionality. For once new nodes can be implemented in C#. Additionally shaders that run on the GPU can be written in HLSL and then added to the rendering pipeline. Since the introduction of DirectX 11 even compute shaders that perform large numbers of calculations in parallel on the GPU are possible. If even more processing power is required, the possibility to distribute work over several machines in a network is part of the standard library under the name ‘boygrouping’.

What can it do for me?

One major field where you might encounter vvvv is advertisement. This pseudo X-ray of a combustion engine is a exemplary for this kind of application. Gesture driven interfaces to present company data are such a common application for vvvv‘s capabilities, that the term ‘Minority Report UI’ has become a cliché among developers working with corporate customers.

The second big field is art. vvvv is free for non commercial use and has been embraced by visual artists and musicians for stage shows. There are also interactive installations like this three player augmented reality game or a virtually empty book.

The community that has grown from this apparent clash of worlds is a good example of the things that can happen when people come together. In the forum you have musicians discussing technology with serious coders that aim to wring the maximum performance out of their graphics cards to create stunning realtime renderings. Often these aspects are combined in a single person.

How do I get there?

‘With great power comes a steep learning curve’, or so they say. Aside from custom nodes and shaders, working with vvvv feels very different from coding as you may be used to. On the other hand, understanding coding techniques like binary logic or state machines -‘automata’ in vvvv– certainly helps.

One of the core concepts in vvvv is the ‘spread’. Its role is comparable with the pointer in c and the list in python. A spread is a one dimensional array that rolls over its index at both ends. So when trying to read behind the end of a spread, it will return the items at its beginning. This is reflected in the way most functional blocks -called nodes- treat spreads: the output will be as long as the longest input spread and shorter input spreads will be repeated as required.

Getting the hang of spreads and the way nodes interact with them is indeed the first step of becoming productive with vvvv and may require some unlearning from people used to object oriented and procedural programming. Luckily there are good resources to get you started as well as in depth examples on most topics. The documentation is spread out over several places. First there are tutorials and reading material aggregated on the vvvv site. Demos for specific topics can be found in a subfolder named ‘girlpower’ that is part of the standard release and also most packs. Then there are videos of workshops and finally integrated help patches for most nodes. They pop up when F1 is pressed and show the currently selected node in a set up that usually explains its behaviour

Why would I want to use it?

vvvv is a toolkit and having more tools is always better. Its use isn’t limited to its natural habitat of trade shows and art festivals. Computer vision, projection mapping, generative audio and video combined with a webcam, a projector and a bed sheet are prime ingredients to scare your neighbourhoods kids. And the software side of a build like the motion activated super squirter can easily be reimagined as vvvv patch. Furthermore I’ve found that a lot of learning takes place when tinkering with vvvv‘s examples and help patches, either when trying to break them or when attempting to make them do things they weren’t supposed to. Which basically is the definition of hacking.