Ma petite entreprise

I just demo'ed a port of Revealicious to a 2 x 42" multi-touch surface built by Christophe Viau at ETS Montréal. It was a lot of fun, and a lot of work too !

I wrote the port in OpenGL and Scala, using TUIO as the interaction communication protocol. Here are some technical notes on the implementation:

• Implementing interactions is hard: bye bye hover/click/drag, you have to deal with multiple cursors, gestures, time(out) and unreliable event sources.
• You need muscle: interfaces are not supposed to hog your CPU, yet the users expect fluid interactions and transitions. Using a fast language and library helps a lot, but you still need to optimise so that you don't eat CPU when the UI is idle (this can be tricky !)
• Tweening will make you happy: as multi-touch, and especially surface interfaces usually require animations and even sometimes 3D, you'll get much better results if you use tweening (like exponential ease out). This small change will make you interface shine at a small cost !
• OpenGL libraries are not designed for UI: almost every single OpenGL library is about games or creating real 3D environments. When you do an interface in 3D, you need a scene graph, but you also really need good picking/selection/events functions. You also need something simple, and an ability to deal with text. While there is tons of OpenGL libraries, I've found none of them to meet these requirements (and I've looked in Python and Java libraries)

Overall, my experience with Scala for this was really great: the performance is there, using immutable data structures is reassuring when you know you have at least 3 concurrent threads in your app (AWT events, TUIO events and OpenGL loop), and the memory footprint is not that bad (you need to constrain the heap consumption). On a language level, the DSL-ability of Scala makes it really convenient to write event handlers, even thought the type systems is sometimes more a burden than a boon. Being able to re-use existing Java libraries is also a big plus.

If you're interested in some design-related notes on this topic, here is the design notes on creating a multi-touch interface on FFunction's blog.