RGB Games Using Fiducials

Development Process

This project evolved as an offshoot of my MFA thesis project. I had planned to use camera tracking software known as reactivision in my thesis and used this game concept as an opportunity to learn more about tracking before implementing it in my thesis. Reactivision can use a webcam to detect premade symbols known as fiducials (see image below). Reactivision can identify the unique ID number of each fiducial as well as its orientation (lots of example videos).

In addition to reactivision, a library known as TUIO can be used to implement reactivion's capabilities in a number of different programming languages (see their downloads page). As part of my thesis, I had originally done some testing using reactivision with the TUIO library in Processing. While this worked, Processing didn't seem like the ideal way to implement my thesis project. The project seemed much better suited for a web browser. So after further research I found a browser plug-in that passes data from reactivision into javascript.

While the TUIO plug-in works, it is not supported by the most current versions of Google Chrome. This was an issue since my project needed a fullscreen browser. After further research I decided to use FireFox for my project since it could use the plug-in and be displayed fullscreen through a kiosk extension.

After I determined the best technology for the project, I began to work on the games themselves. I decided to create a series of three games that could be played with fiducials. Originally, I had planned to connect the browser to an Arduino through node.js. This would allow the fiducials to control the color of RGB LEDs and the position of servo motors. Eventually, these functions were removed due to time and the game became solely web based. An initial sketch is shown below.

To create the hardware for this project, I found wooden cylinders, painted them, and glued a fiducial symbol to the bottom of each. I also created a small platform with a plexiglass top and camera and light inside to "see" the fiducial.

The game begins when the user places the blue fiducial on the platform. The game automatically redirects to a brief instructions page. After several seconds the first game begins. Javascript generates a random RGB color that the user must match. They can change the RGB values of their color by placing a color cylinder on the platform and rotating it to change the intensity of that color. In other words, when the user places the red cylinder on the platform, they can change the red value of their color anywhere from 0 to 255 by rotating the cylinder. The user has 30 seconds to match the red, green, and blue values of the randomly generated color.

The second game operates similary to the first. Javascript randomly rotates a red, green, and blue dial on the screen. The user must then place the color cylinders on the platform and rotate them, to rotate their dials on screen. The user has 30 seconds to match their wheels to the randomly rotated wheels.

The final game randomly displays either red, green, or blue. The user then has 5 seconds to place the corresponding color cylinder on the platform. The user must get 5 correct without missing 3 of the colors. Also, the timer moves faster as each successive color is displayed. After the final game is complete, the page automatically redirects to the start screen for the next user to begin.

Below are photos and video of the game being exhibited.