Coral

Developed by Luke Hottinger and Chris Williams

Coral is a wooden, electronic, audio game for the visually impaired. Housed within a laser-engraved wooden enclosure, the electronics of this toy produce tonal sequences to be matched. A player’s responses are identified by toy rotation in four directions. The highly textured exterior and the stylized marker provide tactile orientation cues. Game states are distinguished by families of distinctive auditory cues. This project explored diverse disciplines: sound design, product design, interaction design, user interface design, and architecture.

Full documentation is also available here.

coral_box_front

Roles:

Luke Hottinger

  • Programming
  • Research
  • Documentation
  • Assembly/Electronics

Chris Williams

  • Game Mechanic Design
  • Sound/Visual Design
  • Research/Documentation
  • Assembly/Fabrication

Implementation:
Product Design:
The initial motivation for Coral was exploration of alternative perspectives, as well as collaboration.

The primary and secondary interaction modes of Coral are auditory and tactile, respectively. The project emerged from a desire to work with the visually impaired. Within that culture, sound and touch are often used in daily interaction with the world.

The form of Coral was simplified to a cube. A cube has discrete surfaces and allowed for easier mapping of tones to surfaces. Historically, the wooden block is one of the oldest and simplest toys for young children.

Coral was designed with intended use by people age seven and older. As a result, the design was lightweight.

The enclosure is 64 cu. in. (or 4 in. X 4 in. X 4 in.). The electronics were optimized to fit in a space of 27 cubic inches (or 3 in. X 3 in. X 3 in.).

coral_box_front

coral_box_back

Sensors:

Coral used a number of inputs and sensors to help the user navigate through the menu as well as play the game. Our project used three main components: an Arduino Uno, an Invensense MPU-6050 Accelerometer, and an NRF-8001 Bluetooth Low Energy (BLE) module. The accelerometer transferred data to the Arduino using the I2C protocol while the BLE module communicated with the Arduino using the SPI protocol. Other components used include: a piezo speaker for playing tones and a 1000 mAh LiPo battery.

Accelerometer:
The MPU-6050 is a 6-axis gyroscope/accelerometer sensor that was the basis of our gesture sensing. The sensor records both the current direction of gravity as well as the rotational speed and position of our box. These gestures were recognized through a state-based sensing mechanism. In this, each side of our cube was given a state number that represented if it was the active side (facing upward). If the cube detects that a state change was made, it begins to record the string of states after it. In 1 player mode, this string of state changes is then compared to the provided string of states randomly chosen by the program to see whether the user entered the correct pattern.

BLE Module:

The NRF-8001 is a UART based BLE module that allows for serialized data to be transmitted between the chip and another BLE enabled device. In our project, this was intended to transmit gameplay data between an iOS device and our cube for a 2 player mode. However, it served as a means of debugging and game servicing. We were able to send commands from an iOS app to the cube to test various functions in the gameplay program.

Arduino:

The Arduino Uno microcontroller acted as the central processing unit for our project, running the gameplay program as well as managing all of the connections and gestures entered. It handled a various number or protocols, such as I2C and SPI to connect to the various sensors and components. The Arduino also served as the project’s power regulator, taking in 12V and stepping it down to a component friendly 3.3V and 5V.

LiPo Battery:

The battery used was a 1000mAh, 3 Cell, 12V HobbyKing battery. This battery had plenty of power to keep our project running for upwards of 15 hours of continuous use.

Piezo Speaker:

The piezo we used had a frequency response range of 30 Hz to 15,000 Hz.

Wiring Diagram:

coral_wiring_diagram

Material:

The enclosure is made of laser-etched and laser-cut poplar wood. Wood was chosen for its organic properties, such as its general texture and relative malleability.

Poplar was chosen due to its softness and straight, uniform grain. The uniform grain allowed for greater resolution of texture into the surface.

poplar

Interface:

Four sides of the enclosure are used during gameplay. Two sides are used for utility: the top and bottom surfaces. The top surface has a shell design that serves as a tactile “home” cue and houses the power switch. The bottom surface has a concentric pattern that led to a proposed, centrally-located USB power jack.

To readily identify the enclosure’s walls, each exterior surface was given a unique texture. The textures were modeled after the organism coral. As a species, coral has diverse textures. The use of coral provided a unifying visual and tactile motif. Similarly, the shell atop the “Home” square is modeled after a gastropod called a “limpet”.

Gameplay encouraged grasping the cube with two hands. The use of both hands insures secure rotation of Coral.

Sound Design and User Experience:

The Neutral state is defined as the Home square facing upward and the tapered side of the shell pointing at the player.

Tones:

Coral has families of codified tones. They are categorized as follows:

A. Utility:

  1. Welcome: A set of three tones indicating that the device was first powered on.
  2. Menu Confirmation: In Utility Mode, a tone indicating that a player’s gesture was received.
  3. Orientation: When the Home square is upside-down, a tone encouraging the device to be returned to the Neutral state.

B. Gameplay:

  1. In-Game: Four tones total. One tone is mapped to one side of the device.
  2. Acknowledgement:
    a. Positive Tone: During Gameplay mode, the tone indicating that a tonal sequence was matched.b. Negative Tone: During Gameplay mode, the tone indicating that a tonal sequence was unmatched.

All tones were distinct. The Utility Tones were differentiable from the Gameplay Tones. Players can understand whether they are in the menu or in the game based on sound.

Similar to the game Simon, Coral has four In-Game Tones, which are loosely based on the F Major chord. These notes are:

F4: 349 Hz
A4: 440 Hz
C5: 523 Hz
F5: 698 Hz

The F notes differed by an octave.

Wherein the tones of Simon are reminiscent of trumpet fanfare, the In-Game Tones of Coral were inspired by an instrument called the “handpan“. All Coral tones were in consonance, with the exception of the Negative Tone.

Example of User Experience (UX):

Player turns on device

Welcome Tone plays

Player selects Game Mode

Game Modes:

1 Player:

Tilt device towards self and return to Neutral State

Play confirmation tone

2 Players:

Tilt device away from self and return to Neutral State

Play confirmation tone

Game Mode: One Player

Play In-Game Tone:

If matched, then play Positive Tone. Then, device plays a different In-Game Tone plus one more In-Game Tone.

If unmatched, then play Negative Tone. Then, game over. Return to Game Mode Selection.

Play two In-Game Tones:

If matched, then play Positive Tone. Then, device plays two different In-Game Tones plus one more In-Game Tone.

If unmatched, then play Negative Tone. Then, game over. Return to Game Mode Selection.

Name:

The name Coral was chosen for numerous reasons. There is an obvious reference to the visual motif. The more subtle aspect is that the name can double as a person’s name. This feature alludes to the history of games, like Simon and Henry.

Discussion:

Humans are visually-oriented. This preference is seen in such forms as movies, books, magazines, and live theater. There are numerous facets of life that demonstrate a bias toward the sighted, such as advertising. Focusing on underexplored aspects can yielded new and surprising results.

There are numerous technologies for the blind. However, they tend to be utilitarian, such as the walking stick. Quality of life includes ways of enjoying life, such as entertainment.

The use of wood in Coral refers to the tradition of wooden toys that spans centuries and diverse cultures, such as Ancient Egyptian. There are designers of contemporary wooden toys, such as Playsam.

Some lessons learned:

Challenge your assumptions:

There are many assumptions that sighted people make in their interactions with the world, such as using landmarks for providing directions visually. When visual cues are inaccessible, a new mode of communication needs to be established. In developing Coral, consideration was taken in making components accessible tactilely, such as the power switch.

Many sighted people were drawn to the textures of Coral. Their visual study of the forms often led to touching and exploring the cube.

Prototypes:

The first prototype was made of cardboard, which was useful in understanding the basic interaction with the cube. The second prototype was made of Open Beam and acrylic, which helped in understanding weight and robustness. The third prototype emerged from tests of wood and textures.

coral_cardboard_prototype

coral_open_beam_prototype

prototype_wood-1

Differences in Visual Texture and Physical Texture:

Laser etching in wood can produce textures that are visually distinctive. However, the textures feel smooth and indistinguishable to the touch. Methods were developed to produce physically distinctive textures for Coral. As a by-product, the visual textures were more striking.

coral_textures

The blind painter John Bramblitt discovered the ability to paint by identifying differences in the textures of color paints. He used “haptic visualization” as a means for seeing based on touch. Similar ideas were useful in developing the textures. Players can “see” their location on the cube by touch.

Happy Accidents:

For our team, new discoveries were made with the laser engraver. We discovered that a laser engraver was capable of such varied, dimensional textures.

Due to the laser engraving method, the wood also developed the appearance of a glazed finish. The laser engraving also created a warm wood scent that lasted for days after engraving.

coral_stipple_texture_detail-1

Conclusion:

Coral introduces a new form of entertainment into an underserved community as well as a new perspective on gameplay to an existing larger group. Coral also occupies a novel niche of wooden electronics. It is more dynamic and interactive than traditional wooden toys. Similarly, it provides a unique tactile interface that is missing in modern audio games. The greater context is the exploration of traditional materials in a non-traditional context.

View code here.


Table of Unique Harmonic Tones

(Pitch Name: Frequency)

Start-Up: Welcome Tone
e.g. Device Turning On

Sequence:
F#6/Gb6: 1479.98 Hz
D#7/Eb7: 2489.02 Hz
A#6/Bb6: 1864.66 Hz


Shutdown: Closing Tone
e.g. Device Turning Off

Sequence:
D#7/Eb7: 2489.02 Hz
F#6/Gb6: 1479.98 Hz
A#6/Bb6: 1864.66 Hz


Menu Confirmation Tone e.g. Indicates when tap was registered

Sequence:
A6: 1760.00 Hz
F6: 1396.91 Hz


In-Game Tones Based on F Major Scale

F Major Chord

Notes:
F4: 349 Hz
A4: 440 Hz
C5: 523 Hz
F5: 698 Hz


In-Game Tone (Positive) e.g. Player matched pattern correctly

Sequence:
G6: 1567.98 Hz
G6: 1567.98 Hz

In-Game Tone (Negative) e.g. Player matched pattern incorrectly

Sequence:
G1: 48.9995 Hz
G1: 48.9995 Hz

 

Leave a Reply