Physical Computing: Inventing and Interactive Sound

This summer, I worked with Haley Gravis for a Smithsonian Sound and Sculpture Jam class that meshed sound, touch-based interactions, and art making. I have plans to include these projects in my Friday STEAM Art Lab sessions with my K-6 students. We use Makey Makeys (MIT Media Lab) for each of these projects. I am going to share how these work and what questions we have the students investigate, but you should watch this brief video if you have never heard of a Makey Makey. Basically, a Makey Makey is a preprogrammed augmented keyboard that uses simple circuits for key strokes. It does not require software because the computer believes it to be a keyboard when plugged into the USB, which also powers the board. If I touch the EARTH and SPACE, the computer reads it as an additional keyboard pressing SPACE BAR. If I then attach a banana to the SPACE with an alligator clip and physically touch the BANANA and EARTH, the computer reads it as an additional keyboard pressing SPACE BAR. Why? Because humans and bananas are CONDUCTIVE and can close a circuit.


Photo Credit:

So how can this be used in STEAM education? The explorations in electricity and circuitry automatically bring science to the table. You are working with a computer and possibly using Game Maker, Virtual Pianos, or Keystroke Software, so technology is present. A major factor of projects with the Makey is engineering, as you have to construct multiple functional open circuits while creating consistency, visual organization, repetitive reliable use, avoid unexpected series and parallel circuits (conductive materials touching each other), and constantly revise wiring plans. Math can be included based on the intention and use of the Makeys, but is not regularly included with my projects. And Art integration key points will be reviewed with each project below.

As you are probably aware of the country’s STEM/STEAM early-stage initiative, many attempts presently fall short of authentic integration, innovation, and investigation (my favorite “I” words). A lot of “STEAM education” focuses on one subject area and touches base with another’s concepts. Does it truly matter if you can touch 2-5 subjects within your projects? Does that make more effective learning? In my opinion, no. If you want to generate meaningful STEAM experiences for your students, you must stop seeing them as separate subjects of study. There is not a single innovator in the field that thinks, “I wonder how much of my work was science, art, engineering, or math… Does the fact that I typed my proposal mean I included technology?”

As an art teacher and a maker, it’s not about how much art goes into the project because I see STEAM as a unit of workable concepts. When we talk about building a wearable circuit that comfortably plugs into the computer and produces programmed sound while giving visual cues to consumers on where to touch, we seamlessly bounce between content areas. I leave it to my students to make the connections with what they’ve done in their other classes and learning experiences, both from the past and in the future. What matters to me is not addressing an initiative or an aesthetically pleasing product, but the way students are connecting information independently and collaboratively while constructing and revising. In Art Lab sessions, we focus on the experience rather than a product.

What is beautiful about Makey Makeys? The product generated is the experience for both the inventor and consumer.


Projects and Experiences Led by Haley Garvis and Meredith Cosier

1. Graphite Piano and Other Conductive Materials:

An excellent way to introduce the Makey to any age is using familiar materials and ideas. What is a piano? Multiple keys that you can press to produce sound. What is a computer keyboard? Multiple keys that you can press to produce letters. When we open virtual pianos on the computer, the keyboard becomes a piano. Let’s take a look at the MOUSE CLICK. When I click on a virtual key, it works like a keystroke and plays the note. Plug in the Makey and touch the earth and mouse click. What is happening? Can I make that happen without touching the earth? I relate it to flashlights. Will the light turn on if you down screw the top down? No, you must complete the circuit. I can complete the circuit because I am conductive. Who else is conductive? What else is conductive?

Students plug into the bottom WASDFG keys and attach to their graphite piano keys. They have to figure out how to make them all work. Sometimes there is NOT ENOUGH conductive material (graphite). Sometimes they are not touching the earth. Sometimes the clips are not touching contact points. Sometimes the clips cross over and all of the keys start playing at once. They discover all of this as they go. Then I like to move on to making a piano with new materials and use it to discover what else can carry electrons and complete a circuit.

My favorite is YARN and CARDBOARD. Then wet them and ask why it suddenly works. You can also make conductive paint with acrylic and powdered graphite. Or rubber cement and powdered graphite, cause everyone likes the smell of rubber cement…

2. Playdoh and Modeling Clay Pianos:

On MoMA Art Lab iPad app, there is a activity where users can attach sounds to Elizabeth Murray’s Do the Dance, “This painting is made up of different shaped canvases that she combined to make one work of art. The title comes from a song that Murray listened to while she made the painting. What sounds do you imagine she heard?” I thought about how this could be applied to a instrument that could stand alone as an artwork, but also work as an interactive object. We looked at a variety of the shapes, abstraction, and color that Murray used. Then we wired our plastic take out containers (**see important advice below) and began with playdoh keys.

Playdoh is conductive. Modeling clay is not. That means it can work as an insulator. Independently, the physically difference is obvious. But when put together, you get a object that a user must feel to find the keys that make sounds. At this point we use Soundation (free, online GarageBand) or choose sounds, so our instruments are unique. You can also have students record sounds. We have used multiple ways to program the keystroke sounds. Game Maker will allow you to do this by loading the sounds and creating simple sound objects. Soundplant allows you to set sounds to keystrokes. If you do not want to use software, you can use online virtual pianos.

* You could add an element of math and home economics by mixing your own salt and sugar based dough. Sugar based dough is an insulator.

**Puncture the containers beforehand. We originally used FOAM-CORE, which IS CONDUCTIVE. We then switched to cardboard. When cardboard absorbed the moisture from the playdoh, it became conductive. The playdoh will dry out. The modeling clay will never dry. You can decide if you want to keep the containers for future use. Make quick work of the “puncturing prep” by using a heated needle (NOT with students around, obviously). I did it at home while watching TV with a candle, pliers, a fan, a thick sewing needle, and an open window.

3. Talking Cities and Communities:

This is a collaborative project with each individual completing a building or element for the community, city, or locational space. They voice record a sound effect for that object. When the individual parts are complete, the group works together to wire the necessary circuits and program the sound and keystrokes. We used Game Maker and handheld .WAV mics. Some of the examples include a hospital with an ambulance sound, a community hot tub with relaxing bubbly sound, and a complete music club with multiple songs and commentary throughout the room.

4. Storytime:

This is akin to having a talking still image. Students individually wrote out a simple story or interaction between 3 elements or characters. They recorded a voice or narration for each element. A user had to figure out the order of the story by touching the elements. Students also created environments or backdrops. Game Maker files were modded to fit the new sounds.

5. Talking Mask/Identity Mask:

Originally, our proposal to students was to create an identity mask. But, we were nearing the end of our camp and most wanted to keep making Soundation music. So, masks became personal track lists and samples. Other options were talking narrations or sound effects. Game Maker was used (most of our kids are familiar with it because I use it in other camps). Next year, we intend to focus on building an identity mask that plays elements of your identity. This is the coolest part of this project- Haley decided everyone should make the earth in the handle and create chest plates. This meant it was easy to switch to the next person and that consumers had to touch the creator and then the mask to complete the circuit.

Game Controller Projects Led by Luke Meeken and Meredith Cosier

Rethink and rebuild the way you physically play a game. Luke and I have taught video game and digital art programs where the students created floor, handheld, and group controllers to play video games. The floor is typically made with aluminum foil and tape. Handheld can be made with anything conductive. We have had kids that created controllers with their lunch grapes, cups of water, playdoh, aluminum foil, copper tape… This year, we even had a student that prototyped a tilt controller without a tilt sensor. The group controllers were created from challenges such as this:

You are going to play space invaders. However- the Earth is on the floor, the left is on the wall, the right is on a desk, and the shoot is 7 feet away from the earth on the floor. How can you play? Students have to find creative solutions to link bodies and act as a circuit.


Things That Relate and Stuff You Should Check Out

Want to know a secret? If you catch me in person, I can talk non-stop about physical computing, maker movement, and game education. In the digital world, its exhausting organizing all my experiences on to a page with photos. So here are some things to check out if you like the ideas you saw here.

Currentlab’s Physical Computing Module: Overview of additional possibilities and lesson ideas

Squishy Circuits:

VERVE: Another type of Augmented Keyboard with 7 Sensors

Pure Conductive Artsy Awesomeness

Little Bits Circuits:

Go to a Maker Faire


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s