MIT Media Lab, 2015-2018
Roles: Project lead; Concept; Design; Development; Research
Collaborators: Anneli Hershman, Juliana Nazare, Jim Gray, Mina Soltangheis, Sneha Makini, Marc Esposito, Eric Chu, Susan Fine. Advisor: Deb Roy
Children are natural explorers and tinkerers. This explains the popularity of toys like LEGO, or programming tools like Scratch. To translate the spirit of open-ended tinkering and play into early literacy learning, we created an early literacy app SpeechBlocks. Check the video above to see how it works!
We ran several semester-long studies with SpeechBlocks: a study in a classroom in 2015, and two studies at home in 2017 and 2018. Our main findings are presented below. Check the linked publications if you would like to read about the studies in more detail.
Children enthusiastically received SpeechBlocks. They particularly appreciated the freedom to build whatever they wanted. In the classroom, children eagerly awaited SpeechBlocks time, shared stories of their play with teachers, and asked us if they can take the app home. Below is an example of how children interacted with SpeechBlocks when they first saw it.
Video credit: Anneli Woolf
Children's agency was visible in their deliberate choices of what words to make. Sometimes they created plans and followed them, as was apparent from their self-directed speech. Children cared about what they made in SpeechBlocks and proudly shared it with others. They also often wanted to preserve what they made, prompting us to create such a capability in SpeechBlocks. In the video below, a child decides on which words to make (his name and the name of a beloved cartoon character), proudly shares them with the facilitator, then combines them and explains why he did that.
Video credit: Anneli Woolf
Unusually for an app, SpeechBlocks prompted children's offline interactions with each other. These interactions involved sharing ideas, joint word play, asking others how to spell words, observing others to learn from them. Below is a video in which two children get intrigued by that they made similar things within the app, and engage in shared play.
Video credit: Anneli Woolf
In the class and home studies, we saw six different types of play with SpeechBlocks.
In one of the videos above, you can see children enjoying the comical effect produced by remixing words. Sometimes, remixing became more complicated and resembled the wordplay in poetry or rap: BARACK -> BE ROCK, KAMALA -> COME MALL, RAINBOW -> BRAIN, JESUS -> JESSE. In addition, in the home studies, several children actively experimented with rhymes, for instance: DOVE-LOVE-HOVE-PUOVE, LOP-HOP-SHOP, BALL-TALL-CALL, and CAT-BAT-FAT-MAT-SAT-HAT-RAT.
The first real words children usually tried to spell were their names. It is interesting to track how their spelling evolved over the course of many days. For instance, a child named Addia spelled her name in the following ways: DD -> ADD -> ADID -> DDAA -> ADD -> ADDI -> ADDIA -> AGIHD -> ADDE -> AWI -> ADDIA. In home studies, children also spelled names of their friends, relatives and teachers: e.g. MRGRAY, GRAMY (grammy), DADY (daddy), BROTR (brother), SITR (sister).
Another popular class of words were characters from cartoons and media, as well as popular personalities: JKROWLIN (J. K. Rowling), BRNOMARS (Bruno Mars, a musician), ZAC EFRON (Zac Efron, an actor and singer; the parent commented: “She is all about The Greatest Showman right now"), JOKER, BANE, VENOM and LOKI (from Marvel universe), DRAGONFIRE, FIREDRAGON, WATERDRAGON, DRAGONGOLD (likely inspired by How to Train Your Dragon).
In home studies, children also made words related to their daily lives. Topics included hobbies (SOCCER, FUBOL, ROCK COLLECSHUN), foods (SPGETE, PAPRRONE, MCDONLS), clothes (DRAS (dress), CAPE, SCARF), holidays (EASTREGG, VALNTIN), brands (KFURIG, PURELL), places (VIDANT MEDICAL CENTER, DADS WORK, THEBANK), news (HILLARY, DONLD TRUMP), inspirational messages (LOVEWELL, COMPASSION, ASKFORHELP). Some children challenged themselves to spell difficult words, such as ONAMANAPIA (onomatopoeia), MULTIBULICATION, METEOROLOGI.
As you may have noticed, the examples above include plenty of cases of invented spelling. Literature shows that invented spelling (when children represent phonemes in the word in their own way) could be beneficial for development of certain early literacy skills. In our later studies, we experimented with incorporating invented spelling into automatic scaffolding mechanisms.
In the words that children made, one can sometimes see glimpses of describing a real or imaginary situation, for instance: FEELBAD DOCTORSHOT COLD SICK FLU GURMS or SHELIKES FANSYCLOE PRETTYHAIR NICE SHOES PLASES FASHEN. In the video below, a child accidentally makes an interesting word, and then comes up with a story to interpret it.
Video credit: Anneli Woolf
Children sometimes built words in SpeechBlocks to make the app "speak" for them. For instance: LOVEYOMOM, I AMSOBROD (I am so proud), IWISHIWAS, I HOPE, I DKARE (I don’t care), HAPPY BIRTHDAY TOO YOU, THANKYOU, SEEYOU. A curious example is the child "talking" to the family's dog: CODYSIT, I SAID COME OVER, OPENTHEDOOR.
Children also tried to make SpeechBlocks "sing" by spelling excerpts from songs and tapping on the words in order: BEBEUGON (“baby, you gone” - likely referring to a song by Brian Adams), THIS IS MY FIGHT SONG TAKE BACK LIFE (mimicking Fight Song by Rachel Platten), WATCH ME WHIPNAENAE (imitation of Watch Me by Silento), OPPAGANANG (imitation of Gungnam Style by Psy).
In the classroom studies, we occasionally saw children choosing words in SpeechBlocks in order to copy them down letter-by-letter into small notebooks that they had.
While above we saw focused, “minds-on” play with SpeechBlocks, there was also a large amount of impulsive interactions: random taps and swipes, cluttering of the canvas with words from the word bank to see how many could fit there, building entirely random strings of letters to say “Look! I made such a long word!”, etc. In one play-testing session, a child used the red (vowels) and blue (consonants) blocks on the app’s canvas to play a “soccer match” between the red and the blue “teams”. We suspect that such behaviors may occur when children find the core SpeechBlocks activity of making words too difficult to be enjoyable.
While children built impressive creations in SpeechBlocks, young children (4-6 years old) had extreme difficulties spelling words without adult guidance. Therefore, in the in-class study, one of the researchers had to perform the role of scaffolder, helping children discover the sounds (phonemes) within the words and connect them to relevant letters. This was a labor-intensive task. With simultaneous requests from multiple children, the scaffolder struggled to keep up, leading to children's disengagement. For example, look at the child in the bottom left corner (highlighted with an arrow) on the video below. It became clear to us that such human scaffolding for such a system is not scalable. In our further work, we approached this issue by implementing automatic scaffolding.
Sysoev, I., Hershman, A., Fine, S., Traweek, C., & Roy, D. (2017). SpeechBlocks: A Constructionist Early Literacy App. Proceedings of the 2017 Conference on Interaction Design and Children. pdf
Nazare, J., Hershman, A., Sysoev, I., & Roy, D. (2017). Bilingual SpeechBlocks: Investigating How Bilingual Children Tinker with Words in English and Spanish. Proceedings of the Annual Symposium on Computer-Human Interaction in Play. pdf
Sysoev, I. (2020). Digital Expressive Media for Supporting Early Literacy through Child-Driven, Scaffolded Play. (Chapter 5) Doctoral dissertation, MIT Media Lab. pdf