Invited Talk @ EdLab, Columbia University on Multimodal Learning Data

Invited Talk @ EdLab, Columbia University on Multimodal Learning Data

Two weeks ago I had the special opportunity to meet with the folks at EdLab – Teacher College, Columbia University to help plan the first event at their new Learning Theatre, a ~2000 sqft space that is going to combine a reconfigurable learning environment with a huge array of data capture and interactive technologies and visualization capabilities. I think this space will really help us think deeply about the design of future active learning spaces, am I’m really excited to be a part of it!

As part of the meeting Matthew Berland and we invited to give a talk on the future of learning spaces and multimodal learning data.

The people at Ed Lab videotaped the event and broadcast using a pretty neat system that allows for participants to annotate the talk in real-time. You can watch the whole thing below (or click here to see the full site):

Connected Spaces

The Connected Spaces Framework (CSF) is a lightweight, customizable framework for connecting multiple distributed makerspaces together. For instance, CSF can log an individual participant’s presence (using either RFID or Bluetooth) in a makerspace within a network of makerspaces and track their current activity, as well as the growth of their personal learning trajectories and “skill” acquisition.

Figure 1: The Connected Spaces Dashboard

In each space, a large ambient dashboard shows the nicknames of all the makers currently present, their current activities, and any skills in which they have gained “proficiency.” The dashboard also shows all other currently connected and active makerspaces and their participants’ names/nicknames, activities, and proficiencies. By providing this information, CSF attempts to answer the two main challenges in distributed learning spaces: 1) How do we give participants insight into the skills of their peers to help know who to reach out to when they need help with a specific maker-skill related problem (e.g., 3D Printing)?; and 2) How do we give learners a sense of the distributed knowledge community they are a part of?

CSF also works to support distributed engagement and collaboration across makerspaces by embedding a lightweight video chat portal in each dashboard, which can be turned on to connect makerspaces together for real-time collaboration (see Figure 1). We are currently developing a series of RFID enabled blocks that can be placed on sensors near the dashboard that will allow one space to reach out and “poke” another space to initiate a chat session. The blocks will be able to specify which space a group wants to talk to, and what they want to talk about (e.g., A question about 3D printing, Coding, or just to say hi). Our goal is to make this customizable by each network of spaces, to allow them to determine which spaces they want to connect to and the topics they would want to connect on.

Each student in the CSF is provided with a low cost NFC card/sticker which they can use to log into the makerspace (or a specific computer/station), using a simple tap system (see figures 2 & 3 below). Once a student logs into the system all the relevant information is are logged in the CSF database (e.g., student ID, login location, time logged in). The collective student data can then be easily exported for further analysis (e.g., looking at collaboration patterns, time spent on tasks). In conjunction with other analytic approaches, this data can also be used to help determine student skill progress and learning trajectories, which can be shown to both students and teachers.

Figure 2: An example of the CSF RFID login screen
Figure 3: An example of the CSF activity picker



Kids playing on the Oztoc tableOztoc is an immersive world where children design circuits to lure bioluminescent creatures from the deep. The game is a first-of-its kind museum installation, combining a state-of-the-art multitouch tabletop display with tactile physical blocks. The result is a hybrid game where students touch physical blocks to create virtual in-game circuits.

Example of the Oztoc table in use

Oztoc situates participants as electrical engineers called in to help fictional scientists who have discovered an uncharted aquatic cave teeming with never-before- documented species of fish. The aquatic creatures who live in this cave are bioluminescent, and the visitors are asked to help design and build glowing fishing lures to attract the fish so that the scientists can better study them. Participants place wooden blocks, which act as electrical components (i.e., batteries, resistors, Light Emitting Diodes or LEDs, and timers), on the interactive table to create simple circuits. (The table recognizes the blocks via fiducial symbols). Creating a successful circuit (one that has the correct ratio of resistors, batteries, and LEDs) causes the LEDs to glow and lures the fish attracted to that type of light out for cataloging. In order to catch all the different fish, players must experiment with creating circuits with different colors (red, blue, or green) and numbers (one, two, or three) of LEDs.

Oztoc Tablet Screen

In my role in the Oztoc project, I am developing analytical approaches for understanding how Oztoc provides participants with unique opportunities to set their own learning goals, collaborate with peers, and act as experts and mentors. I also designed a real-time companion tablet application for the exhibit that uses real-time data mining and user models to let museum guides know when participants may be struggling in order to provide the participants with timely support. This tablet provided the guides with critical information about users interactions that as always available, but was too difficult to track in real-time without technological support. This tablet brought this awareness to the surface and helped the guides make better informed decisions on-the-fly.

What can a management technique teach us about classroom learning? Using the Johari window model to support inquiry learning

What can a management technique teach us about classroom learning? Using the Johari window model to support inquiry learning


At the end of every academic year I like reflect on all the work I’ve done from a broader, more macro perspective. I do this because I find when I’m caught up with the “doing” of something, I miss all the interesting ways it might provide insight into my other work.

A great example of this came up at the end of a third year undergraduate course I teach on Organizational Theory and Behaviour. A major goal of the course is to get the students better understand themselves as members of a team through a series of self-reflective and collaborative management techniques. One of the techniques I use is the Johari window, which helps people better understand the relationship/contrast between how they see themselves and how others see them.

The idea of the Johari window is simple, users and their peers select a set of 5 or 6 adjectives (from a pre-determined list of 56) that they feel describe them, and understanding comes from the similarities and differences between the sets.

The classical Johari window with its four “panes”, the red lines show the goal of the exercise: to grow the “open area” and reduce the other three

There are four “panes” in the window, each showing a different type of personal awareness: the open area, shows what I know about myself and what you know about me; the blind area, is what you know about me and I don’t know about myself; the hidden area, is what I know about myself but you don’t; and finally, the unknown area, which are things that aren’t known by myself or by you. The goal of the Johari window is to expand the open area and reduce the other three through the processes of self-disclosure, feedback, and shared/self-discovery.

In my management class, the students love this activity — it makes them aware of the gaps in their own perspectives of what they know about themselves and what others know about them.

It wasn’t until after the course was done and I took the time to reflect, that I realized my research on learning and collaboration (particularly in K-12 classrooms) could be viewed through a similar lens. Although the traditional Johari window focuses on personality traits, there’s no reason we couldn’t re-apply it as a way to thinking about students’ understanding of any topic under investigation (e.g., climate change or ancient Greek civilizations).

This becomes particularly powerful when students are engaged in inquiry-based curriculums — curriculum which focus on complex, open-ended problems, and where students work together to determine paths for investigation and develop solutions. A major challenge in these kinds of environments is how to make individual students aware what they knowabout a topic, what their peers know about a topic, and also critically whatnobody knows about it. Sound familiar?

The challenge in an inquiry curriculum is how do we make the unknown visible to the students and the teacher?

In an inquiry curriculum, the similarities and differences between individual students’ knowledge and understanding is often more subtle and complex than in a traditional Johari window. Making the nuanced ideas that individual students hold “in their heads” visible to themselves and others requires more than just 56 adjectives.

I started to think about the role that various technologies or technology supported approaches could play in making these ideas visible and to help students in reducing the “unknown” in their investigations. I started to map them to a Johari window in terms of how they could increase the class’ open aera (which I began calling the community’s knowledge), and reduce the other three areas (blind, hidden, and unknown).

A Johari Model for Inquiry Learning

A Johari window reimagined for inquiry education

Searchable/Filterable/Connectable Notes:

By giving students the opportunity record their ideas digitally for the rest of the class to see and respond to, we can provide opportunities for reducing the blind and hidden areas.

These notes can also be a place for students to ask questions of their peers, or to work as a class to develop hypotheses. By making these open to the whole class, we provide opportunities for feedback, discussion, and for identifying unknown areas for further investigation.

Tags and other Semantic Metadata:

The use of user submitted tags and other forms of metadata can allow systems to automatically connect ideas together and help students find trends that they may have otherwise missed, which can significantly increase the class’ open area.

These tags can also be used to create aggregated visualizations of student work (such as polls, or quizzes), which can show gaps in the class’ knowledge or possible conflicts of ideas, making them particularly valuable in reducing the unknown area.

Software Agents and Data Mining:

A combination of software agents (small pieces of software that can respond to emergent patterns of individual or whole class activity — a simple version would be something like recommender systems for Netflix) and data mining can also play a key role in increasing the class’ open area.

For instance, agents can track the work of students and find complementary patterns with the work of their peers and take action, such as recommending the students collaborate or read each other’s notes. Encouraging students to collaborate may also help identify gaps in their knowledge, further reducing the unknown area.

Expert Facilitators:

Although not a technology, expert facilitators, such as the teacher or professionals in the field of study (e.g. a marine biologist) play a critical role in support class inquiry. These facilitators can recommend possible sources of information, best practices, or methods for approaching problems and overcoming roadblocks — which are all critical in reducing the class’unknown area.

The goal of a Johari window for education: An increased open area of community knowledge

Managing any classroom, let alone one engaged in persistent inquiry, is a challenging task. As educators and designers, we need to regularly re-frame how we see the classroom, in order to gain new perspectives on how to support student learning. In this case, applying the Johari window to classroom practices helped me better understand the challenges faced by students when they are collaboratively engaged in classroom inquiry, and how technology could play an important role in it.

Hopefully by sharing my own insights, I’ve helped us as educational researchers and teachers to increase our open area and reduce our collective unknown area. Feel free to share your thoughts on this and let’s grow our community’s knowledge further.

Three possible interfaces for wearables in the classroom: 1) Alerting a teacher that a group has a question; 2) Approving student work or asking them to resubmit it; and 3) Giving quick assessments of student work

Orchestrating classrooms: A huge challenge and a huge opportunity

I’m excited by the recent release of the Android Wear platform, as it adds to a growing suite of technologies that will transform how teachers manage, or “orchestrate”, their everyday classroom activities.

Orchestration can take on many forms – for instance, students may need the teacher to review their work before moving on to another activity. Knowing which students need teacher intervention and when are core orchestration challenges.

Doing orchestration right is both very hard and very critical.

In 2013, the European Network of Excellence in Technology Enhanced Learning (STELLAR), highlighted orchestration as one of its “Grand Challenges”. It has also been one of the main focuses of my own research for many years.

A big question that has come out of this research, and one that wearable technologies is particularly well suited to answering is: How do we support the teacher in being an active teacher — one who is able to respond to changing class dynamics and intervene when they are most needed?

Most technology-based approaches to this challenge have fallen short. Much of this stems from the kinds of interfaces and paradigms of use we’ve forced teachers into using. These often end up either:

1) Promoting a “heads down” experience, where a teacher has to constantly look down at a tablet/computer (rather than the students in the class).


2) Providing information post-hoc (after an activity is completed), which while valuable, does little to help the teacher in at-the-moment intervention.


When a teacher’s head is in a tablet they’re not watching the class
When a teacher’s head is in a tablet they’re not watching the class

Wearable devices have considerable promise in addressing both of these issues — by helping teachers be aware of the state of the class, and empowered to take meaningful action, without requiring them to keep their heads buried in a tablet.

Wearables — lightweight, ambient, and context aware

One thing I have learned from my own work is that if you give a teacher a tablet and they don’t need it all the time, two things are going to invariably happen:

  • The teacher is going to spend lots of time looking at it when they don’t need to.
  • The teacher is going to eventually put it down and miss the things they need to know.

Wearable technologies have the potential to dramatically change classroom orchestration

Free the hands and free the eyes

With wearable technologies such as watches, the teacher doesn’t have to carry around an obtrusive and heavy tablet. Instead the teacher is free to ignore a watch until it is needed. The teacher can focus on what’s happening in the classroom reducing the chances of missing critical information.

Be aware and be meaningful

By its nature, a watch sits at the periphery of the wearer’s attention, only requiring direct (or “center”) attention when needed. The ability to move between these two levels of attention sits at the heart of the profound affect that wearables could have in supporting classroom orchestration.

By connecting the watch to the other devices in the room, we can have the watch alert the teacher when critical events happen. For instance, if a group has a question, they could send an alert (via their group’s tablet) to the teacher’s watch. If the teacher is working with another group, he/she can finish up with them first. The group with the question, knowing that the teacher has been alerted, can continue to try and solve the problem themselves, rather than spending their time with hands in the air trying to get the teacher’s attention.

What’s particularly compelling about this is that the teacher is not forced to immediately act upon the information, but they can be made aware of it, and can act upon when they deem appropriate.

We’ve now maximized everyone’s time while reducing their need for constant monitoring — this is a significant and powerful shift from regular classroom practices.

It’s more than just information, it’s a discussion

Supporting teachers in orchestrating classes with ambient technologies has had significant success with projects like Lantern — which used physical towers of light that students could press to get a teacher’s attention during math tutorial sessions.

Lantern: A light tower used to let teachers know when students have questions in class
Lantern: A light tower used to let teachers know when students have questions in class

However, coupled with rich data mining and real time messaging, the opportunity to make the alerts more flexible and adaptive to emergent class patterns is particularly compelling for wearable technologies. Networked wearables can allow two-way interactions between teacher and students, promoting more dynamic exchanges and flow of classroom activities. The teacher can let the students know he/she will be there shortly, approve their work, ask them to refine their ideas, or even provide quick formative assessment — all with simple commands from their wrist.

Three possible interfaces for wearables in the classroom: 1) Alerting a teacher that a group has a question; 2) Approving student work or asking them to resubmit it; and 3) Giving quick assessments of student work
Three possible interfaces for wearables in the classroom: 1) Alerting a teacher that a group has a question; 2) Approving student work or asking them to resubmit it; and 3) Giving quick assessments of student work

What, and When, and Why for Wearables in Classrooms

One of the big challenges going forward for educational designers will be to understand what information is relevant for the teacher to help them orchestrate class activities. Give them too much or irrelevant information and they’ll tune it out — making it no better than a tablet left on a table!

Not everything needs to be sent to a wearable device, and working closely with teachers on the ground to develop these information flows will be essential to their adoption and success.

Ultimately, smart wearable design should be part of a broader eco-system of technologies, information flows, and teacher moves working together to achieve the desired learning goals of the classroom.


This article was originally published @

LASI Day #3 – Morning Sessions Smorgasbord

Now onto day 3 at LASI 2013 and a lot happened this morning across three panels and a 45 minute breakout (birds of a feather) session so I’m just going to touch on a few things that really stood to me.

Taylor Martin & Nicole Forsgren Velasquez did a really nice talk about the kinds of learning analytics they are using to understand and evaluate student strategies in problem solving (in particular using a game they have developed called Refraction). What they were able to do is break down student solving strategies into 5 different categories (Slow & Steady, Haphazard, Explorer, Strategic Explorer, and Careful), and perhaps more interesting they were able to understand the results each of these strategies tended to produce (here’s link in a bit to their powerpoint to get the nitty gritty). Phil Winne, followed this up with another really interesting talk about understanding thinking by students and trying to get “into” their heads – both these talks really drove home the point that one thing that LA can do is help us get a sense of how kids are thinking about and acting on/within the curricula we design. The last speaker in the session was Sidney D’Mello and he talked about students’ emotions and learning – in particular he offered the approach of “Learning with Contradictions” – an approach that promotes disequilibration of students to promote reflection and learning and definitely resonates with Manu Kapur’s ideas around Productive Failure (2008). Check out the whole session here.

During the breakout session later in the morning Alyssa Wise organized a group of us around trying to get at the big ideas of “What problems do we think learning analytics can solve? WHOSE problems are they?” I thought this was a great way for us to start really thinking deeply about why we’re at this conference and what we’re trying get as a productive outcome to help us in shaping our research and the field. Judy Kay had our group brainstorm what these questions meant to us, and what kinds of small more focused questions could help us answer the larger ones promoted by Alyssa. For me (which is a bit different than some of the others at the conference) the challenge that I believe LA can help address is during complex real-time enactments, especially in unpredictable inquiry activities. TO his end there is an interesting issue about the “granularity” or the scale of the analytics and the interventions that we want to inform/act upon. To me there are at least 3 that stand out as a baseline:

  • Real-time in class, supporting on the fly decisions about classroom orchestration
  • After or between classes, giving more detailed information about the state of the class’ knowledge or performance to aid in scripting upcoming classes
  • After the course/unit, for assessment and also for self-reflection of the teaching and learning outcomes (this might also be really valuable for students too)

Finally Phil Winne reminds us that students are agents, who make choices throughout learning activities, even unexpected ones.

LASI Day #2 – Morning Talks, Learning Analytics, Intuition, and Data Driven Design

A couple of really interesting talks today by some serious heavyweights in the learning, technology, and innovation fields: Stephen Coller (Gates Foundation), Ken Koedinger (CMU) & Ryan Baker (Columbia).

Stephen Coller opened the talks discussing how learning analytics can help us transform education and the challenges of bringing this to “market” (adoption in larger scales). Stephen noted that there is a common goal of most people in education to pursue the improvement of “the instructional core”. What I found particularly interesting is that he touted 3 key ways to  really improve learning gains for students we need: (1) Change the rigour of the content that students are being asked to interact with; (2) Increase the knowledge and the skills of the teachers teaching the content; or (3) Alter the relationship of the student to the teacher and the content. Each of these are really powerful (but complex) means for making these changes and Stephen pointed out that changing one will probably have an effect on the other three as well.

He also noted the minerva project which approaches higher education through an interdisciplinary, real-world, authentic problem based curriculum. He noted that this form of learning may very well find its way into smaller college settings. This growing of authentic skills which are applied to real problems is really interesting and I could see it being a real motivator for students within such curricula.

Ken and Ryan did back to back talks on how they focus on what drilling deeply into student interaction and result data has told them about their designs and how to either improve them or to give direct insight into teaching and instruction practices. What really stood out here was the notion of “expert blind spots” – that we as researchers too often feel we know what we know (when in truth we might be missing critical factors or insights because we aren’t looking at  the issue objectively). Instead Ken and Ryan showed how LA and data mining can reveal things about our curricular designs that we may not have seen (and could be adversely affecting our designs). Ken stated that “Intuitive design is not reliable”, and that careful analysis was more fruitful. I challenged him slightly on this noting that intuition is sometimes need in new or innovative designs (where we don’t have rich data from which to build on), he gave me one of the most memorable quotes of the conference “The hare of intuitive design and the tortoise of cumulative science”.

Quick thoughts on the opening sessions of LASI 2013

After listening to a really nice opening panel at LASI 2013 and the idea of Big Data and Learning Analytics (LA) a couple things came to mind about this emerging field.

I’m always worried about the ideas of LA are going to put learning too much “on rails” – that is to say that we automate the process so much that students and teachers are taken out of the decision and learning processes by simply crunching the “data” and making decisions for them. I’m heartened to see that this concern is shared by the panel as well.

Alyssa Wise mentioned that LA needs to be “learner centered”, which I think is vital, even as we begin to gather and process all of this data to make sense of it we need to remember that it’s about the students and all of our practices need to be focused on this and how we can help and enable learners to learn. I was glad that Dan Suthers also pointed out that learning at it’s core is a complex phenomena, but that there is a promise being held out by LA to help us “understanding and manage learning in its full complexity”, and how we can it help optimize learning. My big question and one that I think should be central to this whole conference is what we mean by optimizing learning? A lot of the ideas in this conference is about this idea of optimization and I hope that we continue to discuss/debate what optimization means within complex and varying learning communities and approaches.

This idea by Dan goes very nicely with George Siemens idea of the increase of learner individualization, and with Phil Winne’s of engaging learners as participants in an ecology of experimentation. We want students to be authentic drivers of inquiry, investigation, and knowledge construction and we want to leverage LA as a means of aiding them in these processess – by connecting them new peers, new resources, new ideas that they may have been otherwise “blind” to (similar to what Dan said about weak ties).

My personal hope is that LA does live up to this ideal of really empowering learners to learn in ways that otherwise would be impossible (or prohibitively time consuming), and also critically giving teachers insight into the state of their class’ knowledge (and perhaps deeper information of the “global” state of knowledge) to drive learning and exploration in exciting new ways.

Only one morning in and so far very interesting and exciting – looking forward to the next few days!

Smart Classroom Concept Video

This is a short video shows one of our earliest prototype concepts around the kinds of interactions we envision taking place in a fully interactive smart classroom.

This video shows a teacher launching an activity using a multi-touch table, which then displays the activity content on the large format displays throughout the classroom.

The students can move freely throughout the room and capture data on their smartphones.

The students then “connect” with an interactive table which displays each students’ collective artifacts. The students can then discuss and share these artifacts using the multi-touch table as the mediator.