MOOC Minder

An interesting paper on using commitment contracts for better completion rates of MOOCs: Can Behavioral Tools Improve Online Student Outcomes? Experimental Evidence from a Massive Open Online Course

Online education is an increasingly popular alternative to traditional classroom-based courses. However, completion rates in online courses are often very low. One explanation for poor performance in online courses is that aspects of the online environment lead students to procrastinate, forget about, or be distracted from coursework. To address student time-management issues, I leverage insights from behavioral economics to design three software tools including (1) a commitment device that allows students to pre-commit to time limits on distracting Internet activities, (2) a reminder tool that is triggered by time spent on distracting websites, and (3) a focusing tool that allows students to block distracting sites when they go to the course website. I test the impact of these tools in a large-scale randomized experiment (n=657) conducted in a massive open online course (MOOC) hosted by Stanford University. Relative to students in the control group, students in the commitment device treatment spend 24% more time working on the course, receive course grades that are 0.29 standard deviations higher, and are 40% more likely to complete the course. In contrast, outcomes for students in the reminder and focusing treatments are not statistically distinguishable from the control. These results suggest that tools designed to address procrastination can have a significant impact on online student performance.

Thoughts?

3 Likes

So cool! Thanks so much for sharing this. Obviously heartening for us to see more scientific evidence that commitment devices actually work. I’m a bit surprised that they got no statistically significant effect from the focusing and reminder tools (even the distraction-blocking tool, which I think of as a mild commitment device).

Related: Beeminder’s Code School integration. And Khan Academy looks like it will be particularly easy for us to integrate with, so you should upvote that suggestion.

That is a pretty weak commitment device though - it doesn’t help at all with the I need to study tomorrow, I need to bind myself to it today problem, which was the key for me.

Admittedly N=1 is too little, but I have made far more progress on my Udacity course with Beeminder than before I used Beeminder. My best guess is that if you made a class use Beeminder we would see two things: a massive increase in posting fake data (self-selection being the reason it is so low now) and a huge increase in participation in the course compared with any control group.

3 Likes