Harvard, MIT researchers look at bright side of data from first year of online learning platform edX
Harvard University and MIT on Tuesday released data from the first year of their joint online learning platform edX, which shows only about 5 percent of course registrants earned certificates of completion, less than 10 percent viewed more than half of the course materials, and more than a third signed up without ever looking at course materials.
But researchers leading the ongoing study of edX’s progress found some positives in the data they’ve compiled thus far.
The findings, for example, show that nearly two-thirds of the 841,687 registrants viewed at least some of the course materials.
“Registrants are engaging with courses in diverse ways, and many instructors are deliberately building courses that honor diverse forms of participation,” the authors of the report, Harvard professor Andrew Ho and MIT professor Isaac Chuang, wrote.
“Certificate earning is one possible learning pathway,” they wrote. “Others include simply watching videos or reading text. Some registrants sample a couple of chapters and then take their interests elsewhere, only to register in other courses and sign up for the second instance of courses. Some registrants focus on assessments to test themselves. Nearly any way that one can imagine a registrant using a course to learn is actually revealed in the data.”
The researchers said course completion rates can be misleading indicators of success.
“A fixation on completion rates limits our imagination of what might be possible with MOOCs,” said a statement from Ho. “A better criterion for success might be for students to complete more of the course than they thought they would, or to learn more than they might have expected when they first clicked on a video or course forum."
The report also says there are some key differences between learning via MOOCs versus learning in a more conventional setting.
“Registration requires no cost or commitment, thus traditional metrics, like certification rates and enrollment rates, miss many new facets of course engagement, such as skilled learners dropping in to learn one specific aspect of a course,” they added.
And, the professors pointed out that while figures like a 5 percent completion rate may seem small, it translates into a large number: 43,196 of registrants who finished their course.
“Small percentages are not small numbers,” they wrote. “An increase in the number of registrants who are not ultimately certified can decrease certification rates, but if it is accompanied by an increase in the absolute numbers of registrants who learn, we argue that it should be regarded positively.”
Yet, even figures in the thousands or tens of thousands can seem small, and perhaps unfairly so, due to the vast reach of the Internet, they said.
“From an on-campus frame of reference, a professor may take years or decades to teach 1,000 students. From an online frame of reference, 1,000 is vanishingly small compared to the sizes of many online populations,” the report said.
The study analyzed a year’s worth of data – roughly 20 gigabytes of data per course – from the inaugural 17 courses edX launched and ran between the fall of 2012 and the summer of 2013. The researchers also interviewed faculty and course teams.
The professors also found that:
- More than 4,000 registrants earned multiple course certificates, including 76 who earned five or more certificates.
- On average, about 50 percent of registrants stopped engaging with a course within the first week or two after enrolling. But after the two-week mark, attrition rates plummeted. The likelihood of a student quitting on a course at a point later than two weeks post-enrollment was 16 percent.
- The most typical course registrant was a man, age 26 or older, with a bachelor’s degree. About 31 percent of registrants fit that category.
- Still, there was diversity among those who signed up for edX courses. About 33 percent of registrants reported that having a high-school education or less; 6.3 percent were 50 or older and 2.7 percent had an IP address or mailing address from a country on the UN’s list of “Least Developed Countries.”
“While typical MOOC registrants have a college degree already, hundreds of thousands of our registrants do not,” said a statement from Chuang. “Many of our MOOC registrants are from the United States, but 72 percent are from abroad. These MOOCs are reaching many nontraditional and underserved communities of students, very different from typical students on campuses at traditional universities.”
The two researchers plan to release more of their findings, including data sets and interactive visualizations in the coming weeks.
“Open online courses are neither useless nor the salvation of higher-education,” the researchers wrote. “Large-scale, ‘low-touch’ learning platforms will have sectors and niches where they are very useful and others where they are less so.”
“This isn’t just about MOOCs,” Ho said in a statement. “This is about the democratization of learning: Learners are in control. We are at the beginning of an exciting effort to understand how people learn and how to educate well and effectively at scale.”