There has been a lot of discussion about MOOC completion, most of which has focused on completion rates: what percent of people complete a MOOC, and how should we calculate that number? However, what has drawn less attention, but is potentially more interesting, is what in-course activities impact completion. Understanding whether or not different course elements in a MOOC affect completion can potentially help us better understand best practices in MOOC design.
We used data from three of Duke’s MOOCs that recently concluded – The History and Future of (Mostly) Higher Education, Introduction to Genetics and Evolution, and Image and Video Processing: From Mars to Hollywood with a Stop at the Hospital – all three of which began in January and ended in March. A total of 4,973 students successfully completed one of these three classes.
We asked students to complete a survey after the course ended. A total of 1,979 students across the three courses completed a survey. Note that this includes both students who received a Statement of Accomplishment and those who did not. The surveys are roughly split 50/50 between those who completed the course and those who did not.
We were interested to find out what course activities increased the likelihood that someone would finish the course. We asked students whether they participated in various activities – things like completing optional readings, participating in forums, or editing a course wiki. The full list of activities we asked about is shown in the table below. We then ran a statistical model predicting whether a student had completed a course based on their involvement in each of the activities.
We found three especially interesting things in our analysis. First, “Watch lectures” is omitted from most of the models. This is because watching the lectures perfectly predicts completion in two of the three classes. Unsurprisingly, no one completed the courses without watching the lectures in either Genetics of Image and Video Processing. What is interesting is that some students in the Higher Education course did complete the course without watching lectures. This is likely because the Higher Education course focused on delivering content in a variety of new and innovative ways using different mediums. Because the “watch lectures” variable is not significant in that model, we believe this suggests that content can be effectively delivered in MOOCs in ways other than only through video lectures.
Second, we find that viewing the course wiki in the Higher Education course was significantly associated with completion. Students who used the course wiki were 75% more likely to complete the course. This may be because this course had a very extensive and active course wiki; 73% of survey respondents viewed the wiki at least once. Our analysis indicates that participating in a student group in this course is associated with a lower likelihood of completion. However, this is likely not a robust finding since less than 10% of the sample reported participating in a study group.
Third, there is a significant relationship between forum activity and completion, but only in the Genetics class. In that class, students who participated in the forums were twice as likely to complete the course. This could be due to the level of difficulty in this course; students who were actively discussing the material with other students may have learned more. The survey respondents from the Genetics class were especially active in the forums; 46% reported some forum activity.
Note – while taking quizzes is obviously the strongest predictor of course completion, this is essentially a non-finding because the quizzes are the main graded elements of the courses. Therefore, completing the course means, by definition, that students completed the quizzes. We kept the variable in the analysis simply to confirm that the analysis was accurately modeling course completion.
In the context of medical research, it was recently confirmed that 80% of conclusions drawn from “retrospective studies” such as yours failed when subjected to double-blind testing. The fact that variable X is statistically correlated with MOOC completion gives only very weak support to the hypothesis that if you achieve an increase in X, there will be more people completing the courses.
How about approaching the subject of video lectures with some facts and some insight? For various courses, Coursera provides one of more of the following: (a) machine-prepared transcripts of the lectures (not infallible, but generally OK down to the level of distinguishing “j” from “J”–which, unfortunately, is important in some quantitative courses); (b) slides, such as are used in the lectures; (c) course notes, sometimes excellent, and found in the “Wiki” section.
Given that half of Internet traffic is NetFlix + YouTube and that TedTalks is popular and Udemy sells, for sure many students want just the video. I would argue, however, that one can learn much more efficiently from the other media, than from the videos, and this effect would be enhanced if the non-video materials were collated and synchronized (which has not happened up to now).
In the “Higher Education” course, unusually for Coursera, there was no transcript, but some weird form of notes was provided instead.
You have left out some very important variables. I’ll bet that the easier it is to get 100% on the quizzes (no time limit, explicitly open book, infinite attempts allowed, no trick questions), the higher the percentage of those who get to the first quiz that then proceed to complete the course. I’ll also bet that Daphne knows this obvious fact, and all Coursera instructors are “coached” accordingly.
Seemingly all except your Cathy N. Davidson play by this unusual form of the rules (while Cathy flaunts her disbelief in all non-peer assessment).
There is an important causal relationship between course length and course completion, also not lost on Coursera, which you do not consider. To get more completions, make the courses shorter as well as easier.
I believe in-video quizzes, a specialty of Udacity, are an engagement device that will increase completion if it is easy to get them all correct (at least on a permitted second or third try).
I’ll also bet that peer assessment is not only invalid (see the unpublished Arkiv paper by Koller, Ng et al. purporting to demonstrate the opposite) but also onerous to the student. It discourages completion.
You beg a very elementary question: In most Mooc’s, it’s easy to estimate the percent of students using the Fora. I believe you will find that only a small percentage of course completers use the Fora more than very occasionally.
Here’s another variable for you, which could easily be estimated on a sample basis: What percentage of substantive questions about course material that are raised by students receive a reasonably timely and authoritative answer (even if by a “Community TA”)? I’d say it’s typically close to zero.
Then try this last additional variable: an “instructor narcissism index” that could for now be calculated as the fraction of words in the transcript that are “I,” “me,” or “my.” I would expect to see a bimodal distribution of completion as a function of this index, depending on whether students identify with or are repelled by the instructor’s persona.
But the overarching question is why we should at this point be aiming to maximize completions, when that number is already astronomical. This would be a worthy endeavor for the “big 3” promoters. Let them pay more people to research it. The financial benefits to Duke (see the contracts available at the “Chronicle”) would be slight.
For those who had some forlorn hope that MOOC’s could be a means of bring valuable learning and credentials within the financial reach of capable and assiduous students everywhere, the relevant question would be quite different from the one you address: Do course completers (vs. on-campus students) actually learn and retain very much?
There’s already some evidence that the answer is negative. In that case, you’re merely contributing to the substantial and sophisticated effort to sell larger numbers of fraudulent certificates (some bearing Duke’s name) to the unwary.
An example: In Ariely’s recent course, the quizzes on required readings could most efficiently be answered by looking at each question, going to the cited reading, and then glancing at the abstract, proceeding to the conclusions if necessary, and, in the rare instances when more than this was required, working back from there to find the answer from among the multiple choice alternatives. But, would the student who did this systematically and thus qualified for the certificate “with distinction” actually know much about the readings a couple of weeks or even a couple of hours later?
Off-late, MOOCs have enabled hundreds of thousands of motivated students around the world, who lack access to elite universities, to embrace a path toward sophisticated skills and high-paying jobs, without paying tuition or collecting a college degree. I count myself in one of those hundreds of students.
I tried various MOOCs such a s Coursera, Open2Study(Australia), EdX,etc for my technology company Techcuppa. They offered me a wide variety of courses to choose from; I wouldn’t have got such a massive choice even if I was enrolled in world’s best university. I could choose courses of my interest from any university. Basically, these courses have enabled people to put their interests over degrees and universities.