Making Mistakes on the Big Screen

I've been watching a massive, open, online course (MOOC) on artificial intelligence (AI), evaluating whether I should email the Vanderbilt students who will be taking my Fall 2013 AI course at Vanderbilt, suggesting that they take the MOOC over summer as (optional) prep for the on-campus course. I hope to make the decision to email the students by July 1 or so; its a decision that will depend on the overlap in the content coverage between my on-campus course and the MOOC, as well as the quality of the MOOC.

Last week I was sampling some of the MOOC online lectures, and on one there was a question from a student regarding an apparent inconsistency (on uniform cost search). I looked closer, and indeed, there was an error, stemming clearly from a "clerical" error, probably because of an interruption in the video shooting of lecture sequence.

Now this wasn't a horrific error by any means, again clearly stemming from a lapse of book-keeping, and its the kind of error that I have made plenty of times in a live, in-class lecture. The difference is that in a live lecture to 25 students or so, some student will usually speak up asking about the inconsistency, perhaps first being prodded by me because of the inexplicable look of confusion on their face, and it will often get cleared up immediately. I've also been known to deliberately plant an inconsistency to see who is paying attention, but those get cleared up quickly too.

But I don't know what I don't know, so who knows how many times a careless mistake in class has gone uncorrected!?!

Like my sometimes in-class lectures, the video's error was also the kind that, if uncorrected, could result in a conceptual misunderstanding (aka a conceptual "bug") on the part of the student, and in this case, there are potentially thousands of students. However, this particular MOOC has no required start time, its completely self-paced, so there may be an insufficient number and diversity of students taking the MOOC now so as to virtually guarantee that at least one of them will adequately address the questioner's confusion. And this is undoubtedly another issue that needs to be addressed with completely self-paced MOOCs.

In any case, I wrote a comment in response the question, noting the error, explaining what I think happened, and gave the correct answer. I got some "karma" points as a result of contributing my comment from the MOOC's host site, which is a nice touch. Within a day, as if to keep me humble, one of MY videos (on B+ tree theory), on my YouTube channel, received a comment pointing out a typo. This was an error buried in some text, easily missed, but it could result in a misconception by some students, so I was glad that I could give the person that pointed it out a thumbs up, and then respond with a comment addressing it.

These errors highlight some interesting things. First, I had thought that if one were giving a lecture for the world to view, rather than simply reviewing your notes as you rushed off to class, then the instructor would make sure that the world-viewable lecture would be virtually error free. Not so, as the MOOC lecture illustrated -- the error was introduced in what was very possibly a rushed production, perhaps a rushed video re-shoot. While I put many hours into nailing down the pedagogy of my lecture with the typo, the typo could have undoubtedly been caught by a careful second reader, and the fact that I didn't have another set of eyes to review my videos illustrates the implications of other resource limitations, like no teaching assistants.

It's tempting to think that improved technology for near real-time shooting and splicing would enable an instructor to very quickly fix errors without having to re-record the entire video sequence, and it might, but such real-time capabilities to correct local errors has to be trivial for it to be used realistically -- that same lack of available time and second eyes that led to errors in the MOOC example and my video, will also be a major impediment to making corrections -- the work to do so must be near trivial, and the technology is not there yet.

However, maybe there isn't a great need for real-time correction capabilities. After all, when the videos are on the Web, comments by a student and me to the MOOC error, stand as a correction for all future students (unless they are erased). And Kay M's correction of my video, and my confirmation, are corrections to that video, which stand "forever" thereafter! And this may actually be an improvement over the in-class situation, where some errors may go missed entirely.

If instructors' egos can take the hit, then these community corrections are probably sufficient, but they also make me a great believer in investing substantial effort in pre-production, to include rehearsal, but also post-production and editing, and investing more time and human resources into what are (in my two examples) "home-spun" videos -- you can't short-cut your way to great product.

Still, its nice to know that even the smartest and most educated people make mistakes, sometimes careless, sometimes more. To some extent though, the clunkiness and carelessness that I am seeing in some computer science MOOCs stems, I think, from a certain "volunteerism" among CS people when it comes to all things technology. For example, it was no accident that when the Web first started up in the mid-to-late 1990's that CS departments were among the first to host Web sites, but they often quickly became clunky relative to other academic sites, where these other sites were run by departments that were paying professionals to design and host sites, while CS departments were still relying on "volunteer" efforts by faculty and graduate students. You get what you pay for, and I don't expect clunky MOOCs to be around too much longer.