Crowd-sourced Curricula

For the past couple of years, in Spring 2012 and Spring 2013, the end of semester project in my database course was the design of a database to support an educational social network (https://my.vanderbilt.edu/cs265/course-customization-and-learning-social-networks/), where online educational materials -- video and otherwise -- can reside, be endorsed by compatriots in social networks, be decomposed into micro structures (e.g., in the way that http://www.teachingtree.co/ , and now YouTube, allows indexing of particular segments in videos) and composed into macro structures, with individual topic videos formed into "courses" and online courses (e.g., MOOCs) being formed into curricula; users in the social network can endorse (or renounce) constructs at all of these levels -- micro, macro, and a lot of in between -- with opinions at one level influencing constructs at lower and higher levels.

The project was inspired by my work on the Chancellor's Social Media and Digital Technology Committee (http://news.vanderbilt.edu/2012/02/committee-social-media/), but particularly the vision of the unbundled university (http://www.educause.edu/annual-conference/2011/chronicle-tech-trends-challenges-future-unbundled-university; http://chronicle.com/blogs/wiredcampus/a-chronicle-educause-panel-challenges-for-the-unbundled-university/33812) -- what is unbundled can and will be reassembled, and according to the early visionaries, reassembled in a wealth of ways.

The crowd-sourced micro structures are interesting, and I'll address them at some point, but I've been enamored of late with crowd-sourced macro structures, particularly crowd-sourced curricula. This slide from an ITHAKA S+R talk (http://vimeo.com/53361649)  illustrates that even in October 2012 (not even a year ago -- seems like ages!) someone could piece together what amounted to a CS bachelor's from free, online classes.

This slide is the first in a four-slide sequence showing the possibilities of a CS major online (https://my.vanderbilt.edu/douglasfisher/files/2013/06/DougFisherMOOCsOnCampusPresentation.pdf ).

In the educational social network context, individual and alternative paths through the course repository (http://blogs.the-american-interest.com/wrm/2013/09/12/create-your-own-college-mooc-style/) get endorsed and rejected, until some emerge as consensus favorites. Just before drafting this post, I searched on Google for "crowdsourced curricula" (no quotes) and sure enough, found this: "Crowdsourcing Curricular Design (Helps & Grant, submitted) at http://micsymposium.org/mics_2013_Proceedings/submissions/mics20130_submission_23.pdf -- a nice paper sketching out a similar vision, though with some interesting differences, some of which I really like, perhaps most notably that the crowd-sourcing be by educational professionals, better insuring that the macro structures that emerge aren't the "easiest", but the most pedagogically sound. The paper also has some interesting background that I didn't know of previously.

Attention on macro-level, curricular constructs is receiving increased attention because of a new partnership between educators at Vanderbilt University and University of Maryland (http://news.vanderbilt.edu/2013/09/vu-maryland-mooc/). In particular, Doug Schmidt and Jules White of Vanderbilt and Adam Porter of the University of Maryland are organizing the first transinstitutional MOOC sequence -- not to be confused with a jointly offered MOOC (singular), but a sequence of two, with Porter's Apps development MOOC serving as a soft prerequisite to Schmidt and White's Pattern-Oriented Software Architecture MOOC. This sequence is a curriculum-level construct, an important nascent step towards crowdsourced curricula.

Don't Forget About Regional Brand!

Lots of schools are developing online programs to increase revenue, notably through increased enrollments, or perhaps to protect against falling enrollments in the future. While there are projections of modest increases in enrollments in institutions of higher education in the near future (http://nces.ed.gov/programs/projections/projections2021/sec5a.asp ), they "do not take into account such factors as the cost of a college education, the economic value of an education, and the impact of distance learning due to technological changes". These are such important factors that I think their exclusion from analysis makes projections rife with uncertainty, because the rules are changing.

Usually, when a university offers an online course (e.g., a MOOC) or a much more comprehensive online program (e.g., Vanderbilt's School of Nursing MS programs: http://www.superscholar.org/schools/online/vanderbilt-university-school-of-nursing/) the emphasis is on reaching students across the World, and because travel and part-time residency can be expensive in terms of time and money, allowing students to be solely online, never having to travel to Vanderbilt (or whatever university we happen to be talking about) is important, particularly for older adults, in contrast to 18-22 year olds, who want to continue their education, but can't take time away from work and/or family.

In addition to revenue, online programs can spread and reinforce a university's brand worldwide, entering the university's name into interpersonal discussions around the World, perhaps attracting some students to attend the school in person, in addition to attracting others online.

In all this attention to reaching the World though, we shouldn't forget about the importance of regional brand and other attractions of our institutions to OUR region (I'm reminded of the movie "Rudy", the scene where Coach Dan Devine of Notre Dame tells his football players before a home game with Georgia Tech that "No one, and I mean no one, comes into OUR house and pushes us around." :-).

I wish I had the data to answer the question "how many older adults who are going through online programs would like to attend formal classes in person from time to time?" I wonder about this question because I wonder whether we are overlooking an important population in all this focus on online education -- older adults living within 120 miles (or so) of our university. These are people who may want to take a degree program ONLINE MOST OF THE TIME (because of work and family), but who are close enough that they can and do aspire to attend in person at least from time to time, to meet the professor and their student colleagues! Indeed, some may have grown up wanting to attend our school; they may want their children to attend our school.

I also wonder whether, at some point, as more and more institutions develop online programs that we won't hit saturation points. Our rankings as a university and rankings of our programs will distinguish us among other online programs to be sure, but it will be our region and the possibility of face-to-face interactions from time to time, that may also distinguish us to regional online students, in our lesser known programs if nothing else.

Moreover, there is much in oped pieces of late about the danger that lesser-known schools face as a result of the rush into online education by many leading institutions. I would extend this to concern for lesser known programs within well-known schools. But I also imagine that the regional affiliation of a small, lesser known school can be a strength, even in the institution's design of online programs. The trick to exploiting regional affiliation, I think, is to design online programs that are intended to be online most of the time, but that allow students to participate in person from time to time, perhaps with in-person attendance happening if and when the student wants (because the online component is synched with an on-campus offering), or requiring (or allowing) students to meet with the professor as a group during selected times of varying duration. This latter condition, of having predominantly online students attend in person at selected times, is built into some programs, but even in these cases, can these programs be further refined to be explicitly concerned with the regional online student? I think so.

There is lots to investigate before concluding that we can leverage high-quality regional brand (deserved, I hope!) to attract regional students to online programs. How good does the brand need to be and how much the desire to participate face-to-face from time to time, before tipping scales towards a predominantly online program at a regional institution, over a globally better known program? Even if institutions aspire to move beyond regional affiliations, they can still embrace those affiliations -- those affections that those in their region hold for them, and certainly I would expect that smaller schools should seek strength in region. An irony may be, however, that only schools with some (inter)national reputation will be able to exploit regional brand in the manor that I am considering -- schools without reputation at a larger geographic scale than their immediate region may have insufficient draw within their region; but they may nonetheless be able to have appeal for other niche populations -- this larger idea of consciously looking for your institution's niche populations was an interesting point by Arthur Kirk, President of Saint Leo's, a pioneer in online education. In the case of Saint Leo's, niche populations include students of the Catholic faith and students in the military, but my thoughts about region grew as a special case of his imperative to identify special populations.

Its probably no accident that "Rudy" came to mind earlier, a story about a young man in love with Notre Dame -- he grew up loving it! "Dreams make life tolerable" is another well-known "Rudy" quote, and we can design programs that allow the possibility of on-campus experiences, certainly a dream for some, in an otherwise online framework.


   

Local Learning Communities, Twitter, and Tobler's Law

Derek Bruff recently posted a summary of this week's discussion of Vanderbilt's local learning group focused on Anthony Robinson's COURSERA geography MOOC (https://www.coursera.org/course/maps). Derek's summary is posted here: https://my.vanderbilt.edu/vandymaps/2013/07/second-week-reflections-social-learning-in-a-mooc/

I wrote a response in Comments that is awaiting moderation, but it's a blog post in itself and I have few spare cycles, and so that comment will do double duty!

---

I have not participated in the local meetings this past two weeks, but I HAVE participated in the local group, thanks to #vandymaps (and thus, twitter), and into the future, maybe more blogging. Apropos your post Derek and this week's discussion, I've been reflecting on the social experience as well. Why have I spent 8 hours each of the past two Sundays on this class, when I am otherwise SLAMMED? This qualifier ("slammed") shouldn't be underestimated in what follows.

I am attempting the MOOC (#mapmooc), in part, because of an interest in geography and I like maps! But that's insufficient to explain 8 hour Sundays, I think -- in fact, I'm sure. 

In part, the attempt is also motivated by the idea that a Director for a digital learning institute should actually take and finish a MOOC, rather than only auditing MOOCs, though I don't think that explains the 8 hour Sundays either -- I could always complete "the next MOOC" rather than this one.

It strikes me that I mostly owe my stick-to-itiveness in this case to #vandymaps.  Initially, #vandymaps was a local learning community (I did skype in from Washington state for the pre-MOOC-kickoff meeting with all my local colleagues, huddled around a table -- very nice). A few of the #vandymaps people I would have counted as friends at the time of our pre-MOOC meeting (and still do, btw!! :-), but most of you I didn't know well, or not at all. Nonetheless, #vandymaps is still grounded in a local learning community (Todd Hughes's big, warm welcome to the group is as an important a qualifier as "being slammed" though with the opposite sentiment). But for me, this local community is one that (a) I am currently operating in virtually through twitter, and that (b) is being expanded through twitter to include others.

Its no accident that I used the #vandymaps hash tag to label the local group!

For me, I had enough social ties with people in #vandymaps initially that even virtual interactions were affective and therefore effective, so I see the "persistence" characteristic as directly causal in my case, but the locality characteristics (in my case) is causal of that. (This group is not shy about computing metaphors, so think Bayesian networks!!! :-)

Another exciting aspect of this MOOC experience is learning twitter. Apropos this, and the influence of "local" and "community", is my nascent but growing interactions on twitter. Here is a tweet that links some themes in these comments:

"28 Jul: Rocketing towards distinction in #mapmooc :-) The local group, #vandymaps, is such a great help -- a variant on Tobler's law!!!"

This has been re-tweeted and favorited by others -- VERY neat, but I only just learned of this (because I am only just learning twitter!). So, this doesn't explain my two previous 8 hour Sundays either, but these and other acknowledgements on Twitter (by non-local, as well as local persons) might be an influence on my future in this course.

But this connection between Tobler's Law and local learning communities is really interesting. Recall that Tobler's Law says that "Everything is related to everything else, but near things are more related than distant things." The Law doesn't ascribe a causal direction. Even though I am operating virtually with #vandymaps, they are "more relevant" to me than the world (at least so far as my #mapmooc behavior to date is concerned)-- its interesting, and will receive much more thought. It's also bringing me back to tried and true sentiments of "think global and act local" and "soldiers fight for their countries but die for their friends", and thinking through more variants on themes of locality, friendship, collegiality, obligation, reputation, etc. More later, I hope!

I see the importance to locality and Tobler's Law (directly or indirectly causal) to other activities too, most recently the 5 hours I spent on a big multi-institution proposal this weekend -- why? Because it was being headed by Vanderbilt, I'm part of this community (though not a co-PI) and "we" were all in this together, though all working virtually and asynchronously (but not through twitter :-) There are studies on locality and collaboration btw -- will dig those up for a scholarly article.

Thanks, Derek, for the post!

A Principle of Mapping Involving Precision and Accuracy

In the Coursera Map course that I am taking (https://www.coursera.org/course/maps), we were asked to place virtual pins on an interactive world map. Many students drilled down deeply to very specific locations in order to place their pins, some of these deliberately lying a bit so as not to reveal their exact location. Others placed their pins at coarse renderings on the map, without drilling down deeply at all, BUT NONETHELESS their pins appeared in rather arbitrary locations when the user drilled down and the map was viewed at a higher resolution. I wrote the following to the course discussion board, with a few revisions here to contextualize it. These maps represent digital tools that we may want to use in Vanderbilt MOOCs and we may want to improve on them.

----

I was one of those who drilled down to the building level in placing my pin, and was quite exact (an exact location in the city of Nashville). I saw one posted pin that was in the middle of Vanderbilt stadium and other pins looked haphazardly placed. Some of these may have been placed intentionally in (modestly) wrong places, as many have outlined on the boards. But some of these may have been placed in the Nashville region at a COARSE grained (low resolution) map by people who did not drill down deeply at all.

                                    snippet from ESRI-generated map on PennState course site

My placement was precisely where the faculty apartment is at McGill Hall (top right yellow pin), but another person's pin in the middle of Vanderbilt stadium, Dudley Field -- mistake?


These latter pin placements were accurate at the grain level in which they were placed, but (unintentionally) wrong at finer grained renderings of the map.

Shouldn't there be a precision principle of interactive maps like that provided in the class (analogous to rules of precision for floating point numbers I suppose)? Shouldn't there be some functionality for interactive maps, perhaps a research topic, that I can identify data with a region in a low resolution rendering of a map without being wrong at higher resolution when a user drills down. I can think of strategies to do this, but given the ambiguities of regions that this course has already made us aware of, there must surely be some fleshing out of these ideas.

I had the same question a couple of years ago when I started placing pictures on Google maps, most very precise down to a few square feet, but I also wanted to place some pictures on a larger region (e.g., a museum, a park, a city) with NO implication that these pictures were intended to be accurate at a finer grained level. (I wrote about this very fun exercise as well -- see http://aicourses.blogspot.com/2013/07/playing-with-pictures.html; beyond being fun though, its also a story that is illustrative of a point that is central in the Coursera-hosted PennState Map course -- that maps and geospatial tools/concepts can be central in telling stories!!!)

To the geographers out there -- is anyone doing research on implementing methods that enforce a precision principle of interactive maps?

BTW -- I think that a significance of this precision principle relates to issues of privacy. I'm guessing that those people who do NOT want to precisely identify their location, or identify their demographics (e.g., age, gender) with their own location, are probably sensitive to misidentifying someone else's residence with them (or with their demographics) -- at least I think that many would be sensitive to that.

But if placing a pin (or picture or ...) on a low resolution rendering of a map "accidentally" places the pin on someone else's residence at a higher resolution, then this misrepresenting of the makeup of that residence is exactly what could happen!

MapMooc, a Location Anonymity Index, and a World Connectness App

I have "audited" several MOOCs, hanging in there a bit before dropping out because I have just too much going on at work. I am now taking a geography MOOC (https://www.coursera.org/course/maps), and doing it with a  group of Vanderbilt colleagues, which I think will help complete this course. That I loved geography as a kid will help, and that I have a class project in mind will help too -- of creating a map of digital learning resources across the Vanderbilt University campus; I think I will end up using crowdsourcing to create this thematic map, with graduate student, staff, and faculty input.

My wife and I listened to a couple of intro lectures while packing up in Bellevue, WA before returning to Nashville. I spent most of this morning reading course material and reflecting on some of the course questions for Week 1, and contributing to the discussion board. Here are some of the questions.

1) What is scary about potentially losing control over your geospatial privacy?

2) What are some positive things that could come from openly sharing your personal location with others?

3) What about geospatial privacy has really changed over time? 200 years ago, would it have been possible to live in your current location without your friends and family knowing where you were most of the time?

Given that the number of discussion board posts is huge, it seems like a reasonable strategy is to join a "temporally local" cohort of discussants, so I contributed to another thread begun just this morning, then went to a couple of "suggested" threads based on keyword matches, and started my own thread on defining a "Location Anonymity Index" (LAI) in answer to question (3) above. As you might guess, as a computer scientist, I am very inclined towards developing metrics like the LAI for all kinds of things. Here it is with modest revision of my MOOC discussion board post.

3) What about geospatial privacy has really changed over time? 200 years ago, would it have been possible to live in your current location without your friends and family knowing where you were most of the time?

Much of my time is spent at work and home, and I think that the number of people who know precisely where I am for a large amount of time, is NOT that different from 200 years ago. In theory, there might be some people who I don't know (e.g., in government agencies), but who know or can access my location from social networks and cell phones, so the theoretical possibilities about location awareness are certainly different now than from 200 years ago. Also, a trip across town or across country would have left my precise location uncertain for long periods of time 200 years ago, and these intervals of location uncertainty are very much shorter today.

All considered, I wonder if it's reasonable to define a "location anonymity index" (LAI) that represents the expected error in OTHER PEOPLES' GUESSES of MY location (at particular times or across all times). A impractical way of computing my location anonymity index is to ask each person (on Earth!) where I am located and then measure the average difference between where I really am (known by an oracle) and where each person guessed I was. There are all kinds of complications to be worked out, like how "I" am identified (e.g., by name, by picture, by driver's license #, etc), and how "my location" is identified (e.g., at what grain size -- region, city, GPS,…? how difference between each guess and my actual location is measured?).

By this theoretically possible, but practically impossible measure, my location anonymity index (LAI) has probably not changed much in the past 200 years, because the vast majority of other people would be making wildly uninformed guesses, BUT ALSO because the much fewer "interested-in-Doug" persons (my wife, family, co-workers, friends) would make very accurate guesses on my location the vast majority of the time (Doug is home at …, Nashville, TN; Doug is at work at Vanderbilt), so even if we reduce the number of people who we ask to guess my location to those who are "interested", my location anonymity index probably doesn't change much over 200 years if measured in this way.

I think its interesting and important to consider the case where we change the computation of a location anonymity index (LAI) from simply asking people "cold" (uninformed), to the case where we allow each person to use technology available to them (e.g., Google, Bing, libraries, newspapers, supermarket rewards programs) to find important facts about me (e.g., home, work, blogs, …) BEFORE their guess on my location. This variation gets much closer to measuring the "worst case" situation where someone might be looking for me that wants to cause me harm.

In this latter variant, there is a huge difference in my LAI between now and 200 years ago, EVEN when the computation takes into account uncertainties that have to do with my "identity" (e.g., there is more than one Doug Fisher, in the world, in the US, in academia, in Nashville, etc … and this illustrates another point, which is that a LAI should take into account uncertainties in the extent to which my identity can be isolated).

Returning to questions of "geospatial privacy", which variants of LAI would be intended to measure under different assumptions, I'm not too scared about this -- I share lots of information on the Web that is intended to lessen my geospatial privacy, in particular to lessen the LAI as I conceptualize it. Nonetheless, I have been talking so far about the "average" LAI as a measure of "geospatial privacy", whereas I think most people worry about the "worst-case" -- what if someone with ill intent could locate me or who happened upon my location? Again, my latter variant of LAI with technology-informed guessers is intended to capture this worst case. In this regard, I may become much more worried about geolocation privacy as mobile technology continues to develop. For example, I recently took a photo of the Seelback Hotel lobby in Louisville, KY, and when I posted on Facebook I was prompted to tag (label) the face on an anonymous passersby who appeared in the photo!!!

Technology already exists that would allow the computer itself to do the tagging in that photo and other photos, and that possibility is unnerving, but I see some possible upsides for for giving up geospatial privacy. I am thinking about some of the positive messages conveyed by travelers like Rick Steves on world connectedness. But I also think that even some of the very unnerving possibilities like automated computer labeling of faces have some neat possibilities. I imagine jogging in Centennial  Park in Nashville TN 10 years from now, and having my wearable smart glasses tell me that it "thinks" I've seen that same passerby in Centennial Park, at the Vasa Museum in Stockholm, Sweden 5 years previously. I know this sounds scary too, but I find something in that level of connectedness to other people quite intriguing.

That said, I am a lot more worried about privacy along other dimensions, such as health care (probably for reasons that are specific to the US relative to much of the rest of the World) and finances/banking privacy. 

Reusing other Instructor's Assignments ... or not?

I am in the Educational Advances in Artificial Intelligence (EAAI-13), and we just concluded a session on educational repositories, particularly online repositories of homework assignments. Repositories of educational resources is a topic near and dear to my heart, but at least in the case of repositories of homework assignments, there appears to be no, little, or at best weak anecdotal evidence that assignments are being reused. At a minimum, don't we want repositories to be "instrumented,", like my (and everyone's) YouTube channel(s), so I can see downloads, likes, dislikes, and more sophisticated measures of usage that are specific to homework assignments?

Its hard to know if a homework assignment that has been posted in a educational repository is actually used by another instructor, unless an instructor who has used it, gets back to me and tells me so. There is some work in thinking about how to do this. But there is also low hanging fruit. First, we can measure downloads, but beyond this, as an educational community can take a small step towards a scholarly culture surrounding education materials by designing licenses specific to this kind of content.

For example, a license for usage of educational content could require that the material can be used by others (e.g., following any of the principles of creative commons licenses: http://creativecommons.org/), but additionally require that the user report back on the usage to the author (typically, the copyright holder), whether the use is as is, or derivative.

I think that this would be an incredible help to evaluating the extent and manner of use of educational material, going well beyond measuring downloads, and ultimately of evaluating the utility of educational materials to the educational community.

Let's ask people about their use, through a license that requires report back (and nothing else), rather than simply depending of the ability of inference by machine methods.

A MOOC is NOT a Textbook

When I first started using video lectures from Jennifer Widom's Database MOOC and Andrew Ng's Machine Learning MOOC in Spring 2012 for my courses at Vanderbilt, I was worried about what people might think.  I did at least two things in response. One, I started creating my own online content that others could use; giving back made me feel much better about using the content of others. Two, I cast the MOOCs as "multimedia textbooks", just the next step in a natural evolution (e.g., https://my.vanderbilt.edu/cs390fall2012/). At the time I really did think of a MOOC as something of a "textbook." But the fact is that a MOOC is NOT a textbook, multimedia or otherwise, perhaps an obvious truth that I only reconsidered recently as I prepared for a panel presentation (https://my.vanderbilt.edu/douglasfisher/files/2013/06/DougFisherMOOCsOnCampusPresentation.pdf) at AAUP 2013 (http://www.aaupnet.org/events-a-conferences/annual-meeting/aaup-2013/program).

Others have also advocated MOOCs as multimedia textbooks for reasons of promoting acceptance by skeptics of online education, and while MOOCs may contain material that will become part of such textbooks, MOOCs aren't textbooks per se. I think that MOOC-offering organizations acknowledge this, perhaps implicitly, in that they are entering into agreements with other resource providers, like textbook publishers, so as to augment the MOOC experience (e.g., http://connection.sagepub.com/blog/2013/05/08/sage-coursera-partnership/).

Beyond stating the obvious, I am thinking aloud here about how community-developed, online, multimedia textbooks might arise.

The authors of even a mediocre textbook aspire to be somewhat comprehensive in their coverage of a field, almost always including more material than any one instructor would cover in any one course, and as importantly, a good textbook will synthesize across that material. A good textbook supports large-scale customization, as different instructors at different universities create their own course variations, each anchored by the same textbook. By and large, MOOCs aren't (yet!) designed with customization in mind. Like other courses, a MOOC is a single trajectory through a single selected subset of a field's content. Some MOOCs (and online courses before the 'MO' movement) have multiple tracks, and this is a BIG innovation relative to what's typically done with an on-campus course. Each track is a trajectory, and the differing, albeit overlapping tracks of a MOOC represent a step towards supporting choice and customization, but only a step.

While supporting limited choice, a MOOC still covers only a small part of the material that a typical textbook might cover. A textbook for artificial intelligence (e.g., http://artint.info/html/ArtInt.html), for example, will often cover natural language processing (NLP) by computer, and different computational approaches in this area, but its rarely material that I cover. This is another example of a textbook providing the freedom to follow one of many different tracks, some that include NLP, and some not.

While courses, online or otherwise, support limited choice of trajectories through material, online repositories of educational content, such a TeachingTree (http://www.teachingtree.co/) or even YouTube (http://www.youtube.com/t/education) on a much larger scale, support "unlimited" choice and customization -- these latter resources are UNDER-constrained in terms of choice, just as MOOCs are OVER-constrained, at least if we want to start thinking of such resources as "textbooks." A good textbook, one might argue, is "optimally" constrained, providing enough choice for instructors to follow different trajectories, but also providing enough structure, constraints, and guidance so that the choice is not overwhelming!

Repositories of online educational material are growing, providing ever increasing choice for educators and learners, but as yet, these repositories provide insufficient constraints and guidance on how choice can be effectively navigated for course customization. There is interest and work in using crowd-sourcing, through mechanisms such as Wikimedia, to build structure on top of these resources (e.g., http://www.aaai.org/ocs/index.php/SSS/SSS12/paper/download/4343/4693/‎), but I think that it will take dedicated authors, working individually or in small groups, to step up to the plate and synthesize across these online resources if we are to see good-quality multimedia textbooks -- resources that effectively tradeoff choice and structure -- not too under or over constrained.

So, perhaps I overreacted in saying that a MOOC is not a textbook. Rather, along this continuum between over and under constraint for purposes of supporting customization and diversity of content and teaching style, a MOOC, or any other course for that matter, lies on the over-constrained end, something of an impoverished textbook at best -- though it may in fact be an excellent course!