Establishing conventions for citing educational materials

I am giving a lightening talk at SIGCSE-2017 on establishing conventions for citing educational materials. SIGCSE is the big computer science education conference. This lightning talk summarizes motivations for and history of citation generally, argues that conventions be adapted to educational materials, and forwards a start at a sample convention used in a survey of coursework (in computing and sustainability).  See links to support discussion at the bottom of this post. What follows immediately below is modestly revised from the abstract submitted for SIGCSE review.

Course designs (manifest as syllabi), exercises, assignments, and project specifications are all creative acts of design by the instructors who create them. But there are no conventions for citing such works, as one would cite a research paper, patent, or certain other results. The core issue behind this suggestion is that we want to track and assess the influence of an educational artifact. This lack of convention, or even a “citation mentality” or expectation in the educational realm, is even true in established educational repositories.

There are several (potentially) important reasons for establishing conventions for citing educational content, such as courses and various assessments used in courses, and thus tracking their diffusion and influence.

(1) Teaching faculty design materials. Design of courseware is at least as important to teaching as is delivering lectures and grading assessments. If citation conventions were established, then perhaps the degree of adoption by others could be part of a teaching faculty member’s professional evaluation.

(2) Citation counts would also useful for flagging and vetting educational materials that are candidates for reuse and adaption by others

(3) Research grants often include education and outreach plans, and again, being able to track the influence of materials under such plans would be valuable material for funding agencies.

(4) Establishing citation conventions online may raise the consciousness of the importance of citing any and all creative works that an instructor uses, particularly when we want our students to do so.

(5) Conferences like SIGCSE are increasingly competitive, and publishing a paper that reports of a particular piece of educational material, thereby enabling “conventional citation” may be difficult.

The real goal of the lightening talk and post is to gauge interest and solicit collaborators, perhaps with a SIGCSE Special Project proposal (http://sigcse.org/sigcse/programs/special/apply). A Google Drive folder and primary discussion document has also been established to continue this conversation and planning.

Fisher, D. H. (2017). Establishing Conventions for Citing Educational Materials. In Proceedings of the SIGCSE-2017 Conference (https://sigcse2017.sigcse.org), Seattle, WA, March 8-11.

On the true significance of self-paced, online courses: the anytime MOOC

Last year I wrote about my experience in a self-paced online course in Artificial Intelligence offered on Udacity (http://cloudandcampus.blogspot.com/2013/05/better-preparation-through-moocs.html). The advantages of a self-paced course are most apparently in the flexibility it affords the student — a student can work their way through the course on their own schedule, taking educational detours along the way, as they see fit. Coursera is now offering self-paced (aka on-demand) courses as well ( https://coursera.desk.com/customer/portal/articles/1639240-about-on-demand).

The individual flexibility associated with technology-enabled self-paced courses is common in other areas touched by the digital revolution. In her introductory post (https://my.vanderbilt.edu/vidl/2014/08/sandra-arch-vidl-graduate-fellow-introductory-post/), VIDL Graduate Fellow Sandra Arch points to the same advantages of online-enabled flexibility in other working environments, but she also notes the broader, potentially negative cultural and societal impacts of individual flexibility:

"While these newer forms of work offer greater independence and autonomy to workers, enabling them to work from anywhere with an internet connection, they also reflect a larger trend of social isolation in the contemporary era, with its declining rates of membership in social, religious, and voluntary organizations." (Sandra Arch)

Slogging through a self-paced course all by yourself takes discipline, and completion among the isolated can be difficult (e.g., http://www.westga.edu/~distance/ojdla/winter84/nash84.htm).  But importantly, the freedoms of self-paced courses are NOT limited to individual students, but self-pacing can also be exploited by instructors.

Shortly after my post on my self-paced experience, I wrote about the anticipated ease of adapting self-paced online courses (which aren't really MOOCs in the conventional sense) to the campus experience (http://cloudandcampus.blogspot.com/2013/06/syncing-moocs-and-campuses.html). Herein lies the real positive significance, I think, of self-paced courses — individual instructors can adopt the “self” paced course so that it keeps pace with their on-campus schedule. Thus, the “self” in self-pacing is generalized to my on-campus student cohort, or synchronization with any local learning cohort for that matter (e.g., my after-school club, my retirement community).

How would adopting a self-paced OPEN Online course for my campus course differ from simply implementing what is called a closed instance, also known as a SPOC (a small, private, online course). In a SPOC, only the campus students see the content — the MOOC is essentially ported to a learning management platform that can only be accessed by the campus students. Wouldn’t that allow the instructor to move through the material at a pace that best suited the campus course? Yes, it would, but what would be missing is participation of students from outside the campus to take the course coincident with the campus students.

Why might I want non-Vanderbilt students to take a course side-by-side with campus students? For several reasons. First, because such students will bring international perspectives to my campus course. For example, two criteria used to accredit computing programs (http://www.abet.org/cac-criteria-2014-2015/) are "(e) An understanding of professional, ethical, legal, security and social issues and responsibilities" and "(g) An ability to analyze the local and global impact of computing on individuals, organizations, and society."  Might campus students taking a course with international students better appreciate these criteria, and be better motivated to excel at them? Its an open question, but worth exploring.

Second, we find that many students of take our MOOCs are professionals. Such students can be valuable mentors for our campus students (e.g., helping them to excel along with of the technical criteria of ABET, for example). This can be particularly so for underrepresented groups in computing (or other fields). Even if potential mentors from some groups are few on a campus, drawing from a global population can bring empathetic and enthusiastic mentors into a course, perhaps helped through collaboration with professional societies (e.g., http://www.nsbe.org/home.aspx; http://societyofwomenengineers.swe.org).

Third, in an open environment, “alums” and advanced students of the self-paced online course (or a corresponding MOOC) can continue to give advice to the current crop of students taking the course, including those who are members of the local learning cohort that an on-campus instructor can organize. In an open environment, it is indeed possible for an on-campus instructor to find some of the world’s best "teaching assistants" for the benefit of the campus population (http://cloudandcampus.blogspot.com/2013/05/finding-best-teaching-assistant-in-world.html).

As an educator, the potential of self-paced courses that I get most excited about is that I can “wrap” my campus course around the self paced course (http://jolt.merlot.org/vol9no2/bruff_0613.htm), thereby benefiting from those outside my course, but also in the self-paced course, I can actually invite outsiders to join my campus cohort to take the course synchronizing with my students — if enough decide to take the course with my campus students then I am essentially the on-the-ground instructor for an anytime MOOC. An anytime MOOC is the MOOC I most want to run. Perhaps I’ll have my chance in Spring 2015 when I next teach Introduction to Databases (https://www.coursera.org/course/db)!

*Cross-posted on the Vanderbilt Institute for Digital Learning blog

Deliberative Design of Online Content by Instructional Communities

An article was posted recently on Inside Higher Ed, speculating about behind-the-scenes maneuvering related to Coursera's  recently announced "specializations", which are MOOC sequences of which Vanderbilt is a pioneer. I posted a couple of comments in response to the article, one in a response to a flippant comment by someone that one company had possibly ripped off another, a view that doesn't recognize the naturalness with which things are unfolding (i.e., any reasonably smart person who is thinking about the issues a non-trivial amount of time will hit upon ideas and implementations, like MOOC sequences):

The idea of sequencing MOOCs undoubtedly began with students themselves, who were piecing together MOOCs that were developed independently at different institutions, but which were nonetheless elaborating well-trodden curricular paths inherited from traditional curricula. The first deliberatively-designed MOOC sequence that crossed institutional borders (that I am aware of) is a partnership between Vanderbilt and Maryland announced Sept 2013, and which has been since incorporated in the Coursera specializations. The idea of giving certifications for a sequence is a natural next step, and pretty soon students will be asking for "transfer" credit ("but I took an on-campus version of MOOC X in the sequence, so why do I have to take it again?"). I think all of this is pretty predictable. What I am so excited about in this space is not the very predictable elements of it, but that we have instructors crossing institutional boundaries to do teaching, not simply research. Trans-intitutional teaching communities are substantially new in my world, and quite welcome (e.g., see last paragraph of http://chronicle.com/blogs/profhacker/warming-up-to-moocs/44022).

"Chuck" responded to me, asking about how instructors who were not Coursera (or edX partners) could participate in trans-institutional teaching. And here I recounted the experience that first got me excited about instructional communities, and very particularly, the idea that instructors can come together and design and produce individual courses together, putting up individual lectures that others can and will use:

Good question, Chuck. Even though I am at a Coursera partner institution (and I have also used the edX platform for my on-campus course in Artificial Intelligence), I am probably not going to do my own MOOC or MOOC sequence any time soon. That said, I do put my own content on Youtube and students who are taking other people's MOOCs (or other on-campus courses) come to my Youtube channel for remediation (and I can verify the former by going to the MOOC's discussion forum). In some sense I am a "silent partner" in an instructional partnership with the MOOC instructors.

But I could go farther too, and I could go through a MOOC and identify where I think that a subpopulation of the students will have problems (or wait for those problems to arise in real time during the MOOC, but I prefer to anticipate), and produce and post content that is deliberatively designed to remediate for that anticipated subpopulation, and additionally, I could produce advanced material that the MOOC would prepare some students to take. In both cases, I could post links to the remedial and advanced material. In this case I would be a less silent partner.

This idea, btw, extends to students too. For example, Vanderbilt's Center for Teaching is running the BOLD program , in which graduate students are mentored in their creation of online content to supplement on-campus courses. Graduate students and mentors could just as easily target MOOCs for such augmentation too, and in doing so, getting a worldwide audience, as well as (we hope) the attention of world-renowned teachers (i.e., the instructors of the MOOC)!!! I want to do this!

My response to "Chuck" continues:

But this possibility goes well beyond MOOCs; I use other people's Youtube content for my courses, and other instructors can certainly point to mine. But its currently very opportunistic. What if this material was designed in collaboration among a small group of like-minded profs (in say Artificial Intelligence), with the intent that they would co-teach each of their individual AI courses.

I don't see MOOCs as a required entry into instructional communities, though its exciting and it allows collaborations that are relatively loosely coupled. Collaborating across institutions by sharing deliberatively coordinated content on Youtube (for example), would require greater coupling between instructors, but its still an exciting possibility.

Our own students (albeit a very few of them) have been producing and sharing excellent instructional content online for years. Instructors are late to the party, but its never too late.

I had an opportunity to begin some of this deliberative design, running some of my AI course on the edX platform, but even more interestedly, using lecture content from UC Berkeley's AI MOOC (the same MOOC that had students visit my Youtube channel for remediation about 18 months ago), my own lecture content, and lecture content from elsewhere (e.g., University of British Columbia). This mixing and matching of content was easy to do on the edX platform, though I didn't have access to the platforms full functionality. It was an interesting experience on my way to more fully explore course customization possibilities -- exciting times for those of us who are teachers, and who so appreciate the idea of community too!

MOOCs, and faculty members as "lead" learners

I've been asked recently, once during an interview with a MOOC provider and once at a meeting with a giving foundation, about the characteristics of colleges and universities that might take to adapting MOOCs for use in their on-campus courses. There are varied reasons that institutions might use MOOCs. Most discussed are top-down, administration-imposed motivations for MOOC use, at junior colleges, larger public universities, and even at research 1 universities (RU/VH). But my response in both recent discussions was that small liberal arts colleges were the most interesting case to me. According to the Carnegie Classification (http://en.wikipedia.org/wiki/Carnegie_Classification_of_Institutions_of_Higher_Education), these might be small (S4) residential (R) colleges, somewhat rural so that faculty live nearby too.
I am guessing that faculty at small, residential schools might enthusiastically embrace the opportunity to use MOOCs, for reasons other than workload reduction; I think that their students would embrace it as well, at least in part because their professors do.

I've only witnessed the "behind-the-scenes" of one small residential college to any significant extent,  but as I describe to other people the behind-the-scenes that I experienced, most recently in NYC at the foundation meeting, some give a knowing smile and affirm that mine is not a unique experience. These are schools in which the CS program, for example, might have 2-4 faculty members, some shared with Mathematics or other programs, with a total of AT MOST 20 majors spread across all fours years. Faculty and students are a tight-knit community, having dinners and participating in other events together. The number and variety of course offerings is limited to what a larger program might view as the absolute core, with faculty members focused on teaching those. A "boutique" course like Artificial Intelligence (ha!) might be offered on occasion. There are also reading groups and other informal learning with faculty and students that are part of the community practice. This is one setting where faculty members actively learn with students, and while the possibility of an "exotic" course like Machine Learning is improbable, its something that faculty and students would both welcome.

This is the setting where MOOCs may be a godsend for all involved, welcomed with open arms, so that students and faculty can learn from advanced courses like Machine Learning, which would not be offered without the MOOC, or in olden times, without a visiting faculty member to campus to teach it.

To me, the defining characteristic is that faculty are unafraid to learn side by side with students, and that students respect and enjoy a faculty member in the learning role, at least with respect to some of the advanced coursework that they wouldn't otherwise enjoy. I am guessing that faculty and students alike would both want the faculty members as "lead" or "elder" learners within the cohort. Such a setting can, of course, exist at larger colleges and universities too, and its where I see the MOST gratifying experiences with MOOCs in the classroom.

Crowd-sourced Curricula

For the past couple of years, in Spring 2012 and Spring 2013, the end of semester project in my database course was the design of a database to support an educational social network (https://my.vanderbilt.edu/cs265/course-customization-and-learning-social-networks/), where online educational materials -- video and otherwise -- can reside, be endorsed by compatriots in social networks, be decomposed into micro structures (e.g., in the way that http://www.teachingtree.co/ , and now YouTube, allows indexing of particular segments in videos) and composed into macro structures, with individual topic videos formed into "courses" and online courses (e.g., MOOCs) being formed into curricula; users in the social network can endorse (or renounce) constructs at all of these levels -- micro, macro, and a lot of in between -- with opinions at one level influencing constructs at lower and higher levels.

The project was inspired by my work on the Chancellor's Social Media and Digital Technology Committee (http://news.vanderbilt.edu/2012/02/committee-social-media/), but particularly the vision of the unbundled university (http://www.educause.edu/annual-conference/2011/chronicle-tech-trends-challenges-future-unbundled-university; http://chronicle.com/blogs/wiredcampus/a-chronicle-educause-panel-challenges-for-the-unbundled-university/33812) -- what is unbundled can and will be reassembled, and according to the early visionaries, reassembled in a wealth of ways.

The crowd-sourced micro structures are interesting, and I'll address them at some point, but I've been enamored of late with crowd-sourced macro structures, particularly crowd-sourced curricula. This slide from an ITHAKA S+R talk (http://vimeo.com/53361649)  illustrates that even in October 2012 (not even a year ago -- seems like ages!) someone could piece together what amounted to a CS bachelor's from free, online classes.

This slide is the first in a four-slide sequence showing the possibilities of a CS major online (https://my.vanderbilt.edu/douglasfisher/files/2013/06/DougFisherMOOCsOnCampusPresentation.pdf ).

In the educational social network context, individual and alternative paths through the course repository (http://blogs.the-american-interest.com/wrm/2013/09/12/create-your-own-college-mooc-style/) get endorsed and rejected, until some emerge as consensus favorites. Just before drafting this post, I searched on Google for "crowdsourced curricula" (no quotes) and sure enough, found this: "Crowdsourcing Curricular Design (Helps & Grant, submitted) at http://micsymposium.org/mics_2013_Proceedings/submissions/mics20130_submission_23.pdf -- a nice paper sketching out a similar vision, though with some interesting differences, some of which I really like, perhaps most notably that the crowd-sourcing be by educational professionals, better insuring that the macro structures that emerge aren't the "easiest", but the most pedagogically sound. The paper also has some interesting background that I didn't know of previously.

Attention on macro-level, curricular constructs is receiving increased attention because of a new partnership between educators at Vanderbilt University and University of Maryland (http://news.vanderbilt.edu/2013/09/vu-maryland-mooc/). In particular, Doug Schmidt and Jules White of Vanderbilt and Adam Porter of the University of Maryland are organizing the first transinstitutional MOOC sequence -- not to be confused with a jointly offered MOOC (singular), but a sequence of two, with Porter's Apps development MOOC serving as a soft prerequisite to Schmidt and White's Pattern-Oriented Software Architecture MOOC. This sequence is a curriculum-level construct, an important nascent step towards crowdsourced curricula.

Don't Forget About Regional Brand!

Lots of schools are developing online programs to increase revenue, notably through increased enrollments, or perhaps to protect against falling enrollments in the future. While there are projections of modest increases in enrollments in institutions of higher education in the near future (http://nces.ed.gov/programs/projections/projections2021/sec5a.asp ), they "do not take into account such factors as the cost of a college education, the economic value of an education, and the impact of distance learning due to technological changes". These are such important factors that I think their exclusion from analysis makes projections rife with uncertainty, because the rules are changing.

Usually, when a university offers an online course (e.g., a MOOC) or a much more comprehensive online program (e.g., Vanderbilt's School of Nursing MS programs: http://www.superscholar.org/schools/online/vanderbilt-university-school-of-nursing/) the emphasis is on reaching students across the World, and because travel and part-time residency can be expensive in terms of time and money, allowing students to be solely online, never having to travel to Vanderbilt (or whatever university we happen to be talking about) is important, particularly for older adults, in contrast to 18-22 year olds, who want to continue their education, but can't take time away from work and/or family.

In addition to revenue, online programs can spread and reinforce a university's brand worldwide, entering the university's name into interpersonal discussions around the World, perhaps attracting some students to attend the school in person, in addition to attracting others online.

In all this attention to reaching the World though, we shouldn't forget about the importance of regional brand and other attractions of our institutions to OUR region (I'm reminded of the movie "Rudy", the scene where Coach Dan Devine of Notre Dame tells his football players before a home game with Georgia Tech that "No one, and I mean no one, comes into OUR house and pushes us around." :-).

I wish I had the data to answer the question "how many older adults who are going through online programs would like to attend formal classes in person from time to time?" I wonder about this question because I wonder whether we are overlooking an important population in all this focus on online education -- older adults living within 120 miles (or so) of our university. These are people who may want to take a degree program ONLINE MOST OF THE TIME (because of work and family), but who are close enough that they can and do aspire to attend in person at least from time to time, to meet the professor and their student colleagues! Indeed, some may have grown up wanting to attend our school; they may want their children to attend our school.

I also wonder whether, at some point, as more and more institutions develop online programs that we won't hit saturation points. Our rankings as a university and rankings of our programs will distinguish us among other online programs to be sure, but it will be our region and the possibility of face-to-face interactions from time to time, that may also distinguish us to regional online students, in our lesser known programs if nothing else.

Moreover, there is much in oped pieces of late about the danger that lesser-known schools face as a result of the rush into online education by many leading institutions. I would extend this to concern for lesser known programs within well-known schools. But I also imagine that the regional affiliation of a small, lesser known school can be a strength, even in the institution's design of online programs. The trick to exploiting regional affiliation, I think, is to design online programs that are intended to be online most of the time, but that allow students to participate in person from time to time, perhaps with in-person attendance happening if and when the student wants (because the online component is synched with an on-campus offering), or requiring (or allowing) students to meet with the professor as a group during selected times of varying duration. This latter condition, of having predominantly online students attend in person at selected times, is built into some programs, but even in these cases, can these programs be further refined to be explicitly concerned with the regional online student? I think so.

There is lots to investigate before concluding that we can leverage high-quality regional brand (deserved, I hope!) to attract regional students to online programs. How good does the brand need to be and how much the desire to participate face-to-face from time to time, before tipping scales towards a predominantly online program at a regional institution, over a globally better known program? Even if institutions aspire to move beyond regional affiliations, they can still embrace those affiliations -- those affections that those in their region hold for them, and certainly I would expect that smaller schools should seek strength in region. An irony may be, however, that only schools with some (inter)national reputation will be able to exploit regional brand in the manor that I am considering -- schools without reputation at a larger geographic scale than their immediate region may have insufficient draw within their region; but they may nonetheless be able to have appeal for other niche populations -- this larger idea of consciously looking for your institution's niche populations was an interesting point by Arthur Kirk, President of Saint Leo's, a pioneer in online education. In the case of Saint Leo's, niche populations include students of the Catholic faith and students in the military, but my thoughts about region grew as a special case of his imperative to identify special populations.

Its probably no accident that "Rudy" came to mind earlier, a story about a young man in love with Notre Dame -- he grew up loving it! "Dreams make life tolerable" is another well-known "Rudy" quote, and we can design programs that allow the possibility of on-campus experiences, certainly a dream for some, in an otherwise online framework.


   

Local Learning Communities, Twitter, and Tobler's Law

Derek Bruff recently posted a summary of this week's discussion of Vanderbilt's local learning group focused on Anthony Robinson's COURSERA geography MOOC (https://www.coursera.org/course/maps). Derek's summary is posted here: https://my.vanderbilt.edu/vandymaps/2013/07/second-week-reflections-social-learning-in-a-mooc/

I wrote a response in Comments that is awaiting moderation, but it's a blog post in itself and I have few spare cycles, and so that comment will do double duty!

---

I have not participated in the local meetings this past two weeks, but I HAVE participated in the local group, thanks to #vandymaps (and thus, twitter), and into the future, maybe more blogging. Apropos your post Derek and this week's discussion, I've been reflecting on the social experience as well. Why have I spent 8 hours each of the past two Sundays on this class, when I am otherwise SLAMMED? This qualifier ("slammed") shouldn't be underestimated in what follows.

I am attempting the MOOC (#mapmooc), in part, because of an interest in geography and I like maps! But that's insufficient to explain 8 hour Sundays, I think -- in fact, I'm sure. 

In part, the attempt is also motivated by the idea that a Director for a digital learning institute should actually take and finish a MOOC, rather than only auditing MOOCs, though I don't think that explains the 8 hour Sundays either -- I could always complete "the next MOOC" rather than this one.

It strikes me that I mostly owe my stick-to-itiveness in this case to #vandymaps.  Initially, #vandymaps was a local learning community (I did skype in from Washington state for the pre-MOOC-kickoff meeting with all my local colleagues, huddled around a table -- very nice). A few of the #vandymaps people I would have counted as friends at the time of our pre-MOOC meeting (and still do, btw!! :-), but most of you I didn't know well, or not at all. Nonetheless, #vandymaps is still grounded in a local learning community (Todd Hughes's big, warm welcome to the group is as an important a qualifier as "being slammed" though with the opposite sentiment). But for me, this local community is one that (a) I am currently operating in virtually through twitter, and that (b) is being expanded through twitter to include others.

Its no accident that I used the #vandymaps hash tag to label the local group!

For me, I had enough social ties with people in #vandymaps initially that even virtual interactions were affective and therefore effective, so I see the "persistence" characteristic as directly causal in my case, but the locality characteristics (in my case) is causal of that. (This group is not shy about computing metaphors, so think Bayesian networks!!! :-)

Another exciting aspect of this MOOC experience is learning twitter. Apropos this, and the influence of "local" and "community", is my nascent but growing interactions on twitter. Here is a tweet that links some themes in these comments:

"28 Jul: Rocketing towards distinction in #mapmooc :-) The local group, #vandymaps, is such a great help -- a variant on Tobler's law!!!"

This has been re-tweeted and favorited by others -- VERY neat, but I only just learned of this (because I am only just learning twitter!). So, this doesn't explain my two previous 8 hour Sundays either, but these and other acknowledgements on Twitter (by non-local, as well as local persons) might be an influence on my future in this course.

But this connection between Tobler's Law and local learning communities is really interesting. Recall that Tobler's Law says that "Everything is related to everything else, but near things are more related than distant things." The Law doesn't ascribe a causal direction. Even though I am operating virtually with #vandymaps, they are "more relevant" to me than the world (at least so far as my #mapmooc behavior to date is concerned)-- its interesting, and will receive much more thought. It's also bringing me back to tried and true sentiments of "think global and act local" and "soldiers fight for their countries but die for their friends", and thinking through more variants on themes of locality, friendship, collegiality, obligation, reputation, etc. More later, I hope!

I see the importance to locality and Tobler's Law (directly or indirectly causal) to other activities too, most recently the 5 hours I spent on a big multi-institution proposal this weekend -- why? Because it was being headed by Vanderbilt, I'm part of this community (though not a co-PI) and "we" were all in this together, though all working virtually and asynchronously (but not through twitter :-) There are studies on locality and collaboration btw -- will dig those up for a scholarly article.

Thanks, Derek, for the post!