Consistency vs Variety in Online Assessments: is the balance important?

I received an email last week from a college professor who had a good question. She is designing her second online course and was wondering what my opinion was on the types of assignments that work best online. Specifically, she wanted my opinion about whether predictability and variety were important facets in good assessment schemes for online courses.

Yes – I vote for predictability and variety; however, there is more than one way to skin the cat. Let me give you a few examples:

Scenario 1:

Assignment 1: Due Sunday of Week 3
Assignment 2: Due Sunday of Week 6
Assignment 3: Due Sunday of Week 9
Assignment 4: Due Sunday of Week 12
Online discussion: weekly
Final Exam: Week 14

In Scenario 1, the student can clearly see the pattern and predict the workflow from week-to-week as the course progresses through a semester. How about variability? Well, there are two types of weeks in this course: i) weeks with two deliverables (i.e. assignment and discussion) and ii) weeks with only one deliverable.

Scenario 2:

Assignment 1: Due Sunday of Week 2
Assignment 2: Due Sunday of Week 3
Assignment 3: Due Sunday of Week 4
Assignment 4: Due Sunday of Week 5
Assignment 5: Due Sunday of Week 6
Assignment 6: Due Sunday of Week 7
Assignment 7: Due Sunday of Week 8
Assignment 8: Due Sunday of Week 9
Assignment 9: Due Sunday of Week 10
Assignment 10: Due Sunday of Week 11
Assignment 11: Due Sunday of Week 12
Online discussion: weekly

In Scenario 2, like Scenario 1, students can clearly see the pattern and predict the workflow from week-to-week throughout the semester. For variability, there isn’t any. Every week is the same.

Scenario 3:

Assignment 1: Due Sunday Week 6
Term Paper proposal: Due Sunday Week 7
Online discussion: Weeks 4-8
Final Term Paper: Due Sunday of Week 12

In Scenario 3, it’s more difficult for students to see a pattern and they can have difficulty predicting the workflow as the course progresses. For variability, it’s pretty high. Some weeks have one deliverable, others have two deliverables and still other weeks have no deliverables whatsoever.

Scenario 4:

Mid-term Exam: Week 7
Final Exam: Week 14

It’s easy to see that Scenario 4 is highly predictable by students but the overall variability in the course is low.

So, given these types of scenarios, which is the ‘best’? In an online course, good course designs must strive to create student engagement. Over time I have learned that the best way to engage students is to provide them two things: 1) a balanced yet consistent level of predictability and variability in the coursework, 2) a balanced level of student-to-student and student-to-instructor interaction. I’ve dealt with the interaction piece before (here or here). So let me just stick to the predictability and variability component for this blog post.

Now if you re-examine the four scenarios above you might come to the conclusion that Scenario 1 is the most balanced as far as variety and predictability are concerned. Can it be improved? Consider Scenario 5:

Scenario 5:

Discussion 1: Due Sunday of Week 2
Assignment 1: Due Sunday of Week 3
Discussion 2: Due Sunday of Week 5
Assignment 2: Due Sunday of Week 6
Mid-term Quiz: Due Sunday of Week 7
Discussion 3: Due Sunday of Week 8
Assignment 3: Due Sunday of Week 9
Discussion 4: Due Sunday of Week 11
Assignment 4: Due Sunday of Week 12
Final Exam: Week 14

What do you think about Scenario 5 for an online course? Is it better or worse than Scenario 1? What do you think about the entire idea of predictability and variability in assessment schemes? Please leave your comment below.

Add Post To:

SchoolTube: Learning Without the Risk of Explicit Content

There's a dilemma in the K-12 Educational Realm: to block YouTube or not to block YouTube. Parents have the same dilemma at home too. There's a lot of explicit and suggestive videos on YouTube that most teachers/parents wouldn't allow children to be exposed to. The opposite is true as well: there's a lot of fantastic content on YouTube that can help kids in the learning journey. Each school and family makes up their own rules on how to navigate this problem. However, the father and son team of Carl and Andrew Arizpe are giving us all an alternative. It's called SchoolTube. It has all the good parts of YouTube without the risks of inappropriate content. The way that SchoolTube creates this safe environment is to empower teachers to moderate the videos posted to SchoolTube by their very own students. NEAT! Like YouTube, no account is necessary to view videos and the collection of interesting clips tops over 400,000 videos at the moment. Here's one example below that provides a fun and useful introduction to the Periodic Table of Elements. Chemistry geeks will love this.
The main page of SchoolTube also contains a "Video of the Day" that is judged worthy by users. Pretty cool. Do you have any experience with SchoolTube? Are there any other advantages or disadvantages? If so, let us know by leaving a comment below.

Add Post To:

Implications for Online Learning - 2012 Canada Copyright Reform

Hot off the presses! Contact North, Ontario's Distance Education and Training Network, has published a very useful summary of the profound copyright changes in Canada that have occurred in 2012 (to date!) Entitled: The Perfect Storm - Canadian Copyright Law 2012 (Making Sense of the Dramatic Changes and the Far-Reaching Implications for Online Learning).

It's obvious by the sub-title that this document discusses the recent changes in the context of distance education and online learning. Refreshing! It is important to note that unlike my last post on the subject of Copyright Reform in Canadathat this document by Contact North also explains five Supreme Court of Canada decisions that took place in July 2002 that contribute towards an expanded definition of "Fair Dealing", technology neutrality and a strong endorsement of end user rights. Have a look at this succinct document. If you are a teacher, instructional designer or distance learning administrator then it will definitely pique your interest.

Add Post To:

Don't Blink Eh?: Copyright Law is Changing in Canada - What It Means to Educators

Bill C-11 passed a House of Commons Vote this week and readings in the Canadian Senate have already begun. (Here's the full text of the bill in full-fledged legalese). So the writing on the wall is that Canadian Copyright Law is about to change. There are a lot of little goodies in the law that may help the average person. Some things that have been going on illegally for years will now be legal. Simple stuff like giving the ability to Canadians to:

• record their favourite TV shows for later viewing (Yeah - TiVo'ing the late game on Hockey Night in Canada so you can watch it the next morning while eating a bowl of Shreddies is now legal!)


• transfer music from your collection of Compact Discs or Vinyl LPs to a digital device (Yeah - ripping your RUSH Records so you can listen to Geddy Lee on your iPhone while eating a Beaver Tail is now legal!)

Canada is truly great, eh! But what's in it for educators? The practical answer to that question is still pending. People need more time to interpret Bill C-11 in the context of classroom courses, class websites, class discussion forums, and distribution of course materials for distance education. Some interpretations are already starting to become available and in the coming months more information will come to light. If you are a teacher, instructor, professor, or instructional designer then you will want to follow this for the next few months so that you can decide how it impacts your practice. So that you can start getting info, please find below some of the early interpretations and commentary on Bill C-11 from an education perspective.

• UBC’s summary of Bill C-11
• NDP Member Rathika Sitsabaiesan’s opinion on Bill C-11
• Bill C-11: A Guide for Academic Instructors by the British Columbia Library Association Information Policy Committee

If you come across any other great links on the subject, please let me know by leaving a comment below. I will be sure to amend this blog post with the resources you provide. Thanks!

Add Post To:

Careful: Test Cheaters are SMART!

A colleague of mine directed my attention to a recent article in the Chronicle of Higher Education that describes a high-tech method where students can cheat on some multiple-choice exams in online courses. It’s a great read really.

Overall, I think that the way the students exploited the weakness in the course’s testing method is truly ingenious. They deserve a bit of credit for detecting, and then so intelligently exploiting, the flaws in this course. (Note: not enough credit to absolved them of their academic dishonesty though!) They honed in on two-specific facts: 1) that students could take the tests twice and get an average of the two scores, and 2) that the correct answers where given to the students immediately upon answering a question. These two facts brought to light a weakness in the course that could be exploited by an elegant collaborative turn-taking method of sharing correct answers using Google Docs among a small group of students.

Although it is not discussed in the article, I imagine that this course did not have a final exam, or if it did, it followed a similar format (multiple-choice, with the answers given immediately) and naturally was un-proctored. Therein lies one problem. If the final exam is not proctored then what incentives do the student have to learning anything during the semester? However, this point is moot if there was no final exam in the course.

In the case where there is no final exam and the weekly tests are essentially summative evaluations, more design features need to be put into place to give incentives to students to learn. Does the feature of being able to take a multiple choice test twice help students learn? In a summative evaluation, does the feature of giving students the answer to a given question immediately after answering help them learn?

What do you think would be a better design for this course? Let us know by leaving a comment below.

Add Post To:

Humanline - more free images for educational use

I know I have written about free photo repositories before (here and here). Here’s the new kid on the block: Humanline. It’s an image library of arts, history and science that is mainly focused on licensing images for use in education (your ears should perk up now) and for commercial use. On the education side, a careful read of the license shows that for most applications usage is free of cost. The usual credit-line requirement is there; much like a Creative Commons-type license. There is a pre-validation step that you must undergo when applying for your free educational account that normally takes 48 hours.

At this time the database of images is not gigantic; however, it is growing. They are actively looking to partner with other image repositories, museums, galleries, private collectors or archives – basically any organization that has public domain images in their possession. Heck – maybe they should partner with Wikimedia Commons - they are HUGE! (*grin*).

You can browse the Humanline repository or search with key words. When I searched for ‘chemistry’ the selection was limited but it did return a very nice portrait illustration of Robert Boyle the 17th century Irish Chemist that is credited with several discoveries about gas laws. So if you were designing and developing a unit for first year chemistry, such a portrait could be useful. A search for the term ‘nightshade’ yielded some interesting classical illustrations of related plants. In an educational setting these engravings would be fun to compare to real life photos of the same plant.

So despite its limited size at this point, Humanline has potential. Do you know of any new image repositories that can be used freely in education settings? If so, please leave a comment below.

Add Post To:

Video is KING in distance education

With the field of Distance Education exploding due to the maturation of the Internet, it’s no wonder more good quality free academic content is finding its way on the web. Here’s a great example that I just learned about from Dr. James Harris at the University of Leeds in England. It’s called The Faculties and it contains video snipits of senior-level highschool (or junior-level university) lectures spanning several fields including Biology, Chemistry, English, History, Math, and Psychology. To date, they have produced over 360 videos with a promise of more to follow. If you are not in the business of distance education this repository remains useful. Instructors can request downloads of the videos so that they can be played for students even in classrooms that are not wired for the Internet.

This semester I am teaching a second year chemistry course via distance and I have some students in the class that have asked me if I could provide links to some useful videos on specific topics. Naturally, I am always on the lookout for them so I was very excited to learn about The Faculties. I’ve watched about a dozen videos so far and they are very good. There is definitively something for everyone in this repository and you can stay up-to-date on the progress in building this repository via Twitter (@thefaculties). Have a look at some of the videos and let us know what you think. Do you know of other similar free video repositories other than the standard YouTube search? If so, let us know by leaving a comment below.

Add Post To:

Teaching and Learning Centres: Who Needs Them?

Short Answer: we do.

Long Answer: I work at an established Military College. By established I mean 1876. Futhermore, this Military College has been empowered to confer university degrees since 1959. So these two facts together will lend most observers to believe that teaching and learning has been going on here for many years.

So why isn’t there a Teaching and Learning Centre here? Hmm. I don’t really know. The question was raised again in my mind when I read the excellent report by the Higher Education Quality Council of Ontario entitled Teaching and Learning Centres: Their Evolving Role Within Ontario Colleges and Universities.

Have a read of this paper. After reading it I have a question for you. Can you think of any good reasons for denying instructors and students access to a Teaching and Learning Centre? If not, why aren’t they found at all post-secondary institutions? Please leave your answers in a comment below.

Add Post To:

75,000 Hit Thursday!

Back in April 2010, I was ecstatic to report that this blog reached 50,000 hits. It took a glorious 5 years

and 8 months of blogging before this blog crossed the 50,000 hit threshold (that calculates out to an average of about 171 hits/week over the 5 year 8 month period). Well, the hit frequency is steadily growing because e-Learning Acupuncture crossed the 75,000 hit mark today and the most recent 25,000 hits were amassed in only 1 year and 9 months. During that time the average weekly number of hits has climbed to 284 hits/week. I love it!

Happy 75,000 Hit Thursday to all the independent bloggers out there! May the hits come fast and furious to you all for years to come!

Add Post To:

Learning because ‘it’s cool’ say 17 year old boys

When was the last time you heard kids say they did a science project outside of class time because learning is cool? Honestly, I’m not sure I’ve heard that very often in my career as a science educator. I’ve seen a lot of students say that learning at school is cool depending on the subject matter or the project, etc; however, this story is a bit different.

Two 17-year old kids from Newmarket, Ontario sent a Lego minifigure carrying a Canadian Flag into space and filmed it all. Yup - still pictures and video were taken for the entire 80,000+ ft (24 km) climb and the 122+ Km distance the contraption traveled. They did all of this on their own time, with their own funds, away from school in a completely unrelated fashion from any assignment, science fair or school project. Why? Because they just thought it would be cool.

Awesome! I agree: cool!

Add Post To:

Top Four HOT Concepts in Distance Learning

1. Distance Learning itself. At my institution, we are scratching our heads a little bit wondering why we have an above average number of enrollments in our distance learning courses. The same enrollment boom is occurring at other institutions like Bryan College. However, if you do a quick Google search and read a few reports it is clear that the pundits have been predicting that distance learning enrollments will rise. Student seem to be gravitating towards learning opportunities that are not tied to them having to be in any particular location at a prescribed time during the day or week. So called ‘anytime, anywhere’ learning. They enjoy the flexibility of distance learning away from the traditional 1-hour (or 3-hour) face-to-face lecture format. So Distance Learning itself is a really hot concept. Be sure your institution plays into the needs and wants of today’s students. When you design your distance learning courses, lean towards asynchronous models that do not prescribe the students to assemble (even virtually) at any particular time during the week. These synchronous models can be less attractive to students especially if they reside in a different time zone that the host institution. If you cannot avoid some synchronous activities in your distance courses, then be sure to be up front with your students and clearly state the degree of synchronous commitment expected of students before they register in the course.

2. Mobile learning. With each passing set of Christmas holidays, more students find themselves joyful recipients of an iPad, iPhone or other fancy tech thingy under the tree. The number of students that come equipped with tablets, smartphones, e-readers and other gadgets is increasing quickly. They are using these gadgets daily for numerous activities and the mobility affordances provided by these items is very attractive to users. Leverage this trend and be sure that your course materials are mobile friendly. PDF files can help but if you want to go one step further, try ePub and Kindle-specific MOBI files. With just a little more effort on your part (more info here), you can provide students with choices in the file formats for their course materials. Choices usually equate in the short run to student satisfaction. Satisfy your students’ desire for mobile learning by making your course materials mobile friendly.
3. eTextbooks. The tide is turning (albeit slowly) in the publishing industry. Attention is slowly drifting away from the model of having a hardcopy textbook with associated electronic resources online towards a more student-centered model of have an integrated electronic textbook that students can either rent or buy, and store on their own electronic devices. With each passing semester I learn from my students that they are interested in these digital versions of textbooks (sometimes it's because they may cheaper than hardcopy!). For example, I just received an email last week from a student expressing great joy because the publisher of the textbook for my course was providing a Kindle version for sale. The idea of keeping all his textbooks assembled into his tiny Kindle was a BIG advantage for this student. This reinforces the previous point above Mobile Learning. The penetration of mobile devices into the student body is now driving textbook publishers to devote more resources to eTextbooks. So what can you do about this? If you are a textbook author, then encourage your publisher to make eTextbook versions of your book. If you are in the process of selecting a book for your course, then take the extra moment to compare if each of the titles you are considering has an eTextbook format. Making wise choices that facilitate students to learn “anytime, anywhere” will prove to be a win:win for both the students and the educational institutions involved.
4. Free Resources. Ask any student and she will tell you “Free is King”. Whether you are an instructional designer or a professor, challenge yourself to scour the Internet to look for freely available resources that relate to the topic of your course. Critically evaluate each one and choose the best ones to integrate into your course. Yes – this takes time. However, if you do just a little bit every semester, you’ll soon have course chock-full of fantastically freely available resources that will make your course better. Better courses without increasing cost equals more satisfied students. That logic is easy to understand. Want an even better idea? Design a non-traditional assignment where you ask your students to scour the net and evaluate resources for quality. Let them do the legwork of finding the material and then you can integrate the best of found treasures into future iterations of the course. Brilliant!

Do you have a comment about these Top 4 concepts or do you have a HOT concept for distance learning that didn’t make it into my Top 4, if so, please leave a comment below.

Add Post To: