Digital teaching resources: How do we measure success?

Those that read regularly, or listen to our podcast, will know that I love technology, and that I also spend a good bit of time cogitating on how we can best use technology in higher education. As part of another paper I wrote I proposed three “rules of thumb” to guide us when we infuse technology into education.  Specifically I ask does the technology:

a) improve the educational content
b) Free up the educator’s to focus on content rather than process, or
c) enable students to grasp the information in a better/faster/cheaper way?
I regularly struggle with each of these, with my focus lately shifting to “c” and how students learn.  I want my students to in some sense control how they learn, so that they will have to rely less on structured “office hour” visits (and the limited time available for them) and more on their own abilities to learn at their own pace.  Given this emphasis I have tried several avenues.
First, I have been recording lectures and making them available as audio podcasts through iTunesU.  If a student didn’t quite grasp what I was saying (often because I talk too fast) then they can go back, listen again, and see if that makes things more clear.  I have in some instances created not only an audio recording of the lecture, but taken the time to provide a “video” version as well, linking the slides from the presentation to the pacing and the voice, so they can more easily follow along if they are at their computer.  [1.  Of course, having the option for both audio, and video, provides the greatest flexibility, allowing students to “take me with them” and listen when and where they wish, and then watch when they have the time to be seated at a display.]
Second, I have been developing video (or “Screencast”) tutorials that step students through solving samples of the problems for each chapter, usually ones that are similar to the homework problems they will be seeing.  Then for a select few problems I make tutorials stepping through the solutions to the homeworks that were assigned.  I have developed a blend of videos that use the powerful Excel tool to solve some of the problems, but also solving some “by hand” through the use of the Tablet PC and the ability for a tablet PC to record what I write.  In this way the students can step through, at their own pace, the problems and the solutions and get a better sense of the thought processes and steps that are necessary to solve these “real world problems.”
Not surprisingly, some students choose to use these resources, and some do not.   So I face a struggle in evaluating the efficacy of these tools.  Are we successful if:
  1. All (or at least a majority) of students use these tools regularly?
  2. Students grades (as measured through standard testing procedures) increase when using these tools?
  3. The number of students seeking assistance through traditional methods (office hours, email, phone calls) decreases?

How do we know if it is working?

I have had a simple measure:  Do some (or even any) students use the tools, and when they use them do they feel that they are grasping the material better?

My thought has been that numbers don’t matter as much as outcomes for individuals and if some students find value then these approaches are worth continuing.

What are your thoughts?  Must we have numbers to be considered successful?

One thought on “Digital teaching resources: How do we measure success?

Leave a Reply

Your email address will not be published. Required fields are marked *