Simon’s Physics Education Blog

A colleague of mine often responds to discussions of the benefits of e-learning to students with the comment:

“ … but how can you show quantitatively that it does any good? “

It’s a fair question and one that I have found myself struggling to answer in the past. There’s a number of inherent problems with trying to do “experiments” in the scientific sense on a learning activity with a class cohort.

• the opportunities for experimentation tend to come around relatively infrequently (say, once every year)
• there can easily have been external reasons why comparing last year’s cohort with this year’s is not a level playing field. Maybe the entrance grades went up, different people taught the course, the structurer of the academic year altered radically (as it did in Edinburgh two years ago).
• the ethics of having half your class as a control group who do not get the benefit of the wonderful new development that you want to evaluate…

 I heard a talk by Norman Reid a few weeks ago, from the Centre for Science Education at Glasgow and he was describing the idea of splitting the class into two and doing two repititions of a similar experiment, thereby allowing each half to act as the other’s control group. Any one student is treated just like any other, insofar as they are in the control group once and the other group once.

I thought about using this to try and evaluate the use of podcasts as a delivery vehicle for pre-lecture preparative work, to aid understanding of what we know to be the traditionally conceptually difficult bits of a course.

Rather than selectively releasing material to a group of students (who might not all listen to it) I planned to broadcast the material at the start of a workshop (tutorial-ish) session fir half the class (and not for the other) about a week ahead of the material being covered in the lectures.

The following week, I would ask the entire class a concept question (we call them time to think’s) at the beginning of the section on the material covered in the podcast pre-lecture.

Based on workshop sign-in sheets, we should be able to assume an even spread of ability (could cross check this with start-of-course diagnostic tests) in each workshop and could then look at the response profiles for the “have’s” as compared to the “have not’s” (collected using an elecctronic voting system during the lecture).

Repeat a few weeks later where the have’s become the have not’s.