Software Carpentry Instructor Training Round 8.3 (Thursday, February 13, 2014)

We had a good turnout this week, with 31 people taking part in three meetings to discuss multiple-choice questions. The notes are below, but a few things that came up in discussion are:

  1. MCQs are best used for formative assessment, to tell you (and your learners) what needs to be covered, because they're quick and scalable. They shouldn't be used for summative assessment (to measure what has been learned) because they don't give you insight into thought processes: if someone does all the steps right, but drops a decimal place, all you'll get is "wrong answer" rather than "right process, a bit sloppy".
  2. It takes several iterations to create an MCQ that's unambiguous and has good distractors. If you're teaching a class several times, you can beta-test MCQs with the current learners for use with the next group (the GRE does this).
  3. A concept inventory uses carefully-validated MCQs to triangulate on learners' misconceptions (see Allison Tew's thesis from Georgia Tech, or this paper, for a concept inventory for basic computing concepts). The problem is, they're a lot of work to develop.

Perhaps the most important discussion point was this: no matter what you do, or how long you spend preparing, it won't be perfect. We have a wide spread of backgrounds, abilities, and interests in every class we teach; nothing we can prepare will be exactly right for everyone, so a "New Jersey" solution (90% right, can ship today) isn't something to be ashamed of: it's still helping them a lot, and the time we would put into polishing would be better spent helping people climb the next hill.

For the next meeting:

As you'll have learned from Chapter 3 of How Learning Works, motivation is often the most important determinant of whether or not someone learns something. It's easy to think of ways to demotivate people who are learning to program (telling them that it's easy, or taking the keyboard away from them because it's easier for you to type in the solution than to watch them struggle, are two ways). Our goal this week is to come up with ways to motivate them instead. Write a blog post detailing something specific you could do at some point during your teaching to get learners excited about the topic you're about to cover, and make them believe that they can learn it, and want to learn it. Be concrete: "tell them a story about how this tool helped me" isn't something that someone else could pick up and use.

Separately, please have a look at these three papers from the web site (and any others from our materials page that catch your eye as well):

19:00 Eastern

14:00 Eastern

10:00 Eastern