Meeting of the Software Carpentry Instructors Study Group
Round 4.2/4.3
2013-04-17
Agenda
- How long did it take to:
- Come up with your questions?
- Revise your questions?
- How confident are you that other instructors would agree with your assessments of levels?
- What did you learn from the feedback you got on your questions?
- How much did you revise them?
- How much did you rethink your novice/proficient/expert distinctions?
- How do you think you’d do on other people’s questions?
- I.e., how often would you be novice, proficient, or expert?
- Next exercise (note: this has changed!)
- Assessment questions
- Took about 30 minutes to put together
- easy to distinguish novice from competant
- harder to come up with expert questions
- Greg: takes 1 day to generate 6-7 questions for an exam, another 1 day to revise based on feedback from colleagues
- “Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.” — Brian Kernighan
- Experts don’t just know when to break the rules: they also know how to reason backward from effects to causes (i.e., how to debug).
- One way to create expert-vs-proficient assessment questions is to give them something that contains an error and ask what they need to fix it.
- Easy to mark: yes or know.
- Gives them an opportunity to show transference of relevant knowledge from related domains.
- Challenges in writing good questions
- removing ambiguities
- leaving opportunities for learners to be partly correct and correct in part (which aren’t the same thing).
- yes/no questions better for quick assessment of large numbers of people but harder to assess shades of grey
- Competent practitioners often better than experts for teaching novices
- Experts:
- grossly underestimate how long it takes to do stuff
- wider blind spot
- too quick to add nodes to their concept map when explaining
- Concepts you use in your assessment questions for experts should not be part of your explanations to novices (or used in a two-day boot camp for beginners)
- Good teachers learn how to see the world through novice eyes
- Experts:
For Thursday, May 2 (note the change of day):
- Read the Hannay and Prabhu papers.
- Read Facts and Fallacies of Software Engineering.
- Pick one or two results from the book that you didn’t already know, and that connect to your own work somehow.
- Put together a very short presentation (no more than ten slides, no more than three minutes) to explain it, the evidence for it, and its implications to our intended audience.
- Put the slides online somewhere and then add a link to it to this post.
It’s OK if several people pick the same topic—you’re unlikely to see the same connections between it and your own work, and much less likely to come up with the same presentation 
What I would like you to do is sketch a very quick concept map for yourself before you start putting the presentation together, and think of a simple question you could use to tell whether people (a) already understand the idea before you start talking, and/or (b) understand it when you’re done, and/or (c) are actually acting on it. You don’t need to post these, but I’d like you to introspect and see whether thinking in these terms helps you put together a better presentation in less time.