2013-02-12 Meeting of the Software Carpentry Study Group
Round 3.2
Agenda:
- If I asked you to put time estimates on getting from novice to competent (with respect to the topic you chose for your post), (a) could you, and (b) what would they be?
- How long would it take you to assess someone’s level if you were watching them work?
- (For next time) How would you assess someone’s level remotely?
Joon Ro:
- http://teaching.software-carpentry.org/2013/02/23/mastery-table-matrix-programming-with-python/
- “10 hours to get from novice to competent”
- Q: One ten-hour day or an hour a day for ten days? A: The latter is better (at least two days).
- Q: Given that answer, is the current intensive boot camp format a useful one? A: Taught NumPy at Austin, couldn’t get into details.
- Q: Is the value in the details, or in the “big picture” of vectorized computation? A: Three hours ought to be enough for that.
Amy Brown:
- http://teaching.software-carpentry.org/2013/02/22/mastery-table-latex/
- “My novice is pretty novice, so that’s a lot of learning curve, so weeks at least.”
- LaTeX builds on so much: the command line, text files, compiling.
- Q: If they’ve used a compiled language like Fortran/C/Java, and now they’re learning LaTeX? A: about five hours.
- Q: How much is detailed (applicable) knowledge, and how much is “big picture”? A: 2/5 concepts, 3/5 details.
- Q: Can the concepts be learned without teaching the details? A: No. Have to show examples (details) or the concepts/theory remains intangible.
Karthik Ram
- http://teaching.software-carpentry.org/2013/02/25/mastery-table-version-control/
- “A day or two” to get to using Git at a reasonable level: half-day of instruction to get them started, plus a couple of eight-hour days of work engaging with the material.
- Q: How much is conceptual, how much is details? A: 70% of two whole days is conceptual (wrapping mind around concepts).
- Many aspects of Git are not intuitive.
- Get them to the point where they know they want to do X or Y, but don’t know the command — that’s easier to tackle than “I don’t know what I’m trying to do”
Greg: has trouble teaching Git. Contrasting the proportion of conceptual for Joon’s matrix programming and Amy’s LaTeX versus Karthik’s Git, it seems Git relies much more on concepts — maybe that’s why it’s so hard to teach? Particularly for someone who is still conceptually in SVN.
Karthik: Teaching Git is a bit about teaching philosophy — give learners enough concepts so they can proceed and so they can’t identify holes in their concept map when they get to them. The Git learning curve continues to slope for weeks and months into using it.
Greg: How ignorant can you be about [the inner workings of] Git and still get things done? This question does not seem to be answered.
How long to figure out what level they’re at?
- Amy: 2-5 minutes with LaTeX
- Joon: 5 minutes with NumPy
- Karthik: 5-10 minutes (assuming they’re doing more than basic Git operations)
Greg: suggests you can’t really tell novice from expert until you see someone deal with a problem. If they are dealing with things that are working, it’s easy to confuse speed (typing!) with competence.
For Round 3.3:
Part 1: Suppose you can’t watch someone work; suppose you can only set them some tasks and see the person’s solutions. What two tasks would you set to separate novice from competent, and what two other tasks would you set to separate competent from expert? (You can either re-use your topic from Round 3.2, or pick a new topic, or (Greg’s favorite) do this for someone else’s Round 3.2 topic.)
Part 2: Discuss how much more insight you would get (if any) if you recorded their screen and them thinking aloud while they worked. Explain why you think this would help (if it would).
Notes:
- Summative assessment: a grade or score for purpose of ranking.
- Formative assessment: information on where the gaps in your knowledge are — what you need to work on.
- Greg: We usually don’t know how long it takes students to do their homework assignments. Even with exams, we don’t know how long individual questions take. Think about tasks which will give you extra insight if you can see how they are solved.
- We are interested in coming up with both kinds of assessment for SWC. We are offering a “driver’s license” for a supercomputer in the UK so we need pass/fail summative assessments. We want formative assessments to give learners useful knowledge about their skills to take away.
Some links:
- Wiggins & McTighe’s “Understanding by Design” framework is a good way to think about how to integrate assessment (or rather, how to work backward from what we want to change, to how we’ll tell if it’s changed, to what we have to teach). This short overview covers the main points; the actual book (linked at the start of this bullet) takes a couple of hundred pages to say twenty pages worth of stuff.
- This three-page summary of the differences between diagnostic, formative, and summative assessment may also be useful.