Despite technical glitches (including an Etherpad failure that cost us many of our notes), we had a good set of meetings this week: there were lots of interesting concept maps, and lots of good discussion about what goes into them and how often the concrete things in the topic being discussed (e.g., servers in a discussion of SSH) are also concepts. Notes are below…
The next exercise focuses on using assessment to steer teaching, and has several parts:
- Read Chapter 3 of How Learning Works.
- Pick a topic from the Version 5 lecture notes that you think you could teach in 5-10 minutes. (It can be the same topic you used in your concept map, but doesn't have to be.)
- Write a blog post by Thursday, October 2 containing two simple assessments you could use to tell whether your learners have understood your lesson well enough for you to move on to the next topic or not. Do not post the answers until you've had at least two people comment on your post (see below). Give your post the categories "Assessment" and "Round 11/3″.
- Comment on two people's posts by answering their questions, and also giving them feedback: were the assessment questions clear? How long did it take to answer? Etc.
The first of your assessments must be a multiple-choice question (MCQ). Think carefully about the distractors (wrong answers) you provide — each should help you diagnose a different misunderstanding. Once you've had at least two comments on your post, add a comment yourself explaining what you think each wrong answer points at.
The second of your assessments can be any other kind of exercise: fill in the blanks, a Parsons Problem (where you provide jumbled lines of code that have to be put in the right order to solve a problem), matching items from Column A with Column B, etc. Whatever you choose, it must be something your learners can do in just a couple of minutes (because there's no point teaching for 5-10 minutes and then waiting 30 minutes to see if people got it or not).
We'll meet again on October 8 and 9, and I'll do my best to show up at the right time on those days. I look forward to seeing you then.
Notes
- A "time map" of British history on display at the Tate Britain museum in London: http://www.tate.org.uk/download/file/fid/39587 (events not arranged chronologically but are instead connected through common concepts or questions — created by kids)
- In previous sessions, Concept Maps were assigned and students were told to pick a topic they'd like to teach. Times ranged from 30 minutes to 5 hours. Students that took 5 hours spent a large chunk of the time coming up with a topic that would make a good concept map. This iteration, students were told to just pick a topic from the Software Carpentry lessons. The time it took to draw concept maps stabilized around 30 minutes.
- From a convo with Ashwin: As soon as you have a concept map on paper, what will you erase in order to add a new concept you've just come up with? Think about what's really important. What will we not do in this class in order to make room for this other thing? Drawing a concept map helps your prioritize, especially if you decide beforehand, how many ideas/the complexity of ideas you want to introduce.
- From a convo with Carl: You have different types of arrows. There are edges which connect an abstract relationships, and there are edges which represent an exchange of information. http://files.software-carpentry.org/training-course/2012/08/cherubini-venolia-whiteboard-2007.pdf What do programmers actually draw when they're talking to each other? What do we define as an entity and an edge? There tend to be no consistent semantics. It's hard to keep a conversation 100% in your head, visual cues help you recall the linguistic memory of the conversation. The point of the diagram is to have bits to point at to remind you what you've said 10 minutes ago. The diagram is a short term memory cue and often loses meaning days later. Maybe this is why UML never caught on. we don't think in semantics.
- From a convo with Lauren: As she was drawing out the concept map, she discovered new relationships that she didn't think of before. She has experience with teaching the concepts in a certain order which she drew the map for and drawing the map helped her discover novel relationships.
- From a convo with Daniel: He believes that assigning students a concept map after a class helps re-affirm the lesson. Greg believes that this should be assigned to teams of ~4 for grading. He was always surprised by the concept maps they drew. Assigning the map is a good way to force consolidation. Compare students' maps to your own and see how it challenges your own understanding of the relationships and how you're teaching them.
- From a convo with Sahar: She has taught the class she drew the map for before. She had never before though think about the logical links and struggled with it. She believes it's because the technique is new to her.
- From a convo with Kirill: It took Kirill 30 minutes to draw the concept map and they had experience with them before. For others it took them 30 minutes but they believe it's because the technique is new. Kirill spent the 30 minutes doing graphical design but the idea was ready instantly.
- Most people are glad for feedback. Most of us are overly concerned that something we say will be hurtful. In Greg's experience only one person was upset by the feedback and it was because the commenter was actually downright rude.
- Different concept maps ~ different paintings of the same thing: done in different ways but expressing the same ideas, and they are all understandable
- Worse is better: better to get something out there then spend too much time polishing
- Testing/Assessment
- Summative assessment: Final exam. wraps everything up at the end (sum of all acquired knowledge)We are not just assessing the learning with summative assessment. We are also assessing whether the teaching was successful. The problem is that as soon as we are assessing teachers, they will teach to the test because it's the only rational response.
- Formative assessment: Coaching during the learning process. It's the assessment that you get over the course of the course.
- Assessment NOT THE SAME as testing
- Testing is a sensitive topic because it can be misused so easily.
- Reverse Instructional Design:
- The idea of test driven design is that if I want to add a feature, the first thing I do is write the test. That's my goal post. Then I write the code that passes the test.
- When teaching, first construct the summative exam to assess whether your teaching was successful. May be useful to present key concepts early on as "key concepts" or "learning goals" (or something like that) to give students some targets to work towards.
- Also design the formative assessments, so that students come into the final assessment well prepared
- If you don't know how to tell whether someone has learned something, you shouldn't be teaching yet. What change will I be able to observe in the learner? The second step is to figure out the formative assessment that you will provide along the way. This is so the learners can assess along the way what they've learned and what they haven't. This is the reverse of how many of us teach.
- You start with what you want to teach and then base an assignment off of that. It helps a lot with scoping. (i.e. When a teacher goes through the first half of the class really slow and then the rest of the course at lightning speed.)
- Tell students early on what you expect them to learn.
- It's very hard to judge expertise but it's fairly straightforward to assess competence.
- On wrong answers:
- Use them as a lens to discover what the student is doing wrong.
- If questions are designed well, wrong answers provide insight to the nature of students' misunderstandings
- On multiple choice
- We like using multiple choice questions because they are quick. If everyone gives you the same wrong answer, you know where the misconception comes from. If the answers are all randomly distributed, then you need to re-examine how the lesson was taught.
- When you're designing a multiple choice question, you want 1 right answer and 3 or 4 plausible distractors with diagnostic power. You want wrong answers that tell you what the student has misunderstood. If you use 27 + 15 = 1001, you don't learn anything about what the student has misunderstood.
- You must have in mind, what is revealed by each of the wrong answers? Every wrong answer should give you insight.
- Options with no diagnostic power are a waste of time and add no value.
- If you push this all the way, you end up with a concept inventory. You give a series of multiple choice questions and then you interview the examinee. You then look for patterns and use that to conclude where the misunderstanding is. You then revise the questions over and over again, until there is very little overlap. You want to be able to assess the precise concept the student is not getting. You want to not need the interview in order to find the pattern.
- In general concept inventories are useful but you cannot share them widely because then they end up on the internet and the tool has become invalidated.
- There are institutional screw ups that lead to people not owning concept inventories that they make.
- If you cover common wrong answers in your lectures and go over them slowly post assessment, the class will know that you care about them. A powerful technique.
- The purpose of a programming exercise is to let you practice a skill. It's muscle building. Not the same as an assessment.
- Using questions as assessment techniques:
- Scaffolding problems (e.g. fill in the blank)
- "Puzzles" to be assembled, e.g. given a set of words, create a sentence (Parson's problem)
Exercises for Next Meeting:
- Read chapter 3 of How Learning Works.
- Take a topic out of SC's lectures, something you can teach in 5 minutes. Come up with 2 questions that you'd ask at the end and you can use to assess whether they understood the lecture or not? One of those two questions must be a multiple choice question. Do not post the answers or the diagnosis value of your questions in the first week. In the second week, comment on other people's multiple choice questions and answer them.
- Multiple choice question ("MCQ"):
- Only after a couple of comments do you post the answer and explain the diagnostic value.
- Your second question should be brief and be anything except the multiple choice question. Is there a style of question that you really like that you think will give you an idea of whether the student has understood the concept? E.g. Parson's Problem.
- Answer and comment on the questions of others.
QuestionsPlease answer these in bullets under your name in the list below.
- How long did it take you to create your concept map, and where did the time go?
- What (if anything) did you learn about your topic or how to teach it from doing your map?
- What (if anything) did you learn about your topic or how to teach it from the feedback you got?
- What (if anything) did you learn about concept mapping from giving people feedback?
- Do you think this is a time-effective tool?
19:00 Eastern on Wednesday Sept 24
- Greg Wilson (Software Carpentry / no post)
- Damien Irving (University of Melbourne / no post)
- Ashwin Srinath (Clemson University, SC) http://teaching.software-carpentry.org/2014/09/17/parts-of-a-function/
- 15-20 minutes
- To stress that the "interface" and "implementation" are the two primary 'parts' of a function, and that the implementation remains always detached from the world. I think the concept map makes this easier to see.
- To talk about the locality of variables (data) defined inside a function
- I realized that smaller concept maps are generally a better idea than more complex ones (I found that I learned more from looking at smaller ones). Maybe many small concept maps are better than one large, complex one?
- Definitely.
- Greg Caporaso (Northern Arizona University / http://teaching.software-carpentry.org/2014/09/16/concept-map-python-if-statements/ )
- about 20-30 mins, spent some time looking at examples and thinking about what i wanted to map out. then figuring out what the labels between concepts should be.
- Don't think I learned anything directly. About how to teach it: relationship between if, elif, and else (i.e., what is the same about them and what is different) is useful.
- It's hard to say, bc I wasn't actually using it for teaching, so I didn't have a basis for comparison (ie., how long it would have taken to prep without it). I did like the idea though, and used it to flesh out ideas for a presentation I gave last week, and that was helpful.
- Carl Boettiger (UC Santa Cruz / http://teaching.software-carpentry.org/2014/09/12/concept-map-ssh-keys-github/ )
- ~ 30 minutes, mostly in picking a topic and thinking through what the nodes would be
- Difficult to deal with 'concepts' vs 'flow diagrams' for this particular topic (e.g. tempting to make nodes represent the computers involved in the ssh auth process and arrows be data transfer, rather than 'concepts' and 'relationships'
- feedback emphasized the problem above, also spatial configuration matters
- some maps showed really good use of spatial configuration
- For some topics yes, for others, no.
- Timothee Poisot, UdeM / U Canterbury ( http://teaching.software-carpentry.org/2014/09/11/concept-map-pipes-and-all-that/ )
- 20 to 30 minutes (I did two versions)
- it was interesting to have to put all of the building blocks together in a coherent way (teaching wise)
- how to integrate examples, and how we tend to read things in the way block are layed out on the page
- we all have different ideas on the topics, and concept maps can be a reflection of our own biases/experience (especially striking when you see several maps on the same topic)
- very — I've used it for other things since
- Lauren Michael (UW-Madison) http://teaching.software-carpentry.org/2014/09/18/concept-map-commands-executables-and-arguments/
- ~10 minutes (plus short spurts of pre-planning the topic and concepts to include); most time went to adjusting the exact wording and visual layout of different terms and connectors.
- I discovered a new/different method of verbally introducing the concepts, as well as how to conceptually link them in ways that I don't typically do when discussing the same topic with a learner.
- Commenters alerted me to ways to connect "input" and "output" back into other terms in the concept map, in ways I hadn't considered before. Typically, input and output come up later in specific examples.
- I learned potential difficulties in constructing concept maps for certain topics. For example, one map attempted to give concrete examples of 'cd' commands within the conceptual discussion of "path", which was more difficult to convey in a concept map (difficulty in establishing connections between actual commands).
- The act of making concept maps and reviewing those of others is great!
- Bennet Fauber (UM Ann Arbor) http://teaching.software-carpentry.org/2014/09/13/concept-map-git-repository/
- ca 20-30 min
- It's difficult to break a complicated 'topic' into small pieces without losing the sense of the whole.
- I picked a 'concept' rather than a 'process', I think, and I was curious how that would play out. I probably should have been more clear about that part.
- I am confused about the difference between a 'concept' map and 'process' map, if there even is such a thing as the latter. It did seem that it was process mapping more often than not. Perhaps I'm too literal in my interpretation of 'concept'.
- This was an exercise about concept maps, so I think it was worth the time in that respect. I'm not sure that I would choose this tool, or the map that I made, to present the topic, but maybe something close. It was worthwhile trying it, and looking at some of the other maps were well worth the time trying to think of comments. I might lift some of those and use them.
- Anja Boskovic (McGill University) http://teaching.software-carpentry.org/2014/09/17/concept-map-shell-interacting-with-the-file-system/
- I'm not sure how long it took exactly. I think approximately an hour. The time mostly went into re-reading the lesson and figuring out which concepts were important. I went through 2 drafts.
- I learned that if you focus on concepts with a lot of connections at the beginning of a lesson, you can repeat those concept as you explore new concepts hence re-affirming them. In my case, if I started the lesson with the concept of current working directory, I can repeat that when I cover ls, cd, etc.
- I learned that directionality is important with respect to the edges between concept and that there's an order that makes more sense.
- I think it was a time effective tool.
- Matt Shirley (Baltimore MD / http://teaching.software-carpentry.org/?p=8567)
- Cody Hinchliff / http://teaching.software-carpentry.org/2014/09/22/concept-map-function/
- birgit bachler (Aotearoa Digital Arts Network) http://teacihing.software-carpentry.org/2014/09/18/concept-map-python-conditionals/
- around 30 min — mostly finding out what a concept map is and what topic to choose, thinking about what i could teach with 5 minutes time
- learned: quite interesting to activate visual/drawing skills to small but complex topics
- feedback: helpful as in how my drawing is more a flowchart than a concept map and how i could have changed bits and pieces
- feedbacking: was easier to feedback on simple concept maps as some maps would need some explanation on where to start, what are the main concepts, it's hard to feedback on complex drawings
- probably a good tool to include in handouts, take longer to create but probably very reusable once done well
- Walter Langelaar (Victoria University Wellington) / http://teaching.software-carpentry.org/2014/09/19/concept-map-absolute-and-relative-paths/
- questions:
- 15-20mins, figuring out what constitutes a 'concept map', thinking about the topic and then putting it down on paper
- I learned how to break down the topic and which parts are exclusively relevant to the 'absolute' and 'relative' domains
- I learned that apparently I got the idea of a concept map wrong, and that I should've identified the relations between boxes
- It's time-effective in the sense that it gets a topic down in a coherent visual manner, in such a way that it makes it easy to talk about the individual components — so short creation time, but effective in approaching the whole topic
- Daniel Smith (Auburn University / http://teaching.software-carpentry.org/2014/09/17/concept-map-numpy-array/ )
- ~30 minutes- planning out the map took the most time.
- No, but it was interesting seeing the connection I had subconsciously made.
- There was a comment that said comparing numpy arrays to python list was confusing- I thought this was interesting and will keep concept maps limited to the topic at hand.
- I actually learned more from reading other concept maps in detail then making my own.
- Absolutely. I believe I am going to give my class concept maps after a lecture to reinforce mental models.
- Alif Zaman (U of Toronto / http://teaching.software-carpentry.org/2014/09/18/linux-shell-creating-things/ )
- About 30 min. I mostly spent for thinking how I should start
- I was mostly focusing on getting a simpler topic to teach because I have no teaching experience for a larger group. So learning something absolutely new was not my primary goal, but how to teach the material was my main goal. If I have to teach, my next step is to generate an teaching outline and go forward from there.
- Based on the feedback, most said my 'concept map is more like a flow chart'. As far as I understand, flow chart has a definitely flow path from start to end point. I tried to represent mostly the inheritance relationship between multiple concepts using arrows. I am a little confused about the comments, maybe needs to learn deeper concept map.
- I think concept map is the primary step before any communication activity like teaching (which is more like an interactive presentation)
- Sahar Rahmani (University of Western Ontario/http://teaching.software-carpentry.org/2014/09/17/concept-map-creating-function-in-python/)
- ~30 min and most of that was about how to connect the concepts and how to show them
- The process of teaching this subject and what should I say first and what next
- Some direction I choose and in my mind was obvious was not really obvious for the others and my idea was not correct
- write my concept map as clear and clean as possible in this case I can read them easily and I can follow the concepts better.
- yes for sure specially if we want to share our teaching plan with somebody else or clear things for ourselves totally worth the time, I did it for my class this week and helped me alot.
- Kirill Palamartchouk, Newcastle University, UK (http:n//teaching.software-carpentry.org/2014/09/11/concept-map-pipes-and-filters-2/
- 30 min
- Not much — I have done this before
- Be very precise but concise. Try being factually correct
- It is difficult because you may hurt the feelings of people you have never met
- Yes. It helps to be prepared to questions
- Juan Nunez-Iglesias, University of Melbourne: http://teaching.software-carpentry.org/2014/09/18/concept-map-fundamentals-of-version-control
- ~30 min. time spent mostly identifying what should be a node. Comments reflect this since all pointed out need to split apart some of the nodes.
- I've taught git before, in a mostly pragmatic way. The concept map would definitely help in guiding the class, making sure everyone is on the same page.
- Every aspect of concept mapping is somewhat challenging. In all cases, what should be a node, what should be a link, and the layout of the map had elements I might change. Even "experts" appear to have trouble — fig 2.1 in HLW felt amateurish to me and did not convey much meaningful information.
- Despite above caveats, there is no question that they are a very useful tool to clarify your thinking and design coursework.
- Cody Hinchliff: http://teaching.software-carpentry.org/2014/09/22/concept-map-function/
- How long did it take you to create your concept map, and where did the time go?
- I spent about 10 minutes making my map. Most of that time was spent thinking about what to put in, with only a little time for actually drawing the map
- What (if anything) did you learn about your topic or how to teach it from doing your map?
- I learned that the notion of a function is actually relatively vague given all the nuances about how they are used and what they do. For instance, I used some relationships like "may have" and words like "usually", which garnered comments from others who suggested that this vagueness might make it more difficult for unfamiliar students to understand.
- What (if anything) did you learn about your topic or how to teach it from the feedback you got?
- Someone suggested that I give an example, which I think would be useful, and I learned that people have very different ideas about how best to present the information, which I think is indicative that different kinds of explanations are more or less accessible to different people. Not necessarily about my topic, but interesting nonetheless
- What (if anything) did you learn about concept mapping from giving people feedback?
- Well, something I found particularly interesting was Greg's comments about the way people think, suggesting that taking out strict "concept" semantics makes more natural maps.
- Do you think this is a time-effective tool?
- Yes. I will likely use this as an exercise in classes and also for myself when designing lessons
10:00 Eastern on Thursday Sept 25
14:00 Eastern on Thursday Sept 25