This training course only a start.
If you’d like to help us make it better,
we would welcome additions discussing:
how education research is done: qualitative studies, quantitative studies, and comparison studies
Dewey, Piaget, Vygotsky, Freire (or, key figures in 100 words each)
the history of distance education (or, everything old is new again)
MOOCs
computer-based homework/teaching systems
other forms of assessment
an overview of research on novice programmers
problem-solving skills overview
setting expectations in the classroom
promoting effective study habits
We would also appreciate additions to this list of things we don’t do,
and explanations of why not:
peer instruction
This powerful teaching method has been proven effective,
but we are already asking workshop participants to assimilate a lot of new things,
and picking up a new learning technique while learning the basics of coding and data wrangling
seems too much to ask.
certification
Many people have asked us to certify workshop participants in the same way that we certify instructors,
but any meaningful certification process would require a lot of resources to set up and run.
I. Disseminating: Curriculum & Pedagogy
Change Agent Role: tell/teach individuals about new teaching conceptions and/or practices and encourage their use.
Diffusion Implementation
II. Developing: Reflective Teachers
Change Agent Role: encourage/support individuals to develop new teaching conceptions and/or practices.
Scholarly Teaching Faculty Learning Communities
Environments and Structures
III. Enacting: Policy
Change Agent Role: enact new environmental features that require/encourage new teaching conceptions and/or practices.
Quality Assurance Organizational Development
IV. Developing: Shared Vision
Change Agent Role: empower/support stakeholders to collectively develop new environmental features that encourage new teaching conceptions and/or practices.
Learning Organizations Complexity Leadership
</table>
The eight italicized approaches are:
* *Diffusion*:
STEM undergraduate instruction will be changed by altering the behavior of a large number of individual instructors.
The greatest influences for changing instructor behavior lie in optimizing characteristics of the innovation and exploiting
the characteristics of individuals and their networks.
* *Implementation*:
STEM undergraduate instruction will be changed by developing research-based instructional "best practices" and training instructors to use them.
Instructors must use these practices with fidelity to the established standard.
* *Scholarly Teaching*:
STEM undergraduate instruction will be changed when more individual faculty members treat their teaching as a scholarly activity.
* *Faculty Learning Communities*:
STEM undergraduate instruction will be changed by groups of instructors who support and sustain
each other’s interest, learning, and reflection on their teaching.
* *Quality Assurance*:
STEM undergraduate instruction will be changed by requiring institutions (colleges, schools, departments, and degree programs)
to collect evidence demonstrating their success in undergraduate instruction.
What gets measured is what gets improved.
* *Organizational Development*:
STEM undergraduate instruction will be changed by administrators with strong vision
who can develop structures and motivate faculty to adopt improved instructional practices.
* *Learning Organizations*:
Innovation in higher education STEM instruction will occur through informal communities of practice within formal organizations
in which individuals develop new organizational knowledge through sharing implicit knowledge about their teaching.
Leaders cultivate conditions for both formal and informal communities to form and thrive.
* *Complexity Leadership*:
STEM undergraduate instruction is governed by a complex system.
Innovation will occur through the collective action of self-organizing groups within the system.
This collective action can be stimulated, but not controlled.
## Why Do(n't) We Teach X?
Workshop attendees and trainee instructors often ask why we don't teach
high-performance computing, machine learning, Perl, or a long list of other topics.
Our answer is that as with every curriculum,
the question is not, "What would we like to add?"
but, "What are we willing to take out in order to make room?"
We believe our core topics are the absolute minimum that researchers need to know
in order to work efficiently and reproducibly.
More importantly,
we don't know what we could take out to make space for something else.
One thing we *do* know is that we do not wish to become embroiled in debates
over the relative merits of different languages or operating systems.
No one has ever demonstrated that R programmers are more productive than Python programmers,
and proficient users of Windows seem just as productive as equally-proficient users of Unix.
If a learner asserts that their favorite tool is better than alternatives in some way,
ask them for their data;
if they don't have any,
point out as gently as possible that we're supposed to be scientists,
and that if we want politicians, business leaders, and the general public
to pay attention to our findings on climate change and drug-resistant diseases,
it behooves us to try to meet those same standards ourselves.
> ## Evidence and Its Absence
>
> As far as is practical,
> our teaching methods are based on the best available evidence.
> We wish we could say the same about our content,
> but very little research has been done on what researchers actually use
> and what impact it has on productivity.
> An example of what we wish existed
> is [this summary][stefik-summary] by Stefik et al
> of empirical research on the usability of programming languages
> (while [this full-length paper][stefik-paper] gives an idea of what's possible).
{: .callout}
## Why We're Not a MOOC
> If you use robots to teach, you teach people to be robots.
{: .quotation}
This difference between what novices are doing when they learn,
and what competent practitioners are doing,
is one of the reasons we have stopped trying to teach via recorded video
with auto-graded drill exercises.
Any recorded content is as ineffective for most learners as broadcast television,
or as a professor standing in front of 400 people in a lecture hall,
because neither can intervene to clear up specific learners' misconceptions.
Some people happen to already have the right conceptual categories for a subject,
or happen to form them correctly early on;
these are the ones who stick with most massive online courses,
but many discussions of the effectiveness of such courses
ignore this survivor bias.
## Program Assessment
The Carpentries' greatest weakness is a lack of systematic assessment:
while we have done some small-scale studies of the impact we have on our learners,
and Dr. Beth Duckles' studies of [why instructors join us](https://software-carpentry.org/files/bib/duckles-instructor-engagement-2016.pdf)
and [why people qualify but then don't teach](https://software-carpentry.org/files/bib/duckles-non-instructor-report-2016.pdf)
are very insightful,
we still don't know what learners actually adopt
or what effect it has on their productivity,
the reproducibility of their work,
and so on.
We have sometimes used this as the basis for an in-class exercise.
Working in groups of four,
trainees brainstorm answers to the following:
"Your dean has provisionally agreed to set aside funds to support some Carpentry workshops over the next year,
but wants to know how you will tell at the end of those workshops whether the money was worth spending.
Given the resources you have,
what information can you collect,
how would you analyze it,
and why do you think it would be convincing?"
Each group then presents its best idea,
which the trainers and other trainees critique.
This exercise always generates a lot of discussion,
but end-of-day assessment has usually indicated that trainees don't find it particularly useful.
We have therefore cut it,
but may re-introduce it if and when we include a module on program assessment.