Approach – Peer Instruction in Computing Science

Summary of Approach

Peer Instruction (PI) is an active learning technique, from the “flip-teaching” stable.  It helps the teacher make best use of their time with students, in particular enabling them to highlight and correct particular student alternative conceptions.

Benefits of Peer Instruction

One of the most important benefits we have found of using PI in computing classrooms is that students practise talking about their subject.  This is an essential part of the PI process.  If students don’t have the language to be able to talk about their understanding, how can they really understand it?

How often do your students get the chance to talk about computing concepts?  This gets at what it is to be a computing scientist, or for that matter, a proficient in any field.  We may acquire an initial body of knowledge and skills independently – but we validate and build on it in discussion with others.

The other key benefit of PI is that it is an efficient way to draw out student misconceptions or alternative conceptions.  Well-crafted PI questions are based around one or more known mis/alternative conceptions – with the different answer options capturing both the correct understanding and the known misunderstandings.  See elsewhere on this website for Alternative Conceptions.

How Peer Instruction Operates

Using the PI approach, students complete preparatory work prior to the PI session which gives them at least an initial understanding of a topic area.  During the PI session, students are asked a sequence of multiple choice questions (MCQs).  A specific protocol is then followed:

1. Each student decides on an answer to the question, registering their answer using some form of response system.
2. Once all students have responded, they discuss their answers in groups of three.  They attempt to persuade each other why their answer is correct, listening carefully to their colleagues’ explanations too.  Discussion continues for between 2-5 minutes – you judge when to draw it to a close.
3. On the basis of the discussion, each student answers the MCQ a second time using the response system.
4. The teacher can see a summary graph of responses and uses this to lead off a class-wide discussion about the question

Keypoints

Peer Instruction is a general educational technique, not specific to computing science.  While many examples used here may derive from programming in particular, PI can be used anywhere in computing science where you know that students are at risk of developing alternative conceptions.

Hints on writing good Peer Instruction questions

Good Peer Instruction sessions depend on well-crafted questions. Here are some hints:

• A good PI question never depends on factual recall alone.  There is nothing to discuss, and so a factual recall PI question would be completely pointless.
• An effective exam MCQ may not be an effective PI MCQ.  In an exam, one guards against guessing by having at least 3-4 distractors (incorrect answer options) as well as the correct answer.  PI is a formative, not summative, technique, and hence guessing is pointless and there is no need to guard against it.  Often, the best PI questions have only 3 answer options, even 2 perhaps, depending on what misconceptions you know about and can craft into your question.
• Use your wisdom as a teacher to identify good question topics.  Where is it that your students have stumbled again and again over the years?  What is/are the fundamental misconceptions that have led to this?
• As ever with any MCQ question, put as much text into the question stem, enabling the answer options to be as short as possible.  This aids reading and comprehension.

Because a PI question typically has to fit onto a slide displayed to the class, neither the question nor the answer options can be all that large.  This may be a benefit, as it highlights PI’s great value in ensuring that students fully understand the operation and development of small code fragments, for example.  Given our increasing understanding of the developmental nature in learning computing, PI therefore supports an essential stepping-stone in students’ development.

Hints on running a good Peer Instruction session

Whilst the basic framework is discussed above, there are nuances as to the best way to run PI sessions:

• Students must have prior experience of the content or skills on which the PI questions are focussed, using whatever methods you choose.  This could be reading or practical exercises for homework or in class.  There is no point in asking PI questions about material/skills that students have not yet had a chance to develop conceptions/understanding.
• The size and makeup of the discussion groups affects the quality of discussion.  The typical size is three – not everyone gets to speak in larger groups, and in smaller groups there won’t be enough difference of opinion to fuel the discussion.  Friendship groups may not be good as power relationships mean that one student may dominate.  We suggest not putting strong students with weaker students, as the weak will tend to defer to the strong, with nobody’s conceptions being challenged.
• Students don’t necessarily know how to have a good discussion that elicits each other’s understandings and compares them.  A discussion sequence of – Student 1 “I think it’s answer A”; Student 2 “I agree”; Student 3 “Yes, I agree too” – is not much use for exploring understanding.  A better response for Student 2, even though he/she agrees, would be to ask “Why do you think that?” to check that Student 1’s reasoning matches his/her own.
• Typically, don’t show the summary response chart to the class before either the individual or the class wide discussion – it is too directive on the discussions.
• Consider the purpose of the group discussion and how you will lead it.  A key purpose is to encourage students to practise speaking about their understanding.  This is both for themselves and so that other students can hear ways of talking about the subject that are not just their own or yours, the teacher’s.  One opening gambits for the discussion could be “Explain why you answered the way you did…” or “If you changed your mind in the discussion, explain your original thinking and how the discussion changed your mind…” or [if this fits] “I see that you are cleanly split between two answer options – tell us how you argued for your answer…”.  The key is that you are not just looking for them to say what the right answer is. You also want them to practise explaining why it is the right answer.

If this kind of active learning is a new approach in your class, be sure to take at least a little time to motivate why you are using it.  Learners can be resistant to new methods!

Limitations

Peer Instruction questions, given their MCQ format, do not lend themselves well to assessing problem solving skills.  The only way is to approach such skills obliquely, e.g.which of the following code fragments solves the following problem?

• Fred has created the following code to solve problem X.  Do any of the following errors occur in Fred’s code? [include a ‘no errors’ option]. Or, which of the following is a good explanation for why lines 4 and 5 have been included?

PLAN C Techniques

PLAN C teachers have developed PI questions for a wide range of topics within the qualifications.  Searching on this website should unearth sets of questions for particular topic areas.

Specific analysis of 100+ PI questions based around introductory programming showed that the questions fell into a range of categories.  These were rather grandly titled the Abstraction Transition Taxonomy, but don’t worry about the crazy name.  The Taxonomy points to some key conceptual question formats to follow that are showing good results in computing classes.  It should help you ensure that you don’t get stuck in one style of question only.  Instead, you can evaluate your students on a broad range of programming skills.

You can find an outline of these different question styles in this Technique post.

Reasons for Recommendation

Peer Instruction has a huge body of evidence in support of its effectiveness in improving learning.  Eric Mazur, the Harvard professor of physics who designed the method originally, has shown “times 2” learning gains over more traditional didactic university teaching methods.  This improvement was sustained over multiple runs of the course and compared against his own teaching of the same course not using PI.  Similar results have been identified in other subjects, including Computer Science.  In CS, studies have been principally led by Beth Simon, Quintin Cutts, Leo Porter and Daniel Zingaro.

Micheline Chi has proposed a framework for comparing the huge number of techniques that come under the “active learning” umbrella.  This is called the Passive-Active-Constructive-Interactive framework.  Active is the lowest level of engagement and involves the learner keeping up at least with what the teacher is saying.  This is in comparison with being completely zoned out, looking out of the window or surfing the web – which is Passive.  Constructive is where the learner is having to create something as part of the learning process – taking some notes, formulating a question, re-writing something in their own words.  Interactive is where the learner must share their construction with other learner(s), discussing their understanding of the topic.  An Interactive learning method is accepted as the gold standard.  Beth Simon ran two identical courses, apart from one using Peer Instruction while the other used traditional “engaging” techniques such as extensive use of demonstrations.  Rating the courses, students in the first noted that they were principally involved in Interactive activities while those in the second course rated their engagement only in the Active category.

Teacher Experiences of Benefits

I found that Pupils were independently holding conversations with each other and making good use of complex terminology to explain what was happening with code and what problems were occurring.

My pupils were able to start identifying misconceptions in areas such as Boolean expressions and confidently describe what problems they had and how they fixed them.

For example one issue that comes up time and again involves complex Boolean conditions and pupils forgetting to put in a second complete expression.

e.g.

REPEAT

RECEIVE newWeight FROM (REAL) KEYBOARD

UNTIL newWeight > 20 AND < 70 

Pupils were able to realise why this code would not work as it only had one complete binary expression. Pupils were then able to correct their mistake.

REPEAT

RECEIVE newWeight FROM (REAL) KEYBOARD

UNTIL newWeight > 20 AND newWeight < 70

I also found that pupils were more confident when writing their own programs and were able to fix more errors themselves. This technique also made pupils focus on the purpose of code fragments.

A. McSwan – Knox Academy

I use PI questions at level 4 and level 5 when teaching Scratch and TrueBASIC respectively. They are extremely useful for dealing with alternative conceptions, and generally getting pupils to discuss and think about code. It works really well with code samples. I have found that group discussion of code is making it easier for pupils to step back from the “live” scenario of coding at a computer to confront the issues in understanding that they develop when learning about programming.

P. Liddle, Bearsden Academy

PI questions have been excellent in encouraging pupils to use terminology correctly, especially when they are attempting to persuade others of their opinion. It helps to emphasise that programming is not code entry.

W Sargent PT Business and Computing Hyndland Secondary School

Research References

• Cutts, Esper, Fecho, Foster, Simon “The Abstraction Transition Taxonomy: Developing Desired Learning Outcomes through the Lens of Situated Cognition”.  In: Clear, A., Sanders, K. and Simon, B. (eds.) Proceedings of the Ninth Annual International Conference on International Computing Education Research, Auckland, New Zealand, 10-12 Sept 2012. ACM: New York, NY, USA, pp. 63-70. ISBN 9781450316040 (doi:10.1145/2361276.2361290)
• Simon, B., Parris, J., & Spacco, J. (2013, March). How we teach impacts student learning: peer instruction vs. lecture in CS0. In Proceeding of the 44th ACM technical symposium on Computer science education (pp. 41-46). ACM.
• Cutts, Q., Esper, S., Fecho, M., Foster, S. R., & Simon, B. (2012, September). The abstraction transition taxonomy: developing desired learning outcomes through the lens of situated cognition. In Proceedings of the ninth annual international conference on International computing education research (pp. 63-70). ACM.
• Simon, B., & Cutts, Q. (2012). How to implement a peer instruction-designed CS principles course. ACM Inroads, 3(2), 72-74.
• Porter, L., Bailey Lee, C., Simon, B., & Zingaro, D. (2011, August). Peer instruction: do students really learn from peer discussion in computing?. In Proceedings of the seventh international workshop on Computing education research (pp. 45-52). ACM.
• Catherine H. Crouch and Eric Mazur (2001), Peer Instruction: Ten Years of Experience and Results, Am. J. Phys., 69, 970-977