«A Paper Submitted to the Graduate Faculty of the North Dakota State University of Agriculture and Applied Science By Keith Stefan Abeyratne In ...»
Despite the possible benefits of pair programming as a teaching strategy, some of its limitations for student assessment have been documented as well. One such frequently encountered limitation is that some students may receive undeserved credit for the successful completion of a program (McDowell, Hanks, & Werner, 2003). The assumption is that it is difficult to assess students’ individual programming abilities reliably in pair programming situations. The first author of this article experienced a similar problem when implementing pair programming as a teaching strategy during a second year Information Technology (IT) course for student teachers. The students achieved high marks for their pair assignments, but significantly lower marks for their individual assignments. This begs the question whether the results of pair programming assignments are reliable indicators of students’ individual programming abilities. Naude and Hörne (2006) even classify undeserved credit under the umbrella of ‘cheating,’ and although cheating is a strong word, the researchers emphasize the seriousness of the situation.
Several researchers highlight the problem of assigning group marks to individual students (Parsons 2004). It is necessary to determine the contribution of each individual member, and according to Parson (2004), this is not an easy task. Some students could contribute little or nothing at all, and if there is no assessment of students’ individual contributions, marks awarded could be an inaccurate reflection of a student’s abilities (Cheng & Warren, 2000). According to Parsons (2004), a fair mark allocated to a given student should reflect that individual’s effort and abilities. This statement is supported by the research of Verhaart, Hagen, and Giles (2005), who also wished to determine whether students’ marks in group assessments correlated with their marks in individual assessments. They proposed two different assessment methods as best practice for assessing an individual’s performance in group work. In the base mark adjusted method, they give individual tests, as well as self- and peer assessments after every group work session. In the task splitting method, they split the group work task and require some work done individually another work done in groups. A specific weighting is allocated to the different tasks and students need to include a peer review to adjust the group contribution. Verhaart et al. (2005) concluded that both these methods are “valid assessment forms, producing marks which seem to reflect the students’ typical level of achievement.” Although it cannot be clearly concluded from previous research whether pair programming reduces the development effort of students it improves the quality of the artifacts.
Additional research may assist practitioners to know when and how to make pair programming more effective. Research has just begun to understand the impact of pairing individuals with different levels of expertise [17, 3]. Combining pair programming with solo programming are also being researched [18, 15]. Furthermore, there are evidence showing that pair programming might help teaching activities in Computer Science programs, for example, improving student confidence and course performance.
At North Dakota State University, pair programming was introduced in the introductory Computer Science programming courses (CS 160 and CS 161) beginning 2010.[26, 27, 28, 29 and 30] A series of empirical studies (SIGCSE’2011, CSEE&T’2011, SIGCSE’2012, ICER’2012, FECS’2014) showed that Pair Programming had a significant positive impact on the students’ acquisition of programming concepts and learning outcomes. Radermacher and Walia et al. 2012, studied multiple aspects of pair programming and have conducted several empirical studies North Dakota State University over the past two years. During this time, researchers received valuable feedback from course instructors about the effects of implementing pair programming in their introductory computer science courses. However these instructors also expressed concerns about the use of pair programming in their courses. These include being able to ensure equal participation from pair members and not being able to assess individual learning outcomes effectively The following section briefly talks about the NDSU studies and the problems and concerns with the use of Pair Programming that led us to the current study (i.e., impact on the learning outcomes by introducing Pair programming in the beginning half vs. later half in the course) being reported in this paper.
NDSU Studies: A series of studies related to pair programming were conducted at NDSU [26, 27, 28, 29, 30]. Radermacher et al. 2012, reports on the results of two different studies conducted during the spring 2010 semester . Subjects in the first study were 35 students enrolled in one section of the CS1 course and the second study included 39 students enrolled in two sections of the CS2 course. Subjects in the CS1 course were split into two groups, one which used pair programming and one that did not; whereas the subjects in the CS2 course were paired based on declared major. Researchers reported that subjects from both the CS1 and CS2 courses indicated that they felt pair programming improved their understanding of programming concepts. Another major result indicated that pairing a computer science (CS) student with a non-computer science (nonCS) student produced less compatible pairs as compared to CS-CS pairs and nonCS-nonCS pairs.
Radermacher et al. 2012, reported another empirical study that investigated the effects of pair programming on student-instructor interactions during programming laboratory sessions.
Subjects in this study were 44 students enrolled in one section of the CS1 course and 53 students enrolled two sections of the CS2 course during the fall 2010 semester. Subjects in the CS1 course alternated between using pair programming and working individually during lab sessions, whereas subjects in the CS2 course only used pair programming. Researchers monitored these lab sessions, marking the number of questions asked, how long it took before an instructor could address the subject’s question, and how long the instructor spent interacting with the subject.
Results of the study indicate that when pair programming is used, students spend less time waiting for assistance from an instructor and spend more time interacting with the instructor, likely due to a decrease in questions related to syntax errors or other minor problems.
Another experiment at NDSU investigated the effects of pairing subjects based on their mental model consistency levels (ranging from highly consistent to highly inconsistent) at the beginning of the semester to evaluate changes in the students’ mental model consistency and their programming performance [27, 28]. The evidence suggest that such a pairing strategy can be an effective way if previous performance data is not available and that certain mental-modelbased pairing arrangements (and not all) are more effective in motivating students towards greater consistency and resulted in better performance on exams.
2.2. PROBLEMS WHEN USING PAIR PROGRAMMING AT NDSUIndividual Assessment: Difficulties assessing individual learning and ensuring that both partners are benefiting from the use of pair programming is a common issue with pair programming. One of the CS1 instructors at our university also expressed concerns that several students, who he felt would not have normally been able to pass the class, had been able to pass the class because their partner was able to help carry them. Based on eight semesters of historic data for this instructor’s class, there was a large increase in the number of students who were able to pass the course when pair programming was used and when students worked individually.
Results for the drop-rate of the course during the fall 2010 semester was about 8% when students used pair programming compared to the historical drop-rate of approximately 18%.
Bevan, et al 2005. Had also reported that students were willing to submit assignments that only one of the students had completed. Williams, et al 2002. Also expressed similar concerns when they discovered instances of students who performed well on pair programming exercises, but scored poorly on exams, suggesting that one partner may have been completing most or all of the work on the programming exercises, this phenomenon also was observed by our own instructors.
Other instructors indicated that this may have been an issue as a large number of laboratory assignments that required substantial out-of-class work in order to complete in summary there was no easy way to ensure that both partners had contributed equally.
How much Pair Programming: All three of the instructors with whom we worked have expressed some concern that using pair programming for every assignment may not be as effective as only using it for only some assignments. One of the CS1 instructors felt that most new students did not have any programming experience and that until they gained some knowledge they wouldn’t be able to effectively pair. One instructor noted instances where one member of a pair would actually be providing the other with incorrect information.
Another instructor felt that some assignments should be completed individually in order to provide the opportunity for students to show that they have mastered the ability to program without the need of a partner. The instructor stated that occasionally one partner would be absent from a lab and that the remaining student seemed to struggle, even though that student had done well on previous lab exercises.
Previous research also has indicated that some students feel that it is important to work individually. There is no doubt that pair programming is beneficial to student learning and enjoyment and has benefits for instructors as well. It has already been suggested that pair work should not constitute the majority of a student’s grade, but has not been determined how much pair activity should be conducted for efficiency of learning.
In introductory computer science courses where a majority of subjects have little or no experience with programming, our instructors felt that they should spend some initial time at the beginning of the course working individually in order to acquire some programming knowledge before working with a partner. Such an arrangement has several benefits. First, it provides performance data that can be used to pair based on ability, as most researchers agree that this is the most desirable pairing strategy. Secondly, it allows some time for students who do not intend to complete the course to drop before pairs are created, minimizing the need to re-pair students.
Additionally, initial programming assignments are more likely be trivial or straightforward and may not benefit from the use of pair programming. However, this has not yet been empirically evaluated. So, the current paper tries to determine which timing of introducing Pair Programming early in the first half of the semester vs. the second half of the semester is the most beneficial to the students learning outcomes.
The main goal of this study was to understand whether introducing pair programming in the beginning of the semester is more beneficial to students compared to introducing it at a later
stage in the semester. The high-level question addressed by this paper was:
When would be the best possible time frame to get students involved in pair programming, in order to maximize the learning capacity of the students in introductory computer programming course?
In order to evaluate the learning efficiency we propose the following hypotheses.
Hypothesis 1: Introducing pair programming in the beginning of the semester significantly improve students learning ability as compared to using Pair Programming later in the semester.
Alternate Hypothesis 1a: Introducing pair programming later in the semester significantly improves students learning ability as compared to using Pair Programming at the beginning of the semester.
Null Hypothesis: Introducing pair programming at any stage are equally effective to improve the students’ learning.
In order to test these hypotheses, a controlled experiment was conducted at NDSU computer science programming course, wherein ones section of the course used the Pair Programming at the beginning of the semester and the other section was introduced to the Pair Programming in the second half of the semester. Details of that study are presented in this section.
These experiments involved two types of learning methodologies: pair programming and a traditional method. It was decided to conduct the experiment in regular laboratory classes.
Here, the traditional method is referring to solo programming where only one student is involved in the development of a program for a laboratory exercise. Thus, the focus of the present study was to compare the learning efficiency of the students when they use pair programming in comparison to the traditional method for laboratory exercises.
Group A: The subjects belonging to this group worked in pairs and used pair programming to complete a programming assignment starting early in the semester. During the mid-way of the semester, the subjects worked individually on the remaining course work.
Group B: The subjects belonging to this group worked individually initially and were paired with a partner to complete the assignment for the reminder of the course.
The high level experiment design is further illustrated in the Figure 1 below.
Independent Variable: The independent variables represent the cause for the effect in any experiment. The two types of independent variables for our study were pair programming and traditional method. The outcome of our study is learning efficiency and the conjectured cause is the learning methodology. Thus, the independent variable for our study is learning methodology. In the present study, we were interested in manipulating the learning methodology to find its effect on learning efficiency and when should pair programming be introduced.
Dependent Variables: Researchers measured the following dependent variables:
Letter Grade: The final letter grade (i.e. `A', `B', etc.) received by the subject was used to evaluate their learning.