«A Paper Submitted to the Graduate Faculty of the North Dakota State University of Agriculture and Applied Science By Keith Stefan Abeyratne In ...»
ANALYZING STUDENT LEARNING OUTCOMES IN PROGRAMMING COURSE USING
INDIVIDUAL STUDY VS. PAIR PROGRAMMING
Submitted to the Graduate Faculty
North Dakota State University
of Agriculture and Applied Science
Keith Stefan Abeyratne
In Partial Fulfillment of Requirements
for the Degree of
MASTER OF SCIENCE
Computer Science August 2014 Fargo, North Dakota North Dakota State University Graduate School Title
ANALYZING STUDENT LEARNING OUTCOMES IN PROGRAMMING
COURSE USING INDIVIDUAL STUDY VS. PAIR PROGRAMMINGBy Keith Stefan Abeyratne The Supervisory Committee certifies that this disquisition complies with North Dakota State University’s regulations and meets the accepted standards for the degree of
MASTER OF SCIENCE
SUPERVISORY COMMITTEE:Dr. Gursimran Walia Chair Dr. Saeed Salem Dr. Janet Knodel
8/25/2014 Dr. Brian Slator Date Department Chair
This study supported the implementation of specific assessment strategies to assess individual programming abilities during pair programming situations. Results found that students perceive pair programming as being beneficial and all of the subjects who used pair programming indicated that they would prefer using it again as opposed to working individually.
ACKNOWLEDGEMENTSThis research could not have been accomplished with only my contribution, but there were many individuals who genuinely helped me in various ways. First and foremost, I would like to thank my advisor Dr. Gursimran Walia for accepting me as a student, and providing me freedom to learn. His guidance, encouragement, patience, thoughtfulness, and genuine support for my research, academic achievements, as well as completing my thesis were enormous. I would also extend my gratitude to Dr. Saeed Salem and Dr. Janet Knodel for accepting to me on my committee in such short notice. This wouldn’t be possible without your help and quick response.
I would like to give my gratitude to my colleagues at NSDU Computer Science department for their support, friendship and collegiality, especially staff at the NDSU Graduate School and computer science department, without your help and guidance this wouldn’t have been a reality.
Last but not least my heartiest gratitude goes to my parents and my family, for giving me their blessings, strength, love, comfort, guidance, support, courage and the best education. I owe them for giving me their best throughout my life.
LIST OF TABLES
LIST OF FIGURES
1.1. Problem Statement
1.2. Motivation and Research Goals
2. BACKGROUND & RELATED WORK
2.1. NDSU Studies
2.2. Problems when Using Pair Programming at NDSU
3. EXPERIMENT DESIGN
3.2. Study Subjects and Course Assignments
3.3. Study Procedure
3.4. Data Collection and Evaluation Criteria
5. THREATS TO VALIDITY
6. DISCUSSION AND CONCLUSION
1: Independent and Dependent Variables.
2: Student Allocation
3: Marks Comparison of Group A and B
4: Final Grade Comparison
5: Mean & Standard Deviation of the Exam Marks
6: Introducing Pair Programming Initially versus Later
1: Experiment Design.
2: Experiment Procedure
3: Average Marks
4: Letter Grade Comparison
Pair programming (PP), by definition, is a programming technique in which two programmers work together at one computer on the same task . The term “pair programming” was first used in 1999 as one of the core practices in the Extreme Programming (XP) software development methodology in industry. As defined by Williams et al. , pair programming refers to a practice in which two programmers sitting side by side using only one computer to work collaboratively on the design, algorithm, code or test. The person typing is called a driver, and the other partner is called a navigator. Both partners have their own; responsibilities; the driver is in charge of producing the code. The navigator’s tasks are more strategic, such as looking for errors, thinking about the overall structure of the code, finding information when necessary, and being an ever-ready brainstorming partner to the driver. This arrangement leaves the driver free to work on the tactical aspect of the program.
Challenges of creating rigorous, syntactically correct code without worrying about the big picture, gives the navigator the opportunity to consider strategic issues without being distracted by the details of coding. Together, the driver and navigator create higher-quality work more quickly than either could produce on their own. Pair programming is one of the key practices in Extreme Programming (XP) . It was incorporated in XP, because it is argued to increase project members’ productivity and satisfaction while improving communication and software quality . Since then, pair programming has become one of the most researched topics in the realm of agile software development techniques .
The practice of pair programming has been widely implemented in the industry as well as in educational settings (Domino et al , Chong et al. ). A vast amount of research on pair programming has been conducted to observe the benefits of the technique and to understand how the practice can improve students’ learning outcomes. The stated benefits of pair programming
were identified as follows:
• Improvement in students’ academic performance such as in final and midterm exams, quizzes, programming assignments and overall course grades [13, 14, 15, 18, 16, 17].
• Improvement in programming productivity in terms of the time spent on coding and quality of the software produced [24, 02, 24, 25, 08].
• Increase in students’ retention rate and course completion rate [13, 14, 18].
• Increased students’ confidence level and enjoyment in learning programming [02, 13, 14, 16, 17].
• Reduced staff workload [22, 23].
• Increased efficiency in helping female students to work in programming tasks [19, 20, 21].
Recently, many researchers have explored the suitability of pair programming to conduct the programming laboratory classes in educational institutions. The pair programming can be considered as one form of collaborative learning. In collaborative learning, small groups of students associate with each other where each member contributes his/her personal experience, information, perspective, insight, skills and attitudes, which can help, improve the learning efficiency of others  (Klemm, 1994). When this kind of collaborative learning is adopted to do programming assignments, generally the students divide the work among them and complete it individually with little or no help from other students of the group.
In literature, many benefits of pair programming have been proposed, such as increased productivity, improved code quality, and confidence, to name a few. On the other hand, pair programming has also received criticism over increasing efforts, expenditure and overall personnel costs, and bringing out conflicts and personality clashes among developers. However, the scientific empirical evidence behind these claims is currently scattered and unorganized, and thus it is difficult to draw conclusions whether the pair programming indeed claims to be beneficial for student learning or not. Researchers have also investigated the effect of the factors (e.g., student’s skill levels) on the pair performance of students.
In fact, Hanks  points out regarding the quality improvement claims that “There does not appear to be any empirical evidence that the programs [produced by pair programming] are better in terms of design, readability, maintainability, or other internal quality attributes.” As a consequence, the industry has been rightfully hesitant in adopting the pair programming practice.
On the contrary a study by John Nosek  examined how pair programming impacted the amount of time it took to complete a programming task. His results found that pairs took about 70% of the time to complete a task as individuals. There are many studies that conclude pair programming helps create better quality code, and improve student learning.
A recent study held at NDSU by Radermacher et.al [26,27,28,29,30] indicates some valid points, which may be overlooked in other studies. Radermacher’s study  provided increased support that the pair programming is more effective in improving student’s learning of programming concepts compared to individual learning Radermacher found that overall assignment scores improved for students participating in pair programming.
Pair programming has proved its usefulness in teaching and learning programming skills.
It also has received many criticism; therefore, the data we have is widely scattered. Furthermore, there are very few studies which indicate when pair programming should be introduced in introductory computer science courses to maximize its benefits. The main objective is to understand when pair programming should be introduced in courses for the students. Our goal is to use pair programming to improve student’s learning capacity.
1.2. MOTIVATION AND RESEARCH GOALSIn recent years, the growth of extreme programming (XP) has brought considerable attention to collaborative programming. Pair programming is a collaborative approach that makes working in pairs rather than individually the primary work style for code development.
Because pair programming is a radically different approach than many developers are used to, it can be hard to predict the effects when a team switches to pair programming. Despite the advantages proposed for pair programming, many still suspect the overall usefulness and benefit over traditional solo programming. One of the most questioned aspects lies on the feasibility of achieving superliner speedup as compared to the legacy pattern .From a simple management’s viewpoint, there is no reason to pair up developers if they cannot do things twice faster. In addition to productivity, people also doubt if the improved software quality deserves hiring twice many programmers . Although pair programming is claimed to cost an insignificant 15% more effort yet achieve a higher quality than solitary programming on the same task , the paradigm is still questioned if solo programming plus an additional review phase, which might be cheaper and equally effective, will achieve the same goal. The purpose of this study was to investigate the effects of pair programming on student performance based on the time it was introduced, and subsequent pursuit of computer science related degrees among college students.
The pattern of pair programming in which two individuals develop one software module together has attracted researcher and practitioner’s interest for almost 15 years. People following the pair programming protocol sit in front of one screen, use a set of keyboard and mouse to collaboratively solve a programming task . While one developer is modifying the source code, another is required to perform continuous code-review. This pattern was claimed by it advocators to yield earlier release and higher software quality , and was included as one of the rules of Extreme Programming (XP) , a popular software development methodology used widely in the software industry.
Studies conducted on pair programming identified some of its advantages for the teaching learning situation (Tomayko, 2002; Williams & Kessler, 2001; Williams & Upchurch, 2001). For example, it has been found that when programmers work in pairs, fewer errors are made than in individual programming situations (Tomayko, 2002), resulting in better programming performance, increased confidence, and decreased frustration levels of the programmers (VanDeGrift,2004). A possible explanation for these findings could be that pair members help each other to solve the problem and complete the programming task together. Thus, there seems to be some agreement among researchers that pair programming could be a promising teaching strategy for teaching programming skills (VanDeGrift, 2004).