«SE 062 649 ED 431 621 Stigler, James W.; Gonzales, Patrick; Kwanaka, Takako; AUTHOR Knoll, Steffen; Serrano, Ana The TIMSS Videotape Classroom Study: ...»
Teachers and Reform A great deal of effort has been put into the reform of mathematics teaching in the United States in recent years. Numerous documentsexamples include the National Council of Teachers of Mathematics Curriculum and Evaluation Standards for School Mathematics (1989) and the National Council of Teachers of Mathematics Professional Standards for Teaching Mathematics (1991)encourage teachers to change the way they teach, and there is great agreement, at least among mathematics educators, as to what desirable instruction should look like. Although most of the current ideas stated in such documents are not operationalized to the extent that they could be directly coded, it is possible to view some of the indicators developed in the video study in relation to these current ideas.
When the video data are viewed in this way, Japanese teachers, in certain respects, come closer to implementing the spirit of current ideas advanced by U.S. reformers than do U.S. teachers. For example, Japanese lessons include high-level mathematics, a clear focus on thinking and problem solving, and an emphasis on students deriving alternative solution methods and explaining their thinking. In other respects, though, Japanese lessons do not follow such reform guidelines. They include more lecturing and demonstration than even the more traditional U.S. lessons, and we never observed calculators being used in a Japanese classroom.
vii and demonstration than even the more traditional U.S. lessons, and we never observed calculators being used in a Japanese classroom.
Regardless of whether Japanese classrooms share features of "reform" classrooms or not, it is quite clear that the typical U.S. classroom does not. Furthermore, the U.S. teachers, when asked if they were aware of current ideas about the best ways to teach mathematics, responded overwhelmingly in the affirmative. Seventy percent of the teachers claim to be implementing such ideas in the very lesson that we videotaped. When asked to justify these claims, the U.S. teachers refer most often to surface features, such as the use of manipulatives or cooperative groups, rather than to the key point of the reform recommendations, which is to focus lessons on high-level mathematical thought. Although some teachers appear to have changed these surface-level characteristics of their teaching, the data collected for this study suggest that these changes have not affected the deeper cultural scripts from which teachers work.
Bearing in mind the preliminary nature of these findings, as well as the interpretations of the findings, we can, nevertheless, identify four key points:
The content of U.S. mathematics classes requires less high-level thought than classes in Germany and Japan.
U.S. mathematics teachers' typical goal is to teach students how to do something, while Japanese teachers' goal is to help them understand mathematical concepts.
Japanese classes share many features called for by U.S. mathematics reforms, while U.S. classes are less likely to exhibit these features.
Although most U.S. math teachers report familiarity with reform recommendations, relatively few apply the key points in their classrooms.
These initial findings suggest a need for continued analysis of these data on eighth-grade mathematics practices. Caution should be exercised in generalizing to other subjects or grade levels.
viii Acknowledgme ts A project as large as this one would not have been possible without the help of many people. A list, hopefully almost complete, of those who contributed to the project is included as appendix C. Aside from this, there are a number of people who deserve special mention. First, we would like to acknowledge our collaborators: Jurgen Baumert (Max Planck Institute on Education and Human Development, Berlin) and Rainer Lehmann (University of Hamburg) in Germany, and Toshio Sawada at the National Institute of Educational Research in Tokyo, Japan. We also wish to thank Clea Fernandez for her input during the early stages of the project; Michael and Johanna Neubrand for help in understanding German teaching practices; Alfred Manaster, Phillip Emig, Wallace Etterbeek, and Barbara Wells for their work on the mathematics content analyses; and Nancy Caldwell of Westat, who helped out in ways too numerous to mention. Shep Roey cheerfully ran many analyses many times, Lou Rizzo carefully developed the weights applied to the data, and Dave Kastberg supervised the final layout of the report (all of Westat); we greatly appreciate their help. The final report was greatly improved by the hard work of Ellen Bradburn and Christine Welch at the Education Statistics Services Institute. Finally, we would like to acknowledge the major contributions of Lois Peak (U.S. Department of Education), without whom this project would never have been done, and James Hiebert, who has improved the project at every step of the way, from coding and analysis to the writing of this report.
Figure 4 Excerpt from the content description column of the lesson table for JP-012 30 Figure 5 Distributions of unweighted average mathematics achievement test scores for classrooms in the Main TIMSS samples and video subsamples from each country 35 Figure 6 Teachers' reports of how nervous or tense they felt about being videotaped 37
Figure 12 Teachers' responses, on the questionnaire, to the question, "What was the main thing you wanted students to learn from today's lesson?" 46 Figure 13 Average number of topics and topic segments per videotaped lesson in each country 47
Figure 21 Average percentage of topics in each lesson that contained applications that increased in complexity vs. stayed the same or decreased over the course of the lesson 54 Figure 22 (a) Percentage of lessons that included teacher-presented and student-presented alternative solution methods; (b) average number of teacher- and student-presented alternative solution methods presented per lesson 55
Figure 30 Percentage of lessons containing links coded as increase in complexity and necessary result/process Figure 31 Average number of codes per node and per link in German, Japanese, and U.S. lessons 67 Figure 32 Percentage of lessons in each country containing mostly single-step, mostly multi-step, or equal numbers of the two types of tasks 68
Figure 34 Percentage of lessons rated as having low, medium, and high quality of mathematical content Figure 35 Arrangement of desks in German, Japanese, and U.S. classrooms 71
Figure 37 Average number of organizational segments in German, Japanese, and U.S. lessons 74 Figure 38 Average number of classwork and seatwork segments per lesson in each country 75
xviii Figure 59 Excerpt from textbook page used in GR-103 Figure 60 Problems from worksheet used in US-016 Figure 61 Average percentage of time in seatwork/working on task/situation segments spent working on four different patterns of tasks and situations in each country Figure 62 Excerpt from chalkboard in JP-034 Figure 63 Excerpt from computer monitor used in JP-012 Figure 64 Excerpt from chalkboard in JP-012 Figure 65 Average percentage of seatwork time spent in three kinds of tasks Figure 66 Categories used for first-pass coding of utterances during public discourse Figure 67 Subcategories of elicitations Figure 68 Subcategories of content elicitations Figure 69 Average percentage of utterances and words spoken by teachers in each country Figure 70 Average number of utterances (out of 30 sampled per lesson) coded into each of six teacher utterance categories Figure 71 Average number of utterances (out of 30 sampled) coded into each of five student utterance categories Figure 72 Average length of student responses as measured by number of words Figure 73 Average number of utterances (out of 30 sampled per lesson) coded into each of five categories of teacher elicitations Figure 74 Average number of utterances (out of 30 sampled) coded into each of three categories of content elicitations
Chapter 1. Introduction The Third International Mathematics and Science Study (TIMSS) is the third in a series of international studies, conducted under the auspices of the International Association for the Evaluation of Educational Achievement (IEA), which has assessed the mathematics achievement of students in different countries.
The first two of these studies (Husen, 1967; McKnight, Crosswhite, Dossey, Kifer, Swafford, Travers, and Cooney, 1987) established that there were large cross-national differences in achievement and provided some information on contextual factors, such as curriculum, that could be related to the achievement differences.
In these prior studies, students from the United States scored low in comparison to other countries.
Not enough was learned, however, about the contextual factors that might help to explain their relatively low performance. Finding out more about the instructional and cultural processes that are associated with achievement thus became a high priority in planning for the TIMSS.
In accordance with this priority, the National Center for Education Statistics (NCES) funded two studies to complement the main TIMSS study. Both of these studies focus on three countries: Germany, Japan, and the United States. The first involves comparative case studies of various aspects of the education systems of each country. The second is the Videotape Classroom Study.
The primary goal of the Videotape Classroom Study is to provide a rich source of information regarding what goes on inside eighth-grade mathematics classes in Germany, Japan, and the United States. We directed our attention to both teachers and students, seeking to describe the classes from both the perspective of teaching practices and that of the opportunities and experiences provided for students.
Aside from these general goals, the study had three additional objectives:
To develop objective observational measures of classroom instruction to serve as quantitative indicators of teaching practices in the three countries;
To compare actual mathematics teaching methods in the United States and the other countries with those recommended in current reform documents and with teachers' perceptions of those recommendations;
To assess the feasibility of applying videotape methodology in future wider-scale national and international surveys of classroom instructional practices.
In this report we will provide a detailed account of the methods used in the study, as well as a preliminary look at the findings up to this point. We have only started to tap the vast wealth of information available in the videos we collected. But we have made great headway in solving the considerable logistical and methodological challenges presented by the study. This report relates what we have learned thus far.
In this introductory section we discuss what can be learned from classroom observation and the advantages offered by the use of video to collect such information. We also discuss the issues and problems that arise in the course of designing and carrying out a large-scale video survey, and we describe some of the approaches we have taken to meeting these challenges. In the Methods section we provide a detailed account of our methods. In subsequent sections we present results, first regarding the content of classroom instruction, then the organization and processes.
21 STUDYING PROCESSES OF CLASSROOM INSTRUCTIONThis is the first large-scale study to collect videotaped records of classroom instruction in the mathematics classrooms of different countries. It also is the first studyfor any grade level or subject matterto attempt direct observation of instructional practices in a nationally representative sample of students within the United States. Thus this study constitutes an important new database and a new approach to data collection for NCES.
Chief among the factors associated with student achievement must surely be the processes of teaching and learning that transpire inside classrooms. Yet, until now there have been no observational data on instructional processes from a national sample of classrooms. In a series of papers commissioned by NCES in 1985, papers designed to set the agency's priorities for the next 10 years, the need for classroom process indicators was raised numerous times (Hall, Jaeger, Kearney, and Wiley, 1985). Cronin (1985), for example, expressed concern with the paucity of data that could document curricular breadth or the actual implementation of curricular reform in the classroom. Peterson (1985) cited a near complete lack of data on the quality of educational activities in the Nation's classrooms, or even on the time teachers devote to various instructional activities. Including such indicators in the future was a recommendation of the 1985 report.
Studies of classroom process can serve two broad purposes: First, they can result in indicators of classroom instruction that can then be used to develop and validate models of instructional quality. That is, we must understand the processes that relate instruction to learning if we are to be able to improve it.
A second purpose of such studies is to monitor the implementation of instructional policies in classrooms. One example of such policies is contained in the National Council of Teachers of Mathematics (NCTM) Professional Standards for Teaching Mathematics (1991). The Standards represents one point of view on what instruction should look like in the classroom. Operationalizing this point of view in a system of classroom-based indicators would allow us to assess the degree to which the Standards are being implemented, and by coupling these indicators with performance measures, the effectiveness of the Standards as educational policy.