# «SE 062 649 ED 431 621 Stigler, James W.; Gonzales, Patrick; Kwanaka, Takako; AUTHOR Knoll, Steffen; Serrano, Ana The TIMSS Videotape Classroom Study: ...»

SOURCE: U.S. Department of Education, National Center for Education Statistics, Third International Mathematics and Science Study, Videotape Classroom Study, 1994-95.

For purposes of content analysis, lesson JP-012 was divided into three content segments. The homework segment at the end was not counted as a separate segment. Each segment became a node on the graph. We will now review the construction of the directed graph segment by segment.

1. First node [(00:27)-(01:26)]. This node consists of a PPD and is identified in this way by the coders. It is worth noting that the PPD is not only stated but also illustrated by several examples using a computer.

2. Second node [(01:26)-(22:57)]. In the second node a problem is posed and then students work on the problem for several minutes. After that, students present two solutions to the problem. The solutions use the mathematical principle presented in the first node and involve explicit use of deductive reasoning in applying that principle: "Since the areas of the triangles..." (19:20). Because the earlier node has a result necessary for the content of this one, this node is directly connected to the first node by a link labeled NR (Necessary Result).

Because the principle of the first node is used to solve the problem of the second node, the second node is labeled "illustrate," as the situation/task in the

## BEST COPY AVAILABLE

3. Third node [(22:57)-(48:58)]. In the third node another problem is posed whose solution depends upon the mathematical principle of the first node. It is, therefore, another illustration of that principle, and the third node too is labeled "Illustrate". This problem is similar to, but more complex than, the problem of the second node. Thus, there is a link between the second and third nodes labeled "C-F" to indicate a more complex conceptual setting, and a link labeled "HP" to indicate that the procedures used in the second node are helpful in solving the problem set in this one. This node was labeled "deductive" because the same reasoning had to be used as in the second node.

Because the third node is indirectly linked to the first one through the second, and because the nature of the linkage between the first and the third is indicated by the two links given, there was no need to provide a direct link from the first node to the third.

The directed graphs took a number of different forms. One additional example is shown in figure

26. Our purpose in presenting this graph is simply to illustrate the general level of complexity of the graphs. We will forgo a detailed explanation of exactly how this graph was derived.

NOTE: PPD = Principle/Property/Definition; HR = Helpful Result; C+ = Increase in Complexity; NP = Necessary Process; I = Inductive Reasoning; D = Deductive Reasoning.

SOURCE: U.S. Department of Education, National Center for Education Statistics, Third International Mathematics and Science Study, Videotape Classroom Study, 1994-95.

Analyses of the Directed Graphs The first analysis based on the directed graphs was simply to count the number of nodes and links of each lesson. The average number of nodes and links for lessons in each country are presented in figure 27. There was no significant difference in the number of nodes or links across countries.

SOURCE: U.S. Department of Education, National Center for Education Statistics, Third International Mathematics and Science Study, Videotape Classroom Study, 1994-95.

**When we examine the structure of the directed graphs, however, we begin to see cross-national differences. Two indicators in particular attracted our interest:**

These indicators seem to measure the coherence of the lesson content because they measure the interconnectedness of different content segments.

The distribution of lessons by number of components and number of leaves is shown in figure 28.

The distribution in Japan was significantly different than that in the United States. For example, Japanese and German lessons were more likely than U.S. lessons to contain only one component. Japanese lessons were also more likely than lessons in the United States to contain one leaf. This suggests that the content of lessons is more coherent in Japan than in the United States.

NOTE: Percentages may not sum to 100 due to rounding.

Further Analyses of Nodes and Links Having examined the number of nodes and links and the structure of how nodes are connected in the lessons of each country, we turn now to examine the characterizations of nodes and links provided by the Math Content Group. A number of characteristics were coded for nodes and for links. For nodes,

**the following codes were applied when the group of coders judged them present:**

MOTIVATIONapplied when a task or situation in the node was clearly used to motivate a PPD that occurred later in the lesson. (Marked as M on the directed graphs.) ILLUSTRATIONapplied if the node included a task, situation, or activity that clearly illustrated a general principle that was explicitly stated earlier in the lesson. (Marked as L on the directed graphs.) DEDUCTIVE REASONINGapplied if there was an explicit use of deductive reasoning within the node. (Marked as D on the directed graphs.) INDUCTIVE REASONINGapplied if there was an explicit use of inductive reasoning within the node. (Marked as I on the directed graphs.) INCREASE IN COMPLEXITYapplied only to nodes that included more than one task or situation when there was judged to be an increase in the complexity of the tasks or situations. (Marked as C+ if increase was clearly and primarily

conceptual, P+ if clearly and primarily procedural. Otherwise, marked simply as

**For links, the following codes were applied:**

NECESSARYapplied when the content of the earlier node on a link was judged necessary in order to take up the content in the later node. The earlier node could be necessary because it provided a result that was used in the later node (marked as NR), or because it described or explained a process that was applied in the later node (marked as NP). The absence of this code might indicate a gap or discontinuity in the development of content.

HELPFULapplied when the content of the earlier node was clearly helpful, though not necessary, for presenting or understanding the content of the later node. Again, it could be either a result (marked as HR) or a process (marked as HP) that rendered the earlier node helpful.

SIMILARapplied when a process (marked as SP), result (marked as SR), or central concept of the later node was similar to a process or result in the earlier node.

DEDUCTIVE AND INDUCTIVE REASONINGcoded when either deductive (marked as D) or inductive (marked as I) reasoning was a significant component of the connection between the linked nodes.

INCREASE IN COMPLEXITYcoded when there was an increase in complexity in tasks or situations from one node to the next. (Marked as C+ if increase was clearly and primarily conceptual, P+ if clearly and primarily procedural.

Otherwise, marked simply as +.) The percentage of lessons that included nodes coded as ILLUSTRATION, MOTIVATION, INCREASE IN COMPLEXITY, or DEDUCTIVE REASONING is shown in figure 29. (Few nodes were coded as INDUCTIVE REASONING, and, therefore, were not included in the following analysis.) There was no significant difference across countries in the percentage of lessons containing ILLUSTRATION or INCREASE IN COMPLEXITY nodes. However, Germany and Japan had significantly more lessons containing MOTIVATION nodes than did the United States. And Japan had the largest percentage of lessons containing DEDUCTIVE REASONING nodes, while the United States had the smallest percentage.

Some indication of how content was developed over the course of each lesson was gained by coding the kinds of links that connected parts of the lesson together (see figure 30). Increasing complexity between two nodes was coded more often in Japanese and German lessons than U.S. lessons. Japanese lessons contained significantly more links coded as necessary than did U.S. lessons.

An overall summary of the Math Content Group's coding can be obtained by adding up the total number of positive characteristics coded for each lesson. Thus, for each directed graph, we simply added up the number of codes that were attached to nodes (i.e., Motivation + Illustration + Deductive Reasoning + Inductive Reasoning + Increase in Complexity), and the number of codes that were attached to links (i.e., Necessary + Helpful + Similar + Deductive/Inductive Reasoning + Increase in Complexity). Figure 31 shows the average number of codes per node and per link for lessons in the three countries. Japanese lessons contained significantly more codes per node than either German or U.S. lessons; and U.S. lessons contained significantly fewer codes per link than lessons in the other two countries.

0.4 0.3 0.2 0.1

Additional Coding of Tasks The Math Content Group developed two more codes to indicate the kinds of tasks which were engaged in during the lesson.

The first of these additional codes was Task Complexity. Each task was categorized as either singlestep or multi-step. Lessons were then categorized as containing mostly single-step tasks, equal number of single- and multi-step tasks, or mostly multi-step tasks. The results are shown in figure 32. None of the pairwise differences was significant.' z On a three-point scale where 1 indicates "More single-step" and 3 indicates "More multi-step," the averages (with standard errors) for German, Japanese, and U.S. lessons were 2.3 (0.17), 2.7 (0.11), and 2.2 (0.19), respectively.

The second coding of tasks was for what the Math Content Group called LOCUS OF CONTROL.

What level of choice did students have in determining how to perform the task? Were they all controlled by the task, or was some of the control left up to the student? For example, if the teacher had just demonstrated how to solve a problem, then asked the students to try applying the same method to a similar problem, it was coded as TASK CONTROLLED. This is because the students were not asked to make any decisions about how to approach the problem, only to follow the exact procedure demonstrated by the teacher. On the other hand, if the teacher asked students to see if they could think of another method for solving a problem it was coded as SOLVER CONTROLLED, because the student had the freedom to decide which of several possible approaches they would take. In figure 33 we show the percentage of lessons that contained all Task Controlled tasks, all Solver Controlled, or a mixture of the two. Seventeen and 48 percent of Japanese and German lessons contained all task controlled tasks, respectively, while the share was 83 percent for U.S. lessons.

Global Ratings of Quality In addition to constructing the directed graph representation of each lesson and coding the nodes and links, the members of the Math Content Group also assigned a rating to each lesson that reflected their overall judgment of the quality of the mathematical content in the lesson. They rated each lesson on a three-point scale: low, medium, or high. (Where raters disagreed on this judgment, the disagreement was resolved by discussion so that, in the end, all four raters agreed.) Although the measure is subjective, there was high agreement among the independent raters. A summary of these ratings is presented in figure 34. The distributions of ratings differed significantly between the United States and the other two countries; German and Japanese lessons were rated higher in quality than U.S. lessons.' Twentyeight and 39 percent of German and Japanese lessons, respectively, received the highest rating; none of the U.S. lessons did. Eighty-nine percent of U.S. lessons received the lowest rating.

On a three-point scale where 1 indicates "low" and 3, "high" quality, estimates (and standard errors) for German, Japanese, and U.S. lessons were 1.9 (0.14), 2.3 (0.12), and 1.1 (0.06), respectively.

NOTE: The numbers in this graph differ slightly from those reported in Peak (1996, page 45) where unweighted averages were mistakenly used instead of weighted averages. Specifically, the percentages of lessons rated as low, medium, and high quality were reported as 40, 37, and 23 for Germany; 13, 57, and 30 for Japan; and 87, 13, and 0 for the United States, respectively. The numbers shown in this graph are the correct ones. Percentages may not sum to 100 due to rounding.