«Title: STUDY ABROAD AS A PASSPORT TO STUDENT LEARNING: DOES THE DURATION OF THE STUDY ABROAD PROGRAM MATTER? Jill M. Neppel, Master of Arts, 2005 ...»
Reliability. Next, the learning outcome constructs were tested for internal consistency using Cronbach alpha coefficients. In keeping with the 2004 NSLLP Residence Environment Survey, the original learning outcome constructs were retained (See Appendix C). As indicated by Table 24, reliability was demonstrated for each construct; Cronbach alpha values for the constructs used in this study ranged from.744 to.827.
Table 24: Reliability of Measured Constructs
Once reliability was demonstrated, it was possible to create the learning outcome scales for cognitive complexity, liberal learning, personal philosophy and interpersonal self-confidence. Frequencies were then run for each of the scales to ensure consistency and accuracy.
Testing of ANCOVA assumptions. A number of assumptions associated with the statistical procedure of ANCOVA were tested for each hypothesis, including the appropriate measurement of covariates, reliability of covariates, correlations among covariates, and homogeneity of regression slopes (Pallant, 2001).
The first assumption (measurement of covariates) required that the covariates be measured as a part of the research design. In accordance with this assumption, data was collected about respondents’ gender and academic class standing. These values were not manipulated as a result of treatment or statistical analysis. The second assumption (reliability of the covariates) presumes that respondents answered questions about their gender and academic class standing accurately.
Another assumption that had to be tested (correlation among the covariates) required the researcher to verify that the covariates were not too strongly correlated with one another. Using SPSS, a test was conducted to determine the amount of correlation between the variables of gender and academic class standing. The resulting Pearson product-moment correlation coefficient was.095, which is a small/negligible correlation, and well below the cut-off value of.8 (Pallant, 2001). Therefore, this assumption was met.
The final assumption that had to be tested concerned homogeneity of regression slopes. This required multiple tests to be conducted for the purpose of determining whether or not there was evidence of statistically significant interactions between the various dependent variables and the individual covariates. Individual analyses revealed that this assumption was not violated. The relationship between the gender covariate and the dependent variables, cognitive complexity, liberal learning, personal philosophy, and interpersonal self-confidence, did not differ significantly as a function of the independent variable, length of program. Similarly, the relationship between the academic class standing covariate and the four dependent variables did not differ significantly as a function of the independent variable.
Analysis of hypotheses. A one-way analysis of covariance (ANCOVA) was conducted for each of the hypotheses associated with this study. The independent variable (length of study abroad program) included two levels (long-term and shortterm). The dependent variables included: growth in cognitive complexity, liberal learning, personal philosophy, and interpersonal self-confidence. Gender and academic class standing of respondents were used as covariates. A separate ANCOVA was conducted to test each of the four hypotheses. A Levene’s Test of Equality of Error Variances was run as a part of each ANCOVA to confirm the null hypothesis that the error variance of the dependent variable was equal across groups. The ANCOVA associated with each hypothesis was statistically significant. The results are even further encouraging, because the mean values for both short-term and long-term respondents are higher than the norm values provided by the NSLLP (Inkelas, 2004). Furthermore, the standard deviation scores are much lower for this research study than for the NSLLP, which suggests greater consistency in response choice. Table 25 provides comparative data for the NSLLP.
Table 25: Descriptive Statistics for the NSLLP
*Inkelas, 2005 [Raw Data]
Results of the first ANCOVA allowed the following null hypothesis to be rejected:
There will be no differences in the amount of growth in cognitive complexity depending on the length of the study abroad program. The ANCOVA revealed significance, F = 8.073, p =.005, ŋ2 =.019 (small effect size). Long-term study abroad participants evidenced a larger amount of growth in cognitive complexity than short-term study abroad participants, as demonstrated by the respective means of 12.037 (long-term) and 11.323 (short-term). Possible composite score values ranged from 4.000 to 16.000, with a score of 4.000 representing “Not grown at all,” a score of 8.000 representing “Grown somewhat,” a score of 12.000 representing “Grown,” and a score of 16.000 representing “Grown very much.” The low standard deviation scores for both short-term and longterm respondents indicates that there was not much variation in how respondents chose to answer this question. (See Table 26) Table 26: ANCOVA of Differences in Amount of Growth in Cognitive Complexity
*Composite score values ranged from 4.000 to 16.000.
The second hypothesis was also rejected: There will be no differences in the amount of growth in liberal learning depending on the length of the study abroad program. Again, the ANCOVA revealed significance, F = 8.318, p =.004, ŋ2 =.020 (small effect size). Respondents who had participated in long-term study abroad programs manifested a significantly larger amount of growth in liberal learning (mean = 17.886) compared to their short-term study abroad counterparts (mean = 16.790). For this particular composite scale, the minimum value was 6.000 and the maximum value was 24.000. A mean score of 6.000 corresponds to “Not grown at all.” A score of 12.000 corresponds to the response choice “Grown somewhat;” a score of 18.000 corresponds to “Grown” and a score of 24.000 corresponds to “Grown very much.” Again, the low standard deviation scores show the relative lack of variation in response choices among both short-term and long-term respondents. (See Table 27) Table 27: ANCOVA of Differences in Amount of Growth in Liberal Learning by
*Composite score values ranged from 6.000 to 24.000.
The third hypothesis: There will be no differences in the amount of growth in personal philosophy depending on the length of the study abroad program, was also rejected. Accordingly, the results of the ANCOVA revealed significance: F = 24.155, p=.000, ŋ2 =.054 (moderate effect size). Table 28 shows that the growth in personal philosophy experienced by long-term study abroad participants was significantly higher than the growth experienced by short-term study abroad participants, as illustrated by the respective mean values of 13.309 (long-term) versus 12.249 (short-term). Potential values for this composite scale ranged from 4.000 to 16.000. As with growth in cognitive complexity, mean scores of 4.000, 8.000, 12.000, and 16.000 corresponded to the following response choices respectively: “Not grown at all,” “Grown somewhat,” “Grown,” and “Grown very much.” The standard deviation scores of.145 (short-term respondents) and.158 (long-term respondents) shows that there was even less variation in terms of how this question was answered than with the previous two questions.
Table 28: ANCOVA of Differences in Amount of Growth in Personal Philosophy
*Composite score values ranged from 4.000 to 16.000.
The final hypothesis to be tested: There will be no differences in the amount of growth in interpersonal self-confidence depending on the length of the study abroad program, was also rejected. The mean amount of growth reported by long-term participants was 9.668; in comparison the mean amount of growth reported by shortterm participants was 9.242. The range of possible composite scale values started at 3.0 (“Not at all confident”) and ended at 12.0 (“Very confident”), meaning that both shortterm and long-term respondents rated themselves as slightly higher than “Confident” as a result of their study abroad experiences. Results of the ANCOVA revealed significance: F = 5.331, p =.013, ŋ2 =.013 (small effect size). (See Table 29) Table 29: ANCOVA of Differences in Amount of Growth in Interpersonal Self
*Composite score values ranged from 3.000 to 9.000.
Ancillary Analyses Additional information was collected in order to assist the Study Abroad Office to understand participants’ feelings and attitudes toward international education, perceived barriers to studying abroad, and future interest in study abroad. As shown in Table 30, participants were asked to indicate the extent to which they agreed with two statements, both of which were designed to elicit respondents’ attitudes toward the respective lengths of their study abroad programs. A chi-square analysis revealed that short-term and longterm participants tended to choose each response choice at nearly the same frequency.
Overall, 92% of all respondents disagreed with the notion that they would have liked to have participated in a shorter study abroad program, and 69% directly affirmed that they wished for a longer study abroad program.
Table 30: Survey Respondents’ Feelings toward the Length of their Study Abroad Programs
Chi-square: χ2=3.928; df=4; p=.416 Overall satisfaction with the study abroad experience is buttressed by the following survey results: 98% asserted that they were happy with their choice of study abroad location and 99% agreed that studying abroad was a great experience.
Furthermore, it is likely that many survey respondents will become advocates for studying abroad. Ninety-seven percent agreed with the notion that more students should study abroad, and ninety-four percent indicated that they would actively encourage more students to do so. Finally, 77% indicated that they would like to study abroad again, and 99% indicated that they hoped to engage in more international travel in the future. (See Table 31) Table 32 provides more insight into respondents’ future study abroad and international travel aspirations.
Table 31: Participants’ Feelings toward their Study Abroad Experiences
Finally, respondents were also asked to indicate which factors they personally considered to be barriers to studying abroad. As shown in Table 33, money was the most oft cited concern, followed by academic major, leaving friends, and leaving family.
Respondents were allowed to choose as many barriers as they thought pertained. They could also choose the option of “other” and specify the barrier they had in mind. These written-in responses ranged from “none” or “no fear,” to academic restrictions (i.e., the 30-credit rule), fear of not being able to graduate on time, leaving a job, and leaving a significant other.
Table 33: Barriers to Studying Abroad
This chapter has reviewed the results of the data analysis procedures as they relate to the original research hypotheses and ancillary analyses. The next chapter will provide further interpretation of the results, including implications for the field and suggestions for future research. Limitations of the study will also be discussed.
This final chapter provides a summary and interpretation of the results of the research study. Limitations of the study and its generalizability are discussed, in addition to suggestions for future research. The study’s results are also discussed as they relate to practical implications for the field.
Summary of Findings To review, the purpose of this study was to compare a group of short-term study abroad participants to a group of long-term study abroad participants in terms of the amount of growth each group perceived on four purposefully chosen learning outcomes.
The study hypothesized that there would be differences in students’ perceived amount of growth in cognitive complexity, liberal learning, personal philosophy and interpersonal self-confidence in relation to the length of the study abroad program. A locally modified survey instrument, which was re-tested for reliability and validity, was distributed to a sample population of 1,363 individuals. Four hundred seventy-one individuals participated in the study, for a response rate of 34.6%. Data analysis, which included four separate ANCOVA tests, revealed that all hypotheses were significant, with the longterm study abroad group evidencing larger perceived amounts of growth on all variables of interest (i.e., cognitive complexity, liberal learning, personal philosophy, and interpersonal self-confidence). Although significant, the effect size was small for three of the four variables; however, the effect size associated with growth in personal philosophy was moderate (ŋ2=.054). The effects of the chosen covariates, gender and academic class standing, were controlled for statistically. Ancillary analyses yielded further information about the participants involved in the study, including demographic and programmatic information.
Interpretation of Findings The ANCOVA results of this study suggest that there are differences in the perceived amount of growth in cognitive complexity, liberal learning, personal philosophy, and interpersonal self-confidence in relation to the length of the study abroad program. As such, this study builds on the small body of research that examines the impact of the length of time abroad on the attainment of learning outcomes. The results of the cross-tabulations and ancillary analyses also provide further insight into short-term and long-term study abroad participants and their respective experiences.