«Title: STUDY ABROAD AS A PASSPORT TO STUDENT LEARNING: DOES THE DURATION OF THE STUDY ABROAD PROGRAM MATTER? Jill M. Neppel, Master of Arts, 2005 ...»
Question 10 of the 2004 NSLLP Residence Environment Survey exists for the purpose of measuring intellectual growth, which includes growth in cognitive complexity, growth in liberal learning and growth in personal philosophy. The original wording of the question was: “In thinking about how you have changed during college, to what extent do you feel you have grown in the following areas?” Students were asked to respond to multiple statements using the following choices: 1) not grown at all, 2) grown somewhat, 3) grown, or 4) grown very much. For the purposes of my study, the wording of the question was changed to: “To what extent do you think that your study abroad experience contributed to your growth in the following areas?” This question appeared in the Study Abroad Survey as question 12. A sample of the individual statements to which students were asked to respond include the following (for the complete list of questions,
please see Appendix B):
Developing your own values and ethical standards • Understanding yourself and your abilities, interests, and personality • Ability to put ideas together and to see relationships between ideas • Appreciation of racial/ethnic differences • Ability to critically analyze ideas and information • Appreciation of art, music, and drama • Openness to views that you oppose • The other substantive question that was retained (question 11) measured selfconfidence (i.e., academic self-confidence and interpersonal self-confidence). For the purposes of my study, only the items that inquired about interpersonal self-confidence were used. The question appeared as question 13 in the Study Abroad Survey. The original wording, “Now that you have been in college for a while, how confident do you feel in the following areas?,” was changed to “As a result of your study abroad experience, how confident do you feel in the following areas?” The response choices included: 1) not at all confident, 2) somewhat confident, 3) confident, and 4) very confident. The statements to which students were asked to respond included: (listed
again in Appendix B):
Expressing ideas orally •
Additional questions were created specifically for this study so that ancillary analyses could be conducted. Respondents were asked to indicate their gender, race/ethnicity, academic college, and current class level. To get an estimate of a respondent’s socioeconomic status and cultural capital, a question was asked about parents’ incomes. Additionally, questions also asked respondents to indicate where and when they studied abroad, the length of time they were abroad, the number of previous trips they had taken abroad before participating in their study abroad program, their satisfaction with their time abroad, and their interest or plans to travel, work, live and/or study abroad in the future. These background questions were included in the study so that it would be possible to illustrate the types of Maryland students who study abroad and the corresponding programs they choose, by duration and location.
This research study employed the statistical method of analysis of covariance (ANCOVA), which allowed two of the variables, gender and academic class standing, to be built into the research design as covariates. These variables were chosen as covariates because of their potential to confound the results of the study, in terms of the effect of program length on cognitive complexity, liberal learning, personal philosophy and interpersonal self-confidence. As suggested by the previous chapter, the research on learning outcomes asserts that male students evidence greater interpersonal self-esteem throughout college than their female counterparts (Astin, 1989), making it necessary to mitigate this phenomenon. Additionally, the research reviewed for this study infers that the effects of class level should be statistically controlled for, since seniors manifest greater levels of cognitive complexity than younger students (Astin; Pascarella & Terenzini, 1996). It is generally assumed that upper-level students are more experienced, mature and educated than lower-level students. Therefore, because all class levels were represented in this research study (in addition to recent graduates), a deliberate attempt was made to statistically control for the many potentially confounding effects of class standing.
Validity and reliability. Due to the fact that the survey instrument was altered a great deal from its original state, it was necessary to re-establish validity and reliability.
Dr. Michael Ulrich, the Director of the University of Maryland’s Study Abroad Office was asked to examine the survey instrument, in order to establish face validity.
Furthermore, the survey was pilot tested with a group of six University of Maryland students, all of whom had previously studied abroad. Their feedback was solicited and subsequently used to revise and improve the survey. In addition, to test for internal consistency, Cronbach alpha coefficients were generated using the final obtained sample for each of the four individual construct scales (i.e., cognitive complexity, liberal learning, personal philosophy, and interpersonal self-confidence). These values will be reported in the next chapter.
Data Collection Procedure
The subjects. The survey was administered to all subjects of the sampling frame:
all University of Maryland students who participated in at least one of the following study abroad programs: fall 2003, winter 2004, spring 2004, academic year 2003-2004, summer 2004, fall 2004, and winter 2005. It was possible to make a fair number of assumptions about this sampling frame, because of the fact that students must meet a number of established requirements specific to their chosen study abroad programs in order to participate in an education abroad experience. In general, most programs require a minimum grade point average, which usually ranges from a 2.5 to a 3.2 minimum.
Furthermore, many programs require students to write personal statements describing their motivations and goals for studying abroad. At the University of Maryland, class level (first year students are only allowed to participate in winter-term programs), the quality of students’ letters of recommendation, and involvement in extracurricular activities are all additional factors that have the potential to affect an individual student’s candidacy.
Administering the survey. The study abroad survey was created with Survey Monkey, a privately held survey software program. The survey was hosted on the world wide web. Email addresses of members of the sampling frame were procured from the Study Abroad Office’s database of past participants, so that a link to the survey could be sent out via email. In total, three email messages were sent to members of the sampling frame. These messages explained the purpose of the study and asked individuals for their participation. They were also used to advertise the research incentives being offered, in order to encourage greater participation in the study. Four randomly chosen participants were selected to receive $50 Target gift cards.
The initial message was sent on March 7, 2005. Immediately upon accessing the web survey, participants were notified of their rights and then asked whether they were eighteen years of age and wished to participate in the study. If they elected not to continue, they were thanked for their time. Participants who responded in the affirmative, thus providing their informed consent, were then linked to the first page of the survey. A total of 232 individuals responded to this first message. On March 11, 2005, a second message was sent to all members of the sample population who did not participate in the study as a result of the first message. An additional 137 individuals responded at this time. A final attempt to obtain more responses was made on March 16, 2005, nine days after the initial message was sent out. This resulted in 102 extra responses. When the data collection period ended on March 22, 2005, a total of 471 individuals had completed the survey.
Data Analysis Scale creation. The composite scales for the four learning outcomes were borrowed directly from the National Study of Living Learning Programs. As stated earlier, the scales were re-tested for reliability, in order to ensure that the Cronbach alpha values were sufficiently high using the data collected for this study. The scores for individual items (e.g., leadership ability, expressing ideas orally, and working as part of a team) were summed to form composite scales (e.g., interpersonal self-confidence).
(See Table 3 for a full list of the composite scales used in this study.) Table 3: Composite Measures
Inferential statistics. Inferential statistics were used to analyze the survey results pertaining to the research hypotheses. Specifically, the statistical procedure of ANCOVA allowed the comparison of groups (long-term vs. short-term) on the following outcomes: cognitive complexity, liberal learning, personal philosophy, and interpersonal self-confidence, all of which were operationalized as dependent variables.
The independent variable in this study was duration of study abroad program. It was hoped that enough responses would be obtained to permit winter-term programs, summer programs, semester programs and academic year-long programs to be treated as four distinct levels of the independent variable. However, because the number of responses for each category was so unbalanced, (in addition to the fact that only twentythree respondents had studied abroad for an entire academic year), it was necessary to collapse the data into two levels: short-term programs (i.e., winter-term and summer programs) and long-term programs (i.e., semester and academic year-long programs). As mentioned earlier, gender and academic class standing served as covariates in the ANCOVA analysis. Because the data was collapsed into two levels of the independent variable, it was not necessary to conduct any post hoc tests to further investigate the nature of any significant differences.
Descriptive statistics. The main focus of this study was to answer the research question, which necessitated that inferential statistics be used. However, ancillary descriptive analyses were also used to describe the demographic and background characteristics (gender, race/ethnicity, socioeconomic status, class level, and previous international travel experience) of survey respondents. This information provides insight into the types of students who choose to participate in both short and long-term study abroad programs at the University of Maryland.
This chapter has described the research methods that were implemented in this quantitative research study of study abroad outcomes. The next chapter will report the results of the data collection and analysis efforts. The final chapter will discuss implications of the research study.
This chapter utilizes descriptive statistics to define the participants involved in the research study, as well as the types of study abroad experiences represented by the respondents. Results of analyses pertaining to the research hypotheses are then reported in the form of inferential statistics. An interpretation of the results of the data analysis is discussed in the subsequent chapter.
Description of the Sample As indicated in the previous chapter, the Director of the Study Abroad Office at the University of Maryland originally estimated that 1,215 students had studied abroad during the time period relevant for this study. In reality, the actual sample size of the study was 1,408 individuals. Forty-five of the email addresses procured from the Study Abroad Office were invalid, making the revised sample size 1,363 participants. A total of 471 participants responded to the survey, equating to a response rate of 34.6%. In addition to the questions at the heart of this study, participants were also asked to respond to a number of demographic questions, the results of which will be presented here.
Table 4 shows the number of respondents who studied abroad on programs of varying lengths. These numbers are compared to the original sample suggested by the Study Abroad Office. Slightly less than half of the survey respondents (44.1%) participated in programs that have been defined by this study as long-term (i.e., programs lasting for an academic year or semester); 52.8% of the total respondents participated in short-term programs (summer or winter programs). The remaining 3.1% of respondents chose the option of “other” to describe the length of their study abroad program. As suggested by Table 4, all program types were represented in this study. Winter-term participants responded at a higher rate (44.7%) than the other program types, however.
Semester participants were slightly under-represented in comparison to the other program types (31.1%).
Table 4: Participation in Study Abroad Programs of Varying Lengths: Survey Respondents’ Participation versus the Study Abroad Office’s Estimation
Of the total respondents, 444 individuals indicated their gender, while 27 chose not to respond to this question. Of those who responded, 329 respondents were female (74.1%), while 115 respondents were male (25.9%). In terms of race/ethnicity, 445 participants responded to the question and 26 skipped the question. Among the respondents, 318 (71.5%) identified as White/Caucasian, 54 (12.1%) identified as Asian or Pacific Islander, 28 (6.3%) identified as African-American/Black, 23 (5.2%) identified as Multi-racial/Multi-ethnic, 12 (2.7%) identified as Hispanic/Latino, and 8 (1.8%) identified as Other. Two respondents (0.4%) indicated that their race/ethnicity was not included in the given response choices, and none of the respondents identified as American Indian or Alaskan Native. Table 5 compares the population of survey respondents (in terms of race/ethnicity and gender) to the profile of the 2002/2003 U.S.
American study abroad participant population using data presented in the Open Doors Report (IIE, 2004). This comparison shows there to be greater racial/ethnic diversity among the sample of UM survey respondents. At the same time, however, the table also shows there to be a larger gender disparity among the sample of UM survey respondents as compared to the national population of study abroad participants.