Faculty- & Staff-Led Study Abroad

Study Abroad Assessment Initiative

Purdue University - West Lafayette has a new mandatory assessment process for departmental faculty- and staff-led study abroad programs, The Study Abroad Assessment Initiative.

Program Leaders:

All current and future Study Abroad Program Proposals must include submission of the Globally Engaged Learning (GEL) Index survey. The GEL must be completed by the faculty/staff member leading the student program. At the program level this assessment protocol is mandatory. CILMAR will, however, provide consultation on the use of additional assessments (e.g., IDI, ASKS2, CQS, IES, etc). Cost-sharing funding may be available. Contact CILMAR at cilmar@purdue.edu for more information.

Students:

All students participating in short-term study abroad programs will complete the Beliefs, Events, and Values Inventory (BEVI) as a pre and post-test. Students may individually opt to complete the alternative qualitative reflective writing survey in place of the BEVI. Instructions for both options are provided to students on their individual My Study Abroad page after program acceptance.

 

Program Assessment Process

  1. Click on the GEL Index survey link embedded in the Study Abroad Program Proposal online form. Complete the GEL index survey* and upload the Survey Summary to the Proposal site.
  2. Program leaders must add completion of the pre- and post-program assessment (BEVI or alternative writing survey) to their program syllabus. Completion points can then be awarded as part of the program grades.
  3. Program leaders should contact CILMAR (CILMAR@purdue.edu) to specify departure and return dates for each program. Access to the BEVI will be set up based on these dates. CILMAR will receive student enrollment information from the Study Abroad Office and set up access to the BEVI assessment for your program's students. Program leaders will then need to verify completion of BEVI submissions via their Program Leader Portal for both the pre and post-tests.

Once initiated, program leaders will recieve the following support from CILMAR in facilitating the Assessment Initiative:

  • Program leaders will receive anonymized program-level pre-test BEVI data prior to their upcoming programs.
  • Program leaders will receive post-test data before they submit a proposal to renew a program.
  • College administrators will receive aggregated college-level data
  • CILMAR and IDA+A will analyze aggregate data at the institutional level.
*Downloadable PDF copy of GEL Index Survey

Study Abroad Assessment Initiative Overview

This plan pairs two instruments that are theoretically aligned and have been successfully implemented together at other institutions—one for program leaders and one for students. The BEVI is a well-researched whole-person transformative learning measure with strong demonstrated validity and reliability that offers many advantages, including:

  • formative assessment potential via automated individual narrative reports returned directly to students and aggregate reports delivered to faculty before departure,
  • cost effectiveness via institutional licensing, and
  • a wide array of measured constructs that meet needs in diverse disciplines.
The BEVI has also already been paired successfully with the GEL Index, suggesting that triangulating GEL and BEVI data will offer us insights into the relationship between design elements and outcomes.

Program Leaders:

Global Engaged Learning (GEL) Index: During the program proposal process, program leaders will complete the (GEL) Index Survey, an instrument developed by CoreCollaborative International in a project funded by the Colonial Academic Alliance to measure high-impact practices in global learning experiences. The survey asks program leaders to reflect and report on design elements of their programs that have been demonstrated in research to engage students and enhance their learning outcomes in all three areas (disciplinary, location-specific, and intercultural/global learning). One benefit is that the GEL is broader than the current intercultural learning plan process that focuses only on the third area. Another is that it streamlines the process for program leaders by eliminating data reporting after they return from abroad, instead asking program leaders renewing a program to reflect on data from its previous iteration during their planning process for the next offering. Finally, program leaders can partner with CILMAR if they would like to submit an exempt IRB proposal and publish on study abroad programs using existing institutional data.

Students:

The Beliefs, Events, and Values Inventory (BEVI): CILMAR will partner with the Study Abroad office to administer the BEVI as a pre- and post-test, relieving program leaders of this responsibility (although program leaders may also still collect and analyze any additional data, intercultural or otherwise, that will benefit their students, their program, or their own research).
Students will access the BEVI through their My Study Abroad page after acceptance to their respective programs. Note: Because not every student may feel comfortable taking the BEVI, any student may request to take an alternate qualitative assessment instead of the BEVI before and after their study abroad program. Student should communicate their desire to complete the alternative and jointly request this option by contacting CILMAR (cilmar@purdue.edu).


Assessment Rationale

There is a wide array of potential student developmental outcomes from faculty- and staff-led study abroad (FLSA), including disciplinary, destination-specific, and generalizable intercultural/global learning outcomes. The breadth and depth of these outcomes, in addition to the intensive nature of the experience, are why study abroad is considered a high-impact practice in higher education. Recent scholarship demonstrates, however, that study abroad is not automatically or necessarily a high-impact practice. 

Departmental short-term study abroad programs need to be designed intentionally to address the following challenges:

  1. Replicating coursework abroad exactly as it is implemented on campus will likely only accomplish disciplinary outcomes. 
  2. Focusing on exploration of the local context might support achievement of destination-specific objectives, but scholarship has noted that some ways of interacting with the local environment seem more impactful than others. 
  3. Research also suggests that mentor-guided reflection on experiences of cultural difference is one of the most effective ways for students to develop intercultural competence in the study abroad context.

Assessment Support

Purdue Global Partnerships and Programs supports departmental program leaders in thoughtfully building toward desired outcomes in these three areas by partnering to gather and reflect on data that can inform curriculum development. In order to offer high quality programs, educators need evidence of what students are taking away from their study abroad experiences, and how those outcomes are related to who students are and what program design decisions program leaders make. Program leaders' time is limited, and assessment processes can be logistically challenging. Purdue can institutionalize data collection for student outcomes in short-term departmental study abroad programs in such a way that:

  • Program leaders are not burdened with clunky processes of administering pre- and post-tests to their program participants.
  • Program leaders have access to the analysis of their program level data so that in the process of proposing the next iteration of their program they can consider this valuable information in their program design. This would not add to the current workload of completing an intercultural learning plan during program proposal and submitting a reflective report of data afterwards; it would instead replace those existing processes. 
  • Program leaders and students benefit from formative assessment results—program leaders, with group level reports to understand who their learners are and what perspectives they bring to program experiences; and students, with individual level reports to motivate developmental work and identify personalized learning goals.
  • Purdue collects a large dataset at the institutional level for big data analysis exploring the connections between program design and student outcomes. We can answer questions about which program elements (for example, the opportunities students have to dialogue with local peers or the extent to which they encounter difference or the ways in which reflection is embedded in the experience) are most impactful for outcomes such as critical thinking, tolerance for ambiguity, openness to cultural diversity, communication skills, adaptability, self-awareness, empathy, and more. 

The more we connect on a deeper level with people who are different from us, the more we expand our experiences of the world and how we want to impact it.

Tara Harvey, PhD

Updated 12/13/2024