The Rubric, the A.S.K.S2, the BEVI and the IDI

[The following information represents the opinion of Katherine Yngve, Assessment Specialist, Equitable & Intercultural Learning, Institutional Data Analytics + Assessment (IDA+A) and is not necessarily the opinion of the Center for Intercultrual Learning, Mentorship, Assessment and Research (CILMAR).]

The American Association of Colleges and Universities Intercultural Knowledge and Competence VALUE Rubric

6 reasons why this rubric is recommended:

  • It's a great formative assessment tool; e.g., it's transparent enough for students as well as for academics who don't specialize in this area of social science inquiry and pedagogy.
  • It's free and open-source.
  • It was designed by recognized subject matter experts, using a consensus-based process.
  • It helps Purdue to practice fair, equitable, and culturally responsive assessment.
  • It allows us to better assess reflective writing as well as other authentic artifacts of learning.
  • Calibration training is available to support scaling up use of the rubric to a disciplinary division, campus or campus-wide system; cross-institutional benchmarking is increasingly possible.

6 facets that the rubric measures:

The Intercultural Knowledge and Competence VALUE Rubric defines intercultural competence as "...a set of cognitive, affective, and behavioral skills and characteristics that support effective and appropriate interaction in a variety of cultural contexts" (Bennett, J. M., 2008). It measures six constructs, each at four levels of possible proficiency:

  • Intercultural attitude of curiosity
  • Intercultural attitude of openness
  • Skill of intercultural empathy
  • Skill of intercultural verbal and non-verbal communication
  • Knowledge of self-awareness
  • Knowledge of worldview frameworks

Validity and reliability information

For the research behind the rubric, please go here.

A.S.K.S2

5 reasons why the A.S.K.S2 are recommended:

  • It's a great formative assessment tool, due mainly to its high face validity.
  • It operationalizes the AAC&U Intercultural Knowledge and Competence VALUE Rubric using a Bloom's taxonomy approach and a Likert Scale type format.
  • It's free and comes in multiple versions, including one that allows for reduction of social desirability error using the retrospective pre-post technique.
  • An enhanced version, the The Intercultural Attitudes, Skills, and Knowledge Short Scale PLUS  (A.S.K.S2+), includes a set of goal-setting questions to activate the students' critical thinking skills on the practice of developing intercultural competence.
  • The A.S.K.S2 builds on the work of recognized subject matter experts through the experience and expertise of another award-winning instructor.

6 facets that the A.S.K.S2 measure:

The A.S.K.S2 measures the same six constructs as the AAC&U Intercultural Knowledge and Competence Rubric:

  • Intercultural attitude of curiosity
  • Intercultural attitude of openness
  • Skill of intercultural empathy
  • Skill of intercultural verbal and non-verbal communication
  • Knowledge of self-awareness
  • Knowledge of worldview frameworks

The Beliefs, Events, and Values Inventory (BEVI)

6 reasons why the BEVI is recommended:

  • The BEVI is a broad spectrum assessment instrument and the only one that was built on transformative learning theory. 
  • Site licenses are available.
  • The BEVI automatically sends a supportive feedback report to every individual test-taker.
  • The standard group report yields rich and sophisticated data on the similarities and differences, by deciles, in any given group of learners.
  • The BEVI has excellent psychometric properties, meaning that it's difficult for learners to manipulate their answers towards a socially desirable outcome.
  • The cost of administrator training is relatively low, in comparison to other intercultural competence instruments.

What the BEVI measures:

The BEVI is a 150-question instrument that captures both quantitative and qualitative data. Among the seventeen constructs it measures are:

  • Needs closure
  • Emotional attunement
  • Sociocultural openness
  • Impact of life events ("grit")
  • Tolerance of disequilibrium
  • Critical thinking
  • Global engagement
  • Self-awareness
  • Positive thinking

Drawbacks

Length of the instrument plus complexity of the group report can make it difficult to establish buy-in with some stakeholders.

Validity and reliability information

For the research behind the BEVI, please go here.

The Intercultural Development Inventory (IDI)

6 reasons the IDI is recommended:

  • For a proprietary instrument, the IDI is relatively short (50 items).
  • The IDI is available in seventeen languages, for greater inclusivity.
  • The IDI was built on Bennett's influential Developmental Model of Intercultural Sensitivity, and it is the only instrument which offers the facilitator tips on how to both support and challenge the learner to the next stage of competence.
  • The IDI is a psychometric survey, thus quite difficult for learners to manipulate their answers towards a socially desirable outcome.
  • The most recently conducted independent validation study found the IDI to be free from bias as regards gender, age, race, ethnicity and hierarchical positionality.
  • The IDI offers the facilitator an organizational power/privilege perspective for work on upgrading organizational levels of intercultural competency.

Note: The individual and group profile reports give both a perceived competency score and a performed score. According to Katherine Yngve, CILMAR's associate director for intercultural outcomes assessment, this can be seen as either a "bug" or a "feature."  The individual report comes with a personalized developmental plan for improvement, which when used well by a trained IDI facilitator can mitigate the shock of an identified perception/performance gap.

What the IDI measures:

The IDI measures an individual or a group's placement along the five-stage Intercultural Development Continuum. 

Drawbacks

  • Cost of administrator training and lack of a site license option can be prohibitive.
  • Requirement that the learner cannot receive individual results without meeting one-on-one with a trained administrator can be offputting.
  • The instrument frequently points out a gap between perceived and performed competency levels, which can be challenging to negotiate for both the survey-taker and the newly trained IDI qualified administrator.

Validity and reliability information

For the research behind the IDI, please go here.