Research Design and Ethics

SoTL Research Designs

Once you have determined your research question, the next step is to decide on a research design. 

As random assignment tends to be challenging in SoTL, most projects tend to fall into the non-experimental or quasi-experimental research designs. See the figure below for some comparisons between different research design types.

Below is a decision tree that can help you identify which research design best fits your SoTL project. Consider the type of data you are interested in collecting (qualitative/quantitative), whether or not you have a specific intervention (e.g., introducing a new group project, trying a different course modality, or an alternative grading system), and then whether or not you will be making any comparisons between different groups.

(Rout & Aldous, 2016 )

Here we have attached a Research Design Worksheet that can help you in this process.

Ethical Considerations in SoTL Projects

It is important to recognize that there are ethical considerations to address when conducting research with your students. As students are often a vulnerable population, we need to consider how we can acknowledge and respect that these projects are based on the participation, engagement and investment of students. Here are several specific issues that you should consider prior to beginning any project.

Fairness to Students

To many, the “ideal” SoTL project would involve two sections of the same course, taught during the same semester by the same instructor in which students are randomly assigned and one section is considered a “control” and the other an “intervention” section. This design is difficult to accomplish in the real world but also needs to be thoughtfully considered. As instructors, we regularly try out new activities, assessments or pedagogies in our classes and we don’t usually know ahead of time how they will ultimately help or harm student learning. Therefore, oftentimes there are no issues using an experimental or quasi-experimental design. However, if you have evidence that an intervention will significantly benefit students, you may need to use a different design, such as a counterbalanced design, where half the students receive the intervention at the beginning of the semester and the other half of the students receive the intervention at the end of the semester.

Additionally, be mindful of who is being included in your SoTL project to ensure that students from marginalized groups have equal opportunities to benefit from participating in your study. Do not exclude any groups of participants unless you have a specific reason related to your research question. For instance, if there are non-native-English speakers in your course, consider finding a collaborator who speaks multiple languages to help with communications related to your study.

Equity-Mindedness in SoTL Projects

When designing SoTL projects, we need to consider our own assumptions about who our students are, how they learn and what might improve their learning. These assumptions are often rooted in hierarchical, colonial, racialized and gendered norms and we need to be careful when introducing pedagogical practices that might benefit one group of students over another. For instance, many instructors use humor to connect with students on an interpersonal level; however, including humor in exam questions or assignment instructions may inadvertently disadvantage international students who are unfamiliar with humorous terms or references. While it may be difficult to anticipate these types of challenges in advance, it is important for instructors to be attentive to the cultural contexts of class, race, gender, disability, and sexuality in the ways in which students respond to instructor interventions. Considering ways to create a culturally responsive curriculum and adopting an actively asset-framing approach to working with students and student work can help avoid some of these challenges (and could be interesting avenues for SoTL projects!)

Issues of Coercion 

If you are conducting a SoTL project using students in your class, you must be careful to avoid any research procedures that might lead to coercion. As the instructor who is in charge of students’ grades, you are in a position of power and need to make it clear that participation in any research is entirely voluntary. For instance, if you offer any extra credit to participate in a research survey, you should offer an alternative extra-credit assignment for students who do not wish to participate in the study. Additionally, you should set up your study so that you are not aware of which students are participating in the research, either by collecting data anonymously or including a research collaborator (who is not involved in the course) to collect, combine and de-identify any data prior to your analysis.

Student Privacy

You must design your study procedures to keep all student data private. This can be done by only collecting anonymous data or by de-identifying data so that they could not be traced back to any particular student through removing names, birthdates, PUIDs, etc. If you have a small course, you should consider whether you need to remove additional data such as gender or ethnicity to avoid being able to identify students from marginalized groups.