The Purdue Civics Knowledge Test

“I know of no safe depository of the ultimate powers of the society but the people themselves; and if we think them not enlightened enough to exercise their control with a wholesome discretion, the remedy is not to take it from them, but to inform their discretion.” [1] -Thomas Jefferson - 1820

Introduction and Background

As Jefferson’s quote implies, a constitutional republic, such as the United States of America, requires informed, effective, and responsible citizens. Indeed, it has been said that ‘democracy is not a machine that will go of itself” and therefore requires each generation of citizens to develop, and employ, a certain level of civic knowledge in order to sustain it. Defining and developing such civic knowledge—that is, the effective preparation of citizens to fulfill their responsibilities to sustain and enhance self-government--is an essential condition for our representative government to survive.

Unfortunately, the civic literacy of Americans has been a concern for over 75 years.  For example, national surveys of college freshman in the 1940s revealed a “striking ignorance” of even the most basic civic knowledge (Fine, 1943, p. 1).”  In 2019, the Woodrow Wilson Foundation surveyed 41,000 Americans using 20 questions drawn from the USCIS Naturalization Panel.  The results “validated what studies have shown for a century: Americans don’t possess the history knowledge they need to be informed and engaged citizens…. (Wilson Foundation, n.p. 2019)”

Additional support for this trend comes from the Congressionally mandated National Assessment of Educational Progress (NAEP).  This assessment measures American K-12 students’ progress in a number of subjects, including civics, every four years.  Results of the most recent Grade 12 NAEP-Civics assessment indicated that only 24 percent of 12 th graders scored at the ‘proficient’ level or above and that 36 percent scored below even a ‘basic’ level of civic literacy.

Civics Knowledge

In order to assess student progress, sophisticated frameworks identifying and detailing key civics concepts have been created.  For example, the most recent NAEP-Civics assessment framework was developed by a panel of more than two dozen experts in civics and civic education. [2]   These authors of the NAEP-Civics Framework determined that essential civic knowledge consisted of answers to questions such as:

  • What are civic life, politics, and government?
  • What are the foundations of the American political system?
  • How does the government established by the Constitution embody the purposes, values, and principles of American democracy?
  • What is the relationship of the United States to other nations and to world affairs?
  • What are the roles of citizens in American democracy?

The National Standards for Civics and Government—developed by the Center for Civic Education (2003) [3] — provides another key reference point for determining key civics concepts.  The authors of this framework extended their document to stress that educational institutions have a responsibility “to prepare informed, rational, humane, and participating citizens committed to the values and principles of American constitutional democracy (page v).

The Purdue Civics Test

As an educational institution, Purdue University recognizes its responsibility to prepare informed citizens who value the principles of American democracy.  The creation of a civics literacy requirement is in-line with that responsibility. The development of the Purdue Civics Knowledge Test is part of the overall literacy plan.  The purpose of the test is  to measure undergraduate students’ mastery of the basic civics concepts and principles underlying the system of government in the United States.  This includes an awareness of their rights and responsibilities, as well as the source of those rights and responsibilities.  Students should also know the structure and function of the government as described in the U.S. Constitution, and the role citizens play in the political and social life of the country.  Finally, students should be familiar with the origin and contents of significant documents and events that have shaped the United States

“Citizenship—commitment to and participation in a community’s civic life—is the engine of constitutional democracy and a free society. Knowledge of the rights, responsibilities, and privileges of citizenship fuel that engine. Without the participation of informed, effective, and responsible citizens, a democratic republic cannot and does not function, nor can it make progress toward its ideals (NAEP Governing Board, p. 1, 2014).

Although it is hoped that civics knowledge will be related to active participation in civic life, the Purdue Civics Knowledge Test is focused directly on assessing student knowledge of key civics principles, facts, and government structures.  It does not try to assess student behavior or beliefs.

Developing the Test

The development of any standardized test should follow clear, well-documented procedures consistent with professional test development practices.  The first step is establishing a clear purpose for the test.  The purpose of the Purdue Civics Knowledge Test was outlined above.  The next steps include (1) identifying eligible content, (2) establishing detailed test specifications, (3) developing items aligned to the targeted content, (4) providing for expert review of test items for content accuracy and freedom from bias, (5) field-testing items, (6) completing the psychometric analysis of field-test data, and (7) creating final test forms. [4]   The development of the Purdue Civics Knowledge Test included attention to each of these steps.

Defining Eligible Content .  The test development team referenced multiple sources to identify eligible content.  The three most significant sources were (1) the National Standards for Civics and Government produced by the Center for Civic Education (2003), (2) The National Assessment of Educational Progress Framework: Civics (2018), and
(3) test items from the Naturalization Test for United States Citizenship managed by the United States Citizenship and Immigration Services (USCIS). [5] In addition, released items from the AP US History and AP US Government tests were also referenced.  As a result, the working group identified five broad constructs that defined eligible civic knowledge content for the test:

  • Civic life, politics and government,
  • Foundations of the American political system,
  • How the government established by the constitution embodies the purposes, values and principles of American democracy,
  • Roles of citizens in American democracy, and
  • Important individuals and events in the history of the United States  

Establishing Test Specifications .  Using these sources, and these constructs, a test blueprint was developed to define the range and balance of eligible test content within a civics knowledge test form.  The blueprint also set targets for the cognitive complexity of the final item pool and became a reference point for the development of assessment support materials including the development of study guides and practice tests. The final test blueprint identified five critical civic knowledge dimensions and the specific content standards to be assessed within each dimension.  The test blueprint is contained in List 1.

List 1. Test Blueprint

  1. Civic Life, Politics, and Government - 11%
    1. Distinguish between civic life and private life - 2%
    2. Identify ways people are represented in government - 3%
    3. Define limited government and rule of law - 3%
    4. Know the purpose of a constitution - 3%
  2. Foundations of the American Political System - 26%
    1. Identify the historical events and documents that led to the Constitution of the United States - 3%
    2. Identify distinctive characteristics of American society (e.g., absence of nobility, religious freedom, history of slavery, etc.) - 2%
    3. Identify distinctive characteristics of American federalism (i.e., overlapping powers at the local, state, and federal levels) - 5%
    4. Identify the different ways the Constitution can be amended and the types and number of amendments that have been added to the Constitution. - 4%
    5. Identify how the idea of natural rights is critical to the Constitution of the United States.
    6. Identify separated and shared powers. - 4%
    7. Recognize the values and principles of American constitutional democracy - 3%
  3. How the Government Established by the Constitution Embodies the Purposes, Values, and Principles of American Democracy - 30%
    1. Identify how the Constitution of the United States distributed and limited government powers among levels, branches, and agencies - 6%
    2. Identify the main branches of local, state and national government - 4%
    3. Recognize how checks and balances influence the making and enforcing of laws - 7%
    4. Identify examples of the rule of law 4%
    5. Identify the role of political parties in the United States - 4%
    6. Identify the role of the free press - 5%
  4. The Role of Citizens in American Democracy - 19%
    1. Identify constitutional amendments regarding citizenship and participation of citizens in politics (ex. 14th, 15th, 19th and 26th Amendment) - 5%
    2. Identify personal, political, and economic rights of citizens - 9%
    3. Identify ways that citizens take part in civic life - 5%
  5. Important Individuals and Events in the History of the United States  - 14%
    1. Recognize the contributions of diverse groups to the political and social development of the United States - 6%
    2. Identify the causes of the American Revolution - 4%
    3. Identify the causes of the Civil War - 4%

Item Development . A team of professional item writers developed well-written items clearly aligned to the targeted learning standard.  Items for the civics knowledge test were either adapted from existing sources (e.g., USCIS Naturalization test items, 12 th grade NAEP history and civics items, released civics items from large-scale state testing programs) or were new items written specifically for the Purdue Civics Knowledge Test.

Expert Review of Item Content and Bias and Sensitivity Review .  The pool of potential items was reviewed for both content accuracy and to ensure the items were free of potential bias and sensitivity issues. [6]   Items were reviewed by expert faculty, and items were edited or eliminated based on this review.  Over 175 new items passed the review process.

The Purdue University item review team included:

  • David Atkinson, Ph.D., Associate Professor, History Faculty
  • Kathryn Cramer-Brownell, Ph.D., Associate Professor, History Faculty
  • James McCann, Ph.D., Professor, Political Science Faculty
  • Christopher Munt, Ph.D., Director of Inclusive Excellence, College of Liberal Arts
  • Yvonne Pitts, Ph.D., Associate Professor, History Faculty
  • Phillip VanFossen, Ph.D., J. F. Ackerman Professor of Social Studies Education
  • Eric Waltenburg, Ph.D., Professor, Political Science Faculty
  • Peter Watkins, Ph.D. Visiting Professor, Political Science

The wordings of the test items were further subjected to a separate bias review by the Purdue University Division of Diversity and Inclusion

Simultaneous with these reviews and subsequent psychometric analysis, Purdue Libraries, under the direction of Professor Zoe Mayhook, created a resource page to provide background information for students preparing for the test.

Field-Testing the Items .  The test development team built four 50-item field test forms.  Each form included a core of 19 common items, and an additional set of 31 unique items.  This design allowed statistical analyses to place all items on a common underlying difficulty scale.  The forms were administered to samples of Purdue undergraduate students in March-April, 2021.

The test was administered through the Brightspace learning management system, with students self-pacing outside of a conventional classroom setting.

The students who took a version of the field test were recruited through a random sample of all undergraduates and two more specialized samples: undergraduates enrolled in political science courses in the Spring-21 term; and international students. In total, 355 students took part in this field test.

Psychometric Analysis of Field Test Data .  Item response data collected during the field test were used to establish the psychometric properties of the items and test forms. Each of the test items was analyzed to gauge its difficulty and functioning. The full battery of items was further assessed to confirm the high reliability of the instrument, and student performance was compared across the different forms and samples to explore the overall fairness of the test.

Establishing Final Test Forms .  The results of the psychometric analyses guided the development of three distinct, parallel, and equivalent 50-item test forms.  The evidence collected during the development process establishing the validity, reliability, and fairness of the final test forms will be summarized in a final technical manual.  

Based on the results of the field test, the decision was made to place the passing mark at 80%. 

[1] Letter to William Charles Jarvis, 28 September 1820; The Writings of Thomas Jefferson, Memorial Edition, Lipscomb and Bergh, eds., vol. 15 (278). More information.

[2] Civics Framework for the 2018 National Assessment of Educational Progress (2018).  More information.

[3] Center for Civic Education (2003). National Standards for Civics and Government. Calabasas, CA.  More information.

[4] American Educational Research Association., American Psychological Association., National Council on Measurement in Education., & Joint Committee on Standards for Educational and Psychological Testing (U.S.). (2014). Standards for educational and psychological testing.

[5] More information

[6] The development process and plans were also submitted for IRB review and approval.  This will allow data from the field test data to be presented outside of the immediate working group.

Purdue University, 610 Purdue Mall, West Lafayette, IN 47907, (765) 494-4600

© 2023-2024 Purdue University | An equal access/equal opportunity university | Copyright Complaints | Maintained by Office of the Provost

Trouble with this page? Disability-related accessibility issue? Please contact Office of The Provost at purdueprovost@purdue.edu.