Author
Maribeth Keitz Web Communications Manager

NAEP’s Technology and Engineering Literacy Assessment

PostedThursday, May 19, 2016 at 8:10 AM

For this week's blog I interviewed LinkEngineering Committee chair Cary Sneider, Associate Research Professor at Portland State University, and member of the National Assessment Governing Board.

Keitz: What is the National Assessment Governing Board?

Sneider: The National Assessment Governing Board is an independent, bipartisan organization that oversees the National Assessment of Educational Progress (NAEP), also known as The Nation’s Report Card. For the past 25 years NAEP has provided periodic reports to the public about our students’ achievements in mathematics, reading, writing, science, social studies. As of May 17 of this year, NAEP also reports on students’ technology and engineering literacy (TEL), rounding out all four of the STEM fields.

Keitz: How many students have taken the TEL assessment?

Sneider: In 2014, 21,500 eighth graders from 840 schools across the country were given the Technology and Engineering Literacy (TEL) NAEP assessment. Schools were selected to obtain a representative sample of our nation’s population in terms of a balance of males and females, different racial/ethnic groups, and students from towns and cities and suburban and rural environments.

Keitz: Is this a fill-in-the bubbles assessment?

Sneider: Absolutely not.  Although there are some individual questions, most relate to a scenario in which students need show their knowledge and skills by actually solving a problem online.

Keitz: What are the concepts and skills that were measured with this assessment?

Sneider: The assessment measures knowledge and skills in three areas: Design and Systems, Technology and Society, and Information and Communications Technology (ICT). For example:

In Design and Systems we determine the extent to which students can diagnose a problem by analyzing a system, and how well they can design a solution through the engineering design process.

In Technology and Society we measure students’ abilities to recognize that all technological decisions have effects on society and the environment; and that it is important to anticipate as many of these effects as possible so as to maximize benefits and minimize negative consequences.

In Information and Communications Technology the assessment measures students’ abilities to gather and evaluate information online, to properly credit intellectual property, and to use digital tools to communicate ideas to a variety of audiences.  

Keitz: Does this assessment add to the students’ “testing burden” that is already high?

Sneider: Because individual or school scores are not released, students do not have to take the entire test.  Our sample is large enough that we can determine the capabilities of boys and girls in different demographic groups by asking the students to spend just one hour taking a section of the test.  And we do not have to test the entire population.  In this case, as I said, we just tested 21,500 students.  If we extend the assessment to determine state scores the sample size would be over 100,000. Even then, however, we would not expect any of the students to spend more than one hour on it.

Keitz: What do we know now that we didn’t know before?

Sneider: We learned that students are more capable than some of us had anticipated.  Forty-three percent of eighth-grade students performed at or above the Proficient level on the 2014 TEL assessment, even though few students take engineering or technology courses.

Female eighth-grade students scored higher on TEL than their male peers. Higher-performing students were more likely to engage in technology and engineering activities in and out of school. There is much more information about the results of the TEL NAEP assessment on the website at: http://www.nationsreportcard.gov/tel_2014/.

However, the numerical results are just part of the picture.  This is the first time that a NAEP assessment has been entirely administered on a computer with scenario-based tasks.  Anyone can access the website and take one of the tasks themselves, and find out how it is scored.  Many of the students who have taken the assessment have told us that it’s “fun” to take the assessment because they are asked to solve challenging meaningful problems. Actually taking these assessments is the best way for someone to find out what TEL NAEP measures and why those core ideas and skills have value in everyday life as well as on the job.

Keitz: Thank you, Cary, for your insights.