Measuring Learning Outcomes in Higher Education: Motivation ...

The educational legislation known as No Child Left Behind was initially embraced by the education and the public policy sectors, but since its inception criticisms have grown as to such effectiveness of policies that advocate for standardized testing versus more traditional assessment methods. Nevertheless, standardized testing has grown to include higher education, spurred in part by a 2005 Department of Education commission report lamenting the lack of transparency in higher ed practices. In response, many U.S. colleges and universities have adopted standardized student outcomes assessments and partnered with organizations such as Voluntary System of Accountability to better meet accreditation requirements, evaluate teaching effectiveness and, on the national stage, provide the foundation for educational policy recommendations.

These tests may prove useful to higher education institutions, but observers say that their accuracy may be suspect, as the tests typically don?t effect a student?s grade and they are less inclined to try to do their best. Attempts to incentivize test takers with better course grades and/or financial incentives have proved successful in the past, but are not always possible to implement on a wide scale.

A 2012 study from the Educational Testing Service in Princeton, NJ, published in Educational Researcher, ?Measuring Learning Outcomes in Higher Education: Motivation Matters,? investigates the links between student motivation, motivational instruction and test format on standardized test scores. More than 750 participants representing a mix of race, gender and academic aptitude with at least one year of college experience were offered $50 to complete a 36-item multiple-choice test and essay question online, followed by a survey that measured student motivation. Students signed one version of three possible consent forms: 1) individual test results could be shared with faculty and potential employers (the ?personal? condition); 2) individual test results would be private, but scores would be averaged from all participants from that college and could be shared with faculty and employers (the ?institutional? condition); or, 3) test results were for research purposes only and would not be shared (the ?control? condition).

Key study findings included:

• ?Students do not exert their best effort in taking low-stakes outcomes assessments ? [C]onclusions about value-added learning changed dramatically depending on the test of choice and the motivation levels.? There was no difference in overall performance outcomes across the three participating higher educational institutions involved in the study:? a research institution, a master?s institution and a community college.
• Student motivation and performance were both consistently higher for the multiple choice component of the test. The researchers suggest that essay tests require more effort and motivation.
• Motivated sophomores performed as well as the typical senior regardless of condition: with respect to test performance, ?the motivational effect for sophomores was as large as 2 years of college education.?
• Students assigned to the ?personal? condition group performed on average 41% better than those in the control group; those assigned to the ?institutional? condition group performed on average 23% better than control group participants. Students in the ?personal? group also scored significantly higher on the personal essay component of the test.
• The researchers suggest that many factors may influence student motivation levels, and that while linking a bonus to student performance may encourage performance, there are ways to motivate students that do not incur financial costs.
• Seniors in the institutional condition did not excel at the essay-writing component of the test: ?it may take a stronger reason than caring about one?s institutional reputation for seniors to be serious about writing an essay.?

The researchers suggest that low student motivation on standardized tests has likely resulted in underperformance; this could influence an institution?s reputation for good or ill: ?Institutions doing a good job of motivating students could achieve significantly higher rankings than institutions doing a poor job of motivating students, even though their students may have comparable academic abilities.?

Past research has also found that motivation and test performance are linked. A 2007 study shows that ?some individuals do not try their best when no performance based incentives are provided, though these individuals are not the less able ones.?

?Tags: youth, higher education

By Margaret Weigel | January 2, 2013

Citation: Liu, Ou Lynda; Bridgemen, Brent; Adler, Rachel M. "Measuring Learning Outcomes in Higher Education: Motivation Matters." Educational Researcher, Vol. 41, No. 9, 352-362. doi: 10.3102/0013189X12459679.

Analysis assignments

Read the issue-related Edmondson.comarticle titled "Time for a $10,000 Bachelor's Degree in Oklahoma?"

1. What key insights from the news article and the study in this lesson should reporters be aware of as they cover these issues?

Read the full study titled ?Measuring Learning Outcomes in Higher Education: Motivation Matters.?

1. What are the study's key technical terms? Which ones need to be put into language a lay audience can understand?
2. Do the study?s authors put the research into context and show how they are advancing the state of knowledge about the subject? If so, what did the previous research indicate?
3. What is the study?s research method? If there are statistical results, how did the scholars arrive at them?
4. Evaluate the study's limitations. (For example, are there weaknesses in the study's data or research design?)
5. How could the findings be misreported or misinterpreted by a reporter? In other words, what are the difficulties in conveying the data accurately? Give an example of a faulty headline or story lead.

Newswriting and digital reporting assignments

1. Write a lead, headline or nut graph based on the study.
2. Spend 60 minutes exploring the issue by accessing sources of information other than the study. Write a lead (or headline or nut graph) based on the study but informed by the new information. Does the new information significantly change what one would write based on the study alone?
3. Compose two Twitter messages of 140 characters or fewer accurately conveying the study?s findings to a general audience. Make sure to use appropriate hashtags.
4. Choose several key quotations from the study and show how they would be set up and used in a brief blog post.
5. Map out the structure for a 60-second video segment about the study. What combination of study findings and visual aids could be used?
6. Find pictures and graphics that might run with a story about the study. If appropriate, also find two related videos to embed in an online posting. Be sure to evaluate the credibility and appropriateness of any materials you would aggregate and repurpose.

Class discussion questions

1. What is the study?s most important finding?
2. Would members of the public intuitively understand the study?s findings? If not, what would be the most effective way to relate them?
3. What kinds of knowledgeable sources you would interview to report the study in context?
4. How could the study be ?localized? and shown to have community implications?
5. How might the study be explained through the stories of representative individuals? What kinds of people might a reporter feature to make such a story about the study come alive?
6. What sorts of stories might be generated out of secondary information or ideas discussed in the study?

?

Source: http://journalistsresource.org/studies/society/education/measuring-learning-outcomes-higher-education-motivation-matters?utm_source=rss&utm_medium=rss&utm_campaign=measuring-learning-outcomes-higher-education-motivation-matters

evan mathis staff sgt. robert bales jason russell norfolk state st patrick s day parade duke invisible children