Story

When Employers Demand Skills Professors Cannot Define

The data in the graph comes from Richard Arum and Josipa Roksa's 2014 book, 'Aspiring Adults Adrift: Tentative Transitions of College Graduates.' The book is a follow-on to their 2011 landmark study of undergraduates’ learning, socialization and study habits, 'Academically Adrift: Limited Learning on College Campuses.' Graphic by Ellen Wexler.

Dan Melzer’s students are used to following instructions — so he doesn’t give them any.

Melzer, a writing professor at California State University at Sacramento, wants to teach his students to think critically. All too often, he sees students who are prepared to absorb and regurgitate information in a structured and predictable format — like a short-answer exam — but when an assignment requires them to think for themselves, they struggle.

“Students are often used to being told exactly what to write and how to write it, and it’s almost like they’re just parroting what the instructor wants,” Melzer said. “Students are kind of used to being parrots.”

Instead of drafting detailed instructions, Melzer asks students to decide what kind of writing is meaningful to them. They analyze examples of the writing style they picked, and then try it out for themselves. Most of the time, these assignments leave Melzer’s students feeling lost.

“They always come to my office and say, ‘Why don’t you just tell us what to write and how to write it.' Or just, ‘Tell me exactly what you want.’ That’s a phrase I hear a lot from students.”

It’s poor instruction, he said, that leads to students’ sub-par critical thinking skills. In August, Melzer published a study of 2,101 writing assignments from 100 U.S. colleges and universities. While professors emphasized critical thinking in their research papers, Melzer was surprised by how many short-answer exams he found. And when professors did give out longer writing assignments, they often emphasized grammar and punctuation more than critical thinking and analysis.

“As I was collecting my research, I was getting depressed,” Melzer said. “It was like one exam after another exam after another exam. Oh my God, I was so empathetic to the students.”

A lot of professors said they valued critical thinking in their syllabi — but when Melzer looked at those professors’ assignments, he found that they didn’t require much critical thinking at all.

Defining critical thinking

The debate over critical thinking in higher education took off in 2011 when Richard Arum and Josipa Roksa published a book called “Academically Adrift: Limited Learning on College Campuses,” which tracked a group of students who had enrolled at four-year colleges and universities in 2005. As freshmen, the students took a test intended to measure critical thinking skills. When the students took the test again their senior year, they made only marginal improvements. Thirty-six percent of students did not improve at all.

The test, called the Collegiate Learning Assessment, doesn’t ask for students to answer questions about specific subject areas, like science or history. Instead, students work through complex thought processes and come up with solutions to problems.

“Students today can no longer rely solely on mastery of discipline-based information for career success,” the Council for Aid to Education, which administers the test, writes on its website. “They need to be able to analyze and evaluate information, solve problems, and communicate effectively.”

Compared to students several decades ago, the students in Arum and Roksa’s study were writing fewer papers and spending less time on schoolwork. Half of students said they didn’t have any classes that assigned over 20 pages of writing in their last semester, and 35 percent said they studied for five hours or less a week.

But even though professors are assigning less work overall, over 90 percent of faculty members say they are dedicated to developing students’ ability to “think critically” and “write effectively.” Just like in Melzer’s study — where professors said they valued critical thinking, but rarely assigned the kind of work that demanded it — part of the problem might be that professors don’t always know what critical thinking means.

While tests like the CLA will provide specifics about the skill it measures (the test’s entire scoring rubric is available to the public), not all professors can do the same. And when professors have trouble defining critical thinking, it can be difficult for them to teach it.

In a 2009 study, a group of professors thought their students had mastered critical thinking when they were able to rephrase a text in their own words. But the researchers, who defined critical thinking as something more than paraphrasing, disagreed.

The same year, another study followed 51 college students as they completed four activities — all labeled in their textbook as critical thinking exercises — over the course of a semester. Based on tests administered at the beginning and end of that semester, the students’ critical thinking scores actually declined. The problem, the researchers wrote in Educational Research Quarterly, was that the textbook writers labeled the exercises arbitrarily. They did not explain why the exercises would help teach critical thinking — which makes sense, considering they did not define critical thinking in the first place.

The job market

In September 2014, Arum and Roksa released a second study that followed the students from “Academically Adrift” as they entered the job market. Those who graduated with higher critical thinking scores were much more likely to be employed and stay employed. Those who graduated with lower critical thinking scores were much more likely to be unemployed — and much more likely to lose the jobs they had.

“When I hire, I need people who have the mental agility that comes with critical thinking to apply those skills in new situations,” said Anna Benevente, an employer at Registrar Corps. Benevente’s new hires will help advise companies about their compliance with FDA regulations — but most of the time, new hires don’t know anything about FDA regulations starting out.

“I’m sure that there are some jobs where the exact work that you did in school and the exact things that you studied in school are replicated exactly in the job that you take,” Benevente said. “But so many jobs — including ours and the type of work that we do — the actual subject matter of college isn’t always necessarily directly applicable.”

The most successful employees Benevente sees are the ones who can quickly synthesize information, and then apply the information to a particular product.

For instance: One of Benevente’s employees is developing a service to help companies navigate infant formula regulations. She doesn’t have a legal background — she was a chemistry major — but “because of these incredible skills that she has, and the adaptability that she has, she can go into a completely foreign area and not only master it but then help us develop a service that the whole company is going to market.”

The weight employers place on critical thinking skills shows up in national survey data. In 2013, the Association of American Colleges & Universities found that 82 percent of employers think colleges should place more emphasis on critical thinking and analytical reasoning. And 93 percent say that the ability to “think critically, communicate clearly, and solve complex problems” is more important than a job applicant’s undergraduate major.

“When you do surveys like this you actually quite rarely get such a high number,” said Debra Humphreys, Vice President for Policy and Public Engagement at AAC&U.

The high numbers, Humphreys said, are “a result of an economy that’s been changing very, very rapidly — an economy in which what you know is less important than your ability to take what you know and put it into action to solve complex problems.”

In the classroom

But the rationale Humphreys describes — that because of a tough economy, employers are putting more weight on critical thinking — is relatively new.

Looking back to the beginning of higher education’s history, colleges originally existed to teach the classical curriculum. The goal of the classical curriculum had more to do with teaching general knowledge than with providing specific skills that would be useful in a career.

Mike Minutello, a Ph.D. candidate in Penn State’s Higher Education program, thinks that the classical curriculum was higher education’s first attempt at teaching critical thinking, even though educators weren’t using the term yet. Students studied subjects like rhetoric, philosophy, and logic, and they learned to construct arguments and defend their positions.

“Their thinking and their education were really firmly rooted in that classroom-based, classical curriculum,” Minutello said. “And that, of course, was really made possible by the fact that colleges in this era were primarily there — well, solely there — to teach.”

That started to change around the Civil War, Minutello said, when colleges started expecting faculty to produce research. Faculty were expected to focus less on teaching undergraduates, and students were expected to focus more on learning practical subjects like the sciences or agriculture. Colleges were seen as training grounds for intellectuals who would use their education to help the American economy.

By the 20th century college curriculums shifted again, in favor of student specialization. Instead of learning the same subjects as their classmates, students started to specialize in knowledge areas that would help them get a job. Education became more about training for a career, and less about acquiring general skills like critical thinking.

But sometimes, the distinction between acquiring a specific skill set and learning to think critically isn’t entirely clear.

For Beverly Peterson, a writing professor at the College of William and Mary, teaching skills like writing is impossible without also teaching critical thinking. Sometimes, Peterson will see papers that present a problem — for instance, an animal in danger of extinction — that then spend several paragraphs talking about “what a shame this is.”

These kinds of papers, Peterson said, do not involve critical thinking. She gives these students a C and asks them to push their papers further.

“It’s a good summary. They clearly understood the information, but that’s not acceptable college work,” she said. “Those are not the papers students are going to remember having written once they graduate.”

Melzer also wants his students to do the kind of work they will remember. At first, his students resist his open-ended assignments that require them to think critically. But they almost always get better. And when they do, Melzer said, there is a sense of liberation in learning how to work through a complex thought process.

“By the end most of them have realized, ‘I write better when I think for myself, and I enjoy it more — and I actually learn something,’” Melzer said. “By the end of the class, they felt a freedom that they hadn’t felt before as writers.”