U.S. News & World Report’s university rankings should abolish peer review

Sonia Cardenas is vice president for academic affairs and dean at Trinity College in Hartford, Connecticut.

Now is the season for university rankings, and educational institutions from all over the country are categorized into lists to help prospective students choose a university. These rankings are often seen as a proxy for academic quality. In reality, they may be unreliable.

I am one of the above 4,000 university administrators are being asked to fill out key elements of U.S. News & World Report’s rankings. Colleague reputation survey. Peer reputation is one of the most weighted variables in the ranking Constantly changing algorithmsIt accounts for a whopping 20% ​​of a university’s overall score.

It’s worth noting that only about one-third of those who receive a peer reputation survey complete the survey. Perhaps this is why US News uses a two-year weighted moving average of responses. To put this peer assessment into context, we now take a closer look at how the survey is completed.

The Peer Reputation Survey asks respondents: President, President, Dean of Admissions, etc. — Assessing the academic quality of hundreds of universities across the country. For the liberal arts college where I work, they offer things like: List of over 200 other liberal arts institutions. When evaluating each university, you should consider the quality of its curriculum, faculty, and graduates.

The challenge is to rate each university on a five-point scale from “outstanding” to “poor.” Respondents who are not familiar with an institution can respond “I don’t know” or leave the scale blank. If you answer the survey honestly, most universities will give you a blank answer.

Judging the quality of an institution where I have never worked, studied, or conducted research. Certification examination It’s like asking a culinary expert to rate the specialties of hundreds of restaurants you’ve never tried. No one, no matter how informed, knows enough about the academic quality of every other university in the country to responsibly rate it on a five-point scale. That’s unreasonable.

Nor can we completely ignore the degree of organizational self-interest. Administrators may score competitors less generously. This is an anonymous survey. And, for better or worse, the outcome could have real consequences for enrollment and tuition revenue.

Brand awareness is the most rigorously measured in peer reputation research. This is the reason statistical research It shows that the biggest predictor of a university’s ranking is the previous year’s ranking. This isn’t all that surprising.

You don’t have to be a social scientist to understand serious issues. methodological flaws In the survey. Any ranking, and certainly one with significant results weight in public opinion and family investment — We should aim to be objective. However, in the U.S. News rankings, one-fifth of each university’s score is based on the subjective opinions of university administrators who responded to the survey.

The academic quality of the universities at the top of US News’ list is undoubtedly high. But are these institutions objectively the best? In the words of Princeton University President Christopher Eisgruberlabeling a university the best is, at the very least, “strange.”

It’s true that several factors that U.S. News considers in its rankings affect the quality of a university’s education. The big difference is Student-to-faculty ratio or full-time to part-time faculty ratio It’s important. However, we find that even highly ranked universities overall can score similarly on these individual criteria. For example, the universities currently ranked 10th, 30th, and 105th all have student-to-faculty ratios of 8:1.

It’s time for the US to do so. news We have abolished peer reputation surveys for rankings. Participants are decreasing from an already low response rate..

This publication will better serve students and their families by presenting objective content. quantitative Indicators for each institution, Instead of subjective survey responses — Effectively more of a sorting mechanism than a ranking table.

Ask prospective students to select the factors that are important to them and see how each university relates to each other. This matches 2023 survey Regarding students’ views on university rankings, we found that students entering university are more interested. Rather than the exact numerical ranking (or “street address”), we focus on where the institution is located overall, for example, whether it ranks in the top 50 liberal arts colleges nationally (its “neighborhood”).

As long as peer reputations are used to evaluate universities, students and families are at risk of confusing overall rankings with academic quality. The fact is that 20% of institutional rankings are based on research It means little to those who have completed it. This is also a scientifically unreliable and publicly irresponsible approach.

If you want to understand a university’s academic quality, don’t forget to dig into the data and listen when students and alumni talk about their experiences.