All Posts Tagged With: "Colleges"

Which universities get top marks? 87,000 students have their say

It’s mid-January, a couple of weeks after the Christmas break, and Mark Woodcroft, a fourth-year biochemistry major at Trent University, is hanging out in the lab with professor Steven Rafferty, his research supervisor and chair of Trent’s chemistry department. Woodcroft is doing what many Canadian undergraduates never get a chance to do: an independent research project under faculty supervision.

Want to read more? Full student survey results are available here.

Click here to go to the web-exclusive student surveys.

So, a reporter asks, what’s your research project about? Woodcroft casts a sly smile at his prof and then launches deadpan into an explanation of the “bioaccumulation of perfluorinated carboxylic acids.” His audience predictably befuddled, Woodcroft stops mid-sentence. He and Rafferty chuckle in unison. It sounds like a well-rehearsed routine. Not something many 22-year-olds get to cook up with a professor.

“In upper-year courses, the class size is small enough for a professor to know each student by name,” says Woodcroft. “I also know everyone in my program by name. I doubt many students at a larger school can say that.”

Personal contact with faculty members, a sense of community among undergrads and classes that push students to their intellectual limits—these are all things that many undergraduate students desire. Research suggests that these also promote learning; in the language of the National Survey of Student Engagement, these and other aspects of student engagement are “correlates of quality.” And according to the NSSE Benchmarks of Effective Educational Practice results appearing on the following pages, undergraduate educational quality at Canadian universities—with only a few exceptions—is below that of American universities.

Below you will also find results from the Canadian Undergraduate Survey Consortium, or CUSC, a Canada-only survey that is much more tilted toward assessing student satisfaction. In 2007, CUSC surveyed first-year students at 32 universities. You can find results for seven additional CUSC student satisfaction questions on our website, at www.macleans.ca/oncampus.

While undergraduate student satisfaction remains relatively high at Canadian institutions, the NSSE benchmark results suggest a different story: satisfied or not, many Canadian university campuses are not as engaging and may not be offering as good an educational experience as their American peers. And the problem is particularly pronounced at Canada’s large research universities—the schools educating the overwhelming majority of Canadian undergrads.

The American-based NSSE survey is a tool widely used by universities to analyze, benchmark and improve their institutional performance. Since 1999, the American-based NSSE (pronounced “Nessie”) has been conducting its survey on a growing number of campuses, and calculating its Benchmarks of Effective Educational Practice for each participating school. Beginning in 2004, a growing number of Canadian universities began to take part in NSSE. The biggest push came from Bob Rae’s 2005 review of post-secondary education in Ontario. Rae called on the province to establish measures for evaluating quality and publicly reporting on system performance. In his review, Rae asked, “How are we doing? How are others doing? Is there a jurisdiction that does it better?” His conclusion: “We simply don’t know enough about how we are doing or how others are doing.” To this end, Rae recommended that all Ontario universities participate in NSSE. All Ontario universities have done so over the past two years, and most universities in the rest of the country have joined them. Several of the 47 universities that Maclean’s surveys in its annual rankings of Canadian universities have never participated in NSSE; they include Bishop’s University, Cape Breton University, St. Francis Xavier University, Memorial University, Université de Moncton and Université de Sherbrooke.

Law has always been among the most competitive of professional schools. So how do Canada’s law schools compare?

For a complete explanation of the methodology behind the rankings, CLICK HERE

Welcome to our first annual ranking of Canadian law schools. The annual Maclean’s university rankings, published each November, have long offered a broad evaluation of the quality of undergraduate education at each university. But this marks the first time that we are ranking a specific program within the university.

Law has long been one of the most competitive of professional schools. Above-average undergraduate marks are generally a must, with some schools being so competitive that they only extend offers to the most outstanding students.

Our law ranking is not, however, a ranking of which schools are the hardest to get into. It is, instead, about measuring the quality of the output of each school.

The methodology behind the Maclean’s law school ranking was created by professor Brian Leiter, the Hines H. Baker and Thelma Kelly Baker chair at the University of Texas at Austin Law School. He is also a professor of philosophy, as well as the director of U Texas’s law and philosophy program.

Leiter may also be America’s most prominent critic of the best-known journalistic ranking of law schools: the annual law school rankings from U.S. News and World Report. Leiter’s criticisms have been directed at the specifics of the U.S. News methodology, which is, among other things, based in part on data provided by schools, contains some data that is open to manipulation, and other data that, even if accurate, may not be measuring anything particularly relevant. On one of his blogs — www.leiterrankings.com — he has for many years compiled and published alternative ways of measuring law school quality. His numbers are often looked to by those in American academe who seek to measure roughly where their schools stand.

We turned to Leiter to help us build a relevant and unbiased assessment of Canadian law schools.

The Maclean’s law school ranking contains only four elements, all drawn from publicly available data. Fifty per cent of the ranking weight is devoted to student and graduate quality; the other 50 per cent is composed of a measure of faculty quality.

To calculate the “Faculty Journal Citations” measure, weighted at 50 per cent of the ranking, we counted the number of tenure and tenure-track faculty at each law school, excluding adjunct faculty, emeritus professors and the like. We then researched each professor’s citation count in Quicklaw’s database of 33 Canadian legal journals. We added up total citations for each school, and then divided by the number of professors at each school.(For more on methodology, go to macleans.ca/oncampus and click on “Rankings.”)

For the “Elite Firm Hiring” measure, worth 25 per cent, we relied on the Lexpert list of the leading Canadian law firms, and Vault’s list of leading New York firms. On each firm’s website, we counted the number of associates from each school. We divided each school’s total by the size of each school’s first-year class, as provided by the website of the Law School Admission Council. The “National Reach” measure, worth 15 per cent, involved using information gathered for “Elite Firm Hiring,” and calculating how many of each school’s graduates had been hired by leading firms other than the three firms that hired the most graduates from that school. This is a rough measure of the extent to which leading firms outside of a school’s region hire its graduates. “A degree that gets you hired from Vancouver to Montreal is a degree that many students may prefer to have,” says Leiter. “That’s what we’re trying to measure here.”

The “Supreme Court Clerkship” measure is worth 10 per cent. We looked at Supreme Court clerks hired over the past six years, and counted the number from each school. Supreme Court clerkships are one-year positions, awarded to the country’s top students, as chosen by the judges. Our source for the list of clerks was Osgoode Hall’s The Court website,(www.thecourt.ca/clerks-of-the-supreme-court).

Is this ranking useful for potential students? “Excellence of the faculty and professional opportunities afforded by an education must surely be two traditional and central markers of academic excellence in law school or any professional school,” says Leiter. “Schools themselves engage in constant self-representations on both counts, and professionals and students tend to have inchoate impressions of their own. Quantitative and systematic study of how schools actually fare along these dimensions should prove a useful corrective to advertising puffery and dated or inaccurate anecdotes.”

A Q&A with Professor Brian Leiter

To create its first annual law ranking, Maclean’s turned to Brian Leiter, a professor of law at the University of Texas at Austin. Leiter is probably the most prominent and long-standing critic of the U.S. News and World Report rankings of U.S. law schools, and has long offered alternative, and more useful and accurate ways, of assessing law school performance.(For more on Leiter’s performance measures for U.S. law schools, see www.leiterrankings.com. For the full run down of all of Leiter’s blog, on a variety of legal topics, visit http://www.naymz.com/search/brian/leiter/793046).

This past spring, I approached Leiter, to ask him to work with us to create an assessment of Canadian law schools. The methodology that was ultimately adopted is simple, transparent and relies entirely on public data. It contains four measures, with a 50% weighting for Student/Graduate quality and a 50% weighting for Faculty quality:

• Faculty Quality(weighted at 50%), measures how often faculty members at each school are cited by other academics in 33 Canadian legal journals found in the Quicklaw journals database. Maclean’s did a citation search for each faculty member, with faculty defined as professors holding titles such as professor, full professor, associate professor and assistant professor, and excluding emeritus professors, adjunct faculty, law librarians, administrators who are not professors, and the like. The list of faculty was drawn from each law school’s website in August, 2007.

• Elite Firm Hiring(worth 25%)uses the Lexpert list of leading Canadian law firms as its basis. Because a small number of highly qualified Canadian law graduates also move to the US, we also counted hires by the five leading New York law firms, as measured by Vault. For a list of firms studied, CLICK HERE Maclean’s examined the website of each of these leading law firms, and counted the number of associates from each law school at each firm.(We counted only first law degrees, and for Ottawa we reached one grand total, rather than dividing its graduates up as Ottawa-Civil and Ottawa-Common). Young lawyers start out as associates and are usually either partners(or no longer at the firm)in seven to 10 years, so this measure captures hiring from the mid- to late-1990s to the present. To scale these figures to school size, the totals for each school were then divided by the size of the 2006 first-year class at each school. First year class-size numbers were taken from the Law School Admission Council website. Of 2103 associates found at the firms on the Lexpert and Vault lists, 47 had their first law degree from a non-Canadian law school, and 42 had no biographical information.

• National Reach(worth 15%): This measure looks at how widely spread are the graduates from each school. The idea is to get a sense of whether a law school is able to place its grads at leading firms beyond its region and beyond a small network of firms. The elite firm hiring count from the previous metric was examined to determine what percentage of a school’s graduates are at elite firms other than the three elite firms that have the most associates from that school. If School X has 100 associates at elite firms, and 45 associates at its top three firms, it would have a “reach quotient” of 55/100 = 0.55.

• Supreme Court hiring(worth 10%): Clerks are hired by the Supreme Court for a term that is usually one year; the clerks are selected by the judges and are generally chosen from the country’s top graduating students. We measured clerkship hiring over the past six years. Starting with the list of law clerks on Osgoode Hall’s The Court blog, at http://www.thecourt.ca/clerks-of-the-supreme-court, we researched which law school each clerk attended for their first degree. We were able to find first-degree information for 142 out of 162 clerks. For the University of Ottawa, we did separate calculations for each branch of the law school(common and civil). We tallied the number of clerks from each program, added the number of clerks with both civil and common law degrees from Ottawa to each of those tallies, then divided by the number of first year students in each respective law stream at U Ottawa. There were 20 clerks from Ottawa common law, two from Ottawa civil law, and three with degrees from both.

Just prior to publication, I asked Professor Brian Leiter to answer some questions about the rankings.

Here you will find the NSSE, CUSC and Maclean’s student survey charts and an explanation of how they were done

You will find results from three surveys here: the National Survey of Student Engagement(NSSE), the Canadian Undergraduate Survey Consortium(CUSC), and the Maclean’s University Student Survey. The NSSE and CUSC surveys, which were commissioned by the universities, ask more than 100 questions about specific aspects of the undergraduate experience—inside the classroom and beyond—designed to provide universities with data to help them assess programs and services.
Launched in 1994, CUSC is coordinated through the University of Manitoba’s department of housing and student life. In 2006, 25 universities took part, sending surveys to a random sample of approximately 1,000 senior-year undergrads at each university. A total of 10,464 students responded.

The U.S.-based NSSE began as a pilot project in 1999 and is distributed to first- and senior-year students. In 2004, 11 Canadian universities participated for the first time with 14,267 students completing the survey. Last year, that number had grown to approximately 60,000 students at 31 Canadian institutions taking part.

Nine institutions ranked in the annual Maclean’s University Rankings issue did not participate in either the 2006 CUSC or NSSE surveys. To provide student feedback from these institutions, Maclean’s asked them to take part in a short survey using questions drawn directly from the CUSC questionnaire, addressing such issues as the quality of teaching and the overall educational experience. Eight of the nine universities agreed(Université de Moncton declined). CUSC wording was followed and CUSC methodology was also employed: participants in the Maclean’s University Student Survey were randomly selected from students currently in their final year. Universities contacted selected students by email, inviting them to participate. Large universities contacted 1,000 students; smaller universities, with fewer than 1,000 students in their graduating year, surveyed the entire cohort.

The survey was conducted online by Angus Reid Strategies, and was active from Feb. 14 to March 12. To ensure that only those who had been chosen could take part, each individual was assigned a unique PIN. These PINs allowed Maclean’s to identify students by university while guarding their anonymity.
The Maclean’s survey achieved a 43 per cent response rate, with 2,683 students from eight universities taking part. The results, when presented for all universities, are accurate within 1.52 per cent, 19 times out of 20. Individual institutional accuracy varies from plus or minus 3.06 per cent to plus or minus 8.88 per cent.

The Maclean’s survey asked only eight questions from a much longer CUSC survey. As the Maclean’s survey questions were not asked within the context of the larger CUSC survey, the inherent question-ordering and placement bias may have been different in both surveys. This may have had an impact upon the comparability of the Maclean’s results to the CUSC results, since measuring satisfaction across an extensive battery of specific questions can result in lower satisfaction scores than when asking fewer, more generalized questions. We have therefore chosen to present the CUSC and Maclean’s surveys separately.