Ranking Canada’s law schools
They’re all hard to get in to. Which one will you get the most out of?
Why does the Maclean’s ranking focus on the most competitive jobs, when by definition most law grads will not end up landing one of these positions? “Elite employers have their pick of new law school graduates, and their decisions tell us something about the reputation of schools and the quality of education they impart to their students,” says Leiter. “What they tell us about law school reputation and education matters to every law student, regardless of what professional trajectory they want to pursue.”
Faculty quality was assessed by the Faculty Journal Citations measure, worth 50 per cent. We employed the HeinOnline database of legal periodicals and, in a break from last year, our search included not only citations in Canadian journals, but international publications as well. The inclusion of international journals was recommended by many Canadian academics, and reflects the reality of a globalized academy. “That was one point on which feedback from Canadian law professors was uniform,” says Leiter. We measured the number of citations recorded by each faculty member; the tally for each school was then divided by the size of that school’s faculty.
Why does the Maclean’s ranking not take into account such things as the different courses, clinics, styles and approaches offered by each law school? These matter, says Leiter, and are worth considering, but “none of them really compensate for the scholarly excellence of a faculty and professional opportunities.”
All data was compiled by researcher Jonathan Heppner. Ranking on each indicator and overall rank was determined using the statistical percentile method that Maclean’s has long employed in our annual university rankings. Our statistician was Hong Chen of MacDougall Scientific Ltd. Statistical Consultants.
GO TO FIRST CHART: Overall ranking
page 1 page 2
One of the problems with this ranking is that it implicity assumes the students entering law school are homogeneous. They’re not. The top students enter the top schools, and we should expect them to garner better jobs even if the law schools are all identical.
What we really need is a ranking that somehow computes “value added” by law schools – or anything else one cares to evaluate. I have yet to see this accomplished and cannot think of how to do so without violating privacy laws.
Andrew, how does the ranking implicitly assume that students entering law school are homogeneous? I don’t follow.
Andrew raises a good point…. A very good point, in fact, which is why I’m going to devote such a long response to it.
But first of all, a qualifier: Andrew’s point doesn’t apply to the “Faculty Quality” portion of the law school ranking, worth 50%. Faculty Quality is an output measure that can’t be subjected to the “input subtraction” that he is suggesting. So put Faculty Quality aside. Let’s just talk about the remaining 50% of the ranking, namely “Graduate Quality.”
The Graduate Quality measures on the law ranking show that graduates of U of T law are more successful at landing the most competitive legal jobs than graduates of, say, U Moncton law. That is something worth knowing. But Andrew is pointing out that the law ranking doesn’t answer another question. It’s a question we didn’t set out to answer, but certainly a question worth pondering. Question: Do U of T grads enjoy greater labour market success than U Moncton grads due to the fact that U of T students may have already been ahead of U Moncton students before they even entered law school (with higher undergraduate averages, for example)? Or is the gap between the job market outcomes of U of T and Moncton grads partly or wholly attributable to some unknown X factor that we’ll call U of T’s valued added? The X factor might be explained by better teaching at U of T, or better career counseling, or job placement services, or a better alumni employer network or… or, well, who knows? If we could compare the entering quality of students at each Canadian law school, and then compared those inputs with the rankings outputs, which law schools’ graduates would we find went on enjoy better than expected job market outcomes, and which experienced worse than expected job market outcomes? All interesting questions. Just not the questions that we were asking.
Our ranking results tell and reveal a great deal -– much of which has been only discussed anecdotally until now, in the absence of this kind of hard data. But our ranking should not be taken as proof that U of T does a better job of increasing the intellectual capital of students than, say, Moncton. The fact that the profs at U of T are on average more prominent in their fields, as measured by the citations indicator, and the fact that grads of U of T are more likely to land the most competitive private and public sector jobs suggests that something positive is happening at U of T. But the ranking doesn’t explore the factors that went in to creating those positive outcomes, or the relative difference in student quality inputs that went into the outcomes. The ranking just measures the outcomes – and those outcomes are important.
How important? Well, if you were the parent of a 12 year old, and I told you that one high school in your neighbourhood sent 100% of its graduates to university and another high school sent 50% of its graduates to prison, I’m guessing that you would probably find this data to be of considerable interest in selecting which school your son or daughter should attend—even in the absence of a statistical analysis of the quality of each high school’s incoming students, and a calculation of each institution’s valued added as measured by the degree to which spending five years at high school X did or did not add to its students’ likelihood of achieving post-high school educational success.
But let’s go back to critiquing the ranking. The job market outputs that we measured in the ranking are to some degree — we obviously don’t know to what degree — dependent on the inputs, i.e. incoming student quality. Better incoming students to some degree equals better graduates. It might even be possible to measure this relationship between inputs and outputs.
Given a bit of time and money (can anyone spot me a few grad students, an economics professor and a SSHRC grant?) and willingness on the part of universities to open up several years or decades worth of admissions files, and an ability to track those graduates into the labour market, it should be possible to calculate an output-input difference at each law school, by doing a study similar to the one done a few years ago by Princeton economist Alan Krueger. It would be fantastic if someone were to do such a study in Canada. Krueger compared the career outcomes of students who studied at Harvard college with those who were accepted at Harvard, but instead went to less prestigious U.S. state schools. What he found is that inputs in that case were important, even determinative: two decades after graduation, the high performing high school students who chose not to go to Harvard enjoyed careers and incomes that were not substantially different than those of the high performing high schoolers who chose Harvard. It’s one of the reasons I’ve argued, in this magazine, that where you go to university for your undergrad degree matters and is important and will make a difference in your life in a host of ways, but going to university at all – and what kind of a student and person you are, and what you want to get out of university and are willing to do at university and after university to make the most of your life-– will have far more of an impact in terms of career success.
Until a few years ago, the annual Maclean’s University Rankings issue included a rough value-added measure of undergraduate programs. Our statisticians compared the quality of entering students at each university — as measured by the high school grade averages of entering undergrads and percentage of students with high school grades over 75% — with two output measures: the percentage who graduated on-time from university, and those who won national student awards. It was a rough attempt to establish a relationship between the quality of a university’s student inputs and the quality of its graduate outputs. The university that finished first on value added for several years in a row, Lakehead, even made “Canada’s best valued added university” a campus slogan and centerpiece of a national marketing campaign.
Lakehead was able to claim that title because, though it was among the least competitive universities in terms of the students it took in, when compared to other Canadian universities, the percentage of Lakehead students graduating on-time and winning national awards was higher than statistically expected.
Denouement: a few years ago, a number of Canadian universities unfortunately decided to stop making public some of the information that we used to calculate this measure. We can’t calculate it anymore. The data isn’t available.
But back to law schools. So what’s the bottom line? What does the Maclean’s law school ranking tell us? It tells us how Canadian law schools compare in terms of faculty quality, according to one metric, and how the most competitive employers in the private sector, academia and government believe the law schools and their graduates compare, as revealed by whom these employers have chosen to hire. All of which is worth knowing, whether one is a student, a potential student, a faculty member or an interested member of the public. Does the ranking tell you something important about each law school? Yes. Does it tell you everything? No.
[...] This year’s ranking – the second that Leiter has done – has just been published at Macleans (here). In many ways Leiter’s rankings are a useful contribution. As he has suggested of rankings [...]
[...] Here’s their Canadian law schools rankings. [...]
The measure “Elite firm hiring” is fairly misleading and seems to rest on a number of faulty assumptions. And it is not at all clear how it measures “graduate job quality” or “graduates able to land the most competitive public and private sector jobs”.
First of all, it rests on the assumption that “large” means “elite”. What evidence is there that the largest firms actually have the most competitive or sought after positions? In my experience, the most sought after firm positions were with boutiques. Public sector positions with federal and provincial justice were also highly sought after: neither is addressed in the survey. Not to mention that since the largest firms tend to be clustered in the larger centres, this probably biases the survey toward schools in larger centres: no surprise then that U of T, McGill and UBC come out on top.
Second of all, what evidence is there that large firms provide the best “job quality” In this regard, I would think that a job satisfaction survey would be a better measure.
The fact that this flawed measure is then extended into the “National Reach” category such that it ultimately determines 30% of a school’s raises serious questions about the utility of this ranking.
THe most successful lawyers aren’t always the ones who have entered expensive schools. Its all about their own preparations and development. Its all about the attitude and pressure that makes the best lawyers (which does appear to be some of the more expensive universities) if you fail you’ve just wasted money.
Many of the McGill Faculty publish in French. Does the Faculty Citations criterion take this into account?
[...] Every year, the Internationally Trained Lawyer Program will aim to help 100 lawyers who need accreditation to practice law in the province. Michael Chan, Ontario’s minister of citizenship and immigration, launched the program a ceremony last Tuesday. More: Ranking Canada’s law schools [...]
[...] Ranking Canada’s law schools : Macleans OnCampus [...]
It’s all about their ethics, and it appears, that they are desperately needed:
http://ahabit.com/letter.htm
New graduates will hopefully not tolerate the miscarriages of justice that destroy the lives of victims and innocent people.