If you’re a regular listener of Business Casual, you’ll know that John, Maria and Caroline aren’t the biggest fans of the various MBA rankings and are often of the opinion that they’re simply “a story” that publishers like to shuffle every year to keep things interesting. As they’ve done with other rankings this year, in this episode, the team digs deeper into the methodology (or lack thereof) behind the new “Best Business Schools” ranking from Bloomberg Businessweek. 

Episode Transcript

[00:00:07.330] – John

Well, hello, everyone. Welcome to Business Casual our Poets and Quants weekly podcast. Today we’re going to talk about the new ranking from Bloomberg Businessweek. That ranks MBA program full time in the US. Europe and elsewhere in the world. Today’s podcast is sponsored by the Gies College of Business at the University of Illinois. Are you thinking about earning your MBA with the fully online IMBA from the University of Illinois East college of business. You can earn your degree on your schedule without ever leaving your home. You’ll learn from Gies College of Business top faculty and built a global network of experienced peers at a cost of just $23,000. It’s no wonder the IMBA comes with a 96% student satisfaction rate. Apply by October 6 to start classes in January. Learn more at onlinembaillinois.edu. Okay, another ranking, another wacky result on some level. At the top of the ranking, it may look pretty solid. Stanford is in first place again. This is the fourth consecutive time in the Bloomberg business ranking, Gregson, that Stanford took top honors. Chicago, Booth and Harvard are tied for second. Northwestern is fourth. Dartmouth five. MIT six. Wharton still lagging.

[00:01:33.410] – John

Last year, you recall, Wharton had its lowest Businessweek ranking every 9th place. This time it’s 7th. Columbia, 8th. UVA Darden, nine. Yale, ten. If you look at the European results, it’s kind of interesting. IMD is number one. INSEAD number two, IESC in Barcelona, Spain. Three, London Business School for Baconi in Italy, number five. What do you make of these rankings? Caroline, we’ll start with you. Given the fact that I just rolled off the European yes.

[00:02:09.280] – Caroline

Well, I do think that they like to make a story, right? So they have to shuffle things around a little bit every year. Otherwise what would be the point of publishing it and how would they make a story out of it? So I’m a little bit skeptical to know about the rankings. I think there can be value in them. But it’s really important for students and candidates to delve into the data and understand the methodology and because, as we’ve discussed before, the methodologies do vary dramatically and it all depends on what criteria are important to you -as an individual, right, and whether those are reflected in the ranking or not. And that may not be the case. And there may be certain criteria that give some schools a natural advantage over others. So one example of that, as you’ve highlighted before in the FT ranking, is the weighting of international students in the international element of the program. And that obviously gives an advantage to schools like INSEAD, which have that very much as part of the school DNA. And in this particular ranking, IMD has an advantage amongst the international schools because salary is given such great weight in the ranking.

[00:03:30.510] – Caroline

And given that IMD students are typically a bit older than the average of other MBA amongst MBA programs and therefore more experienced and probably earning more money when they go into the program and therefore looking to earn more money when they graduate, that gives the school an advantage. So not saying that’s necessarily wrong, but it just means that it’s important to understand what’s going on and what’s driving these dynamics. And for individuals who are looking at this as a guide to where they should apply, they really need to delve into that data and understand what is going on.

[00:04:08.530] – John

Yes, we should point out how does Businessweek actually rank these programs? And Caroline referred to the element of compensation that clearly is the single most heavily weighted factor in these rankings. The other factors that Businessweek contends considers is learning, although we’re going to get into the vague explanation that they have in the methodology for what exactly constitutes learning, which I know is a pet peeve of Maria’s, and she dug through the methodology and found very little explanation for what learning is or how it’s ranked. Here networking, entrepreneurship, and diversity. In each of these, there are some weird, quirky results that definitely make you incredibly skeptical about the entire ranking. Now, here’s one in the learning index. For example, Hulk Business School places higher than Chicago, Booth, MIT, Sloan, Columbia, Duke, Berkeley, Michigan, NYU, Stern, or UCLA. Now, I don’t know how that is even remotely possible, and that’s no dissonance, but there’s just no way that anyone, any faculty member, any dean, any real student, any alum, any employer would say that MBAs at Hulk learn better or learn more than the MBAs at places like Wharton, MIT, Sloan, Columbia, Chicago, Booth, Duke, Berkeley, Michigan and others.

[00:05:44.100] – John

There’s just no way. I mean, it’s insane to even think that that’s possible. The other issue here is diversity. How do you measure diversity? Well, the number one ranked school for diversity by Businessweek is Howard University, which probably has the least diverse MBA program. Now, if you were ranking on the basis of just simply underrepresented minority, sure, Howard should be number one because the entire student body is black. But that is not made for a diverse MBA classroom. Just because you have one race represented in a classroom and none other, that’s the opposite of diversity. And there are all kinds of problems with each of these metrics in the way Businessweek mindlessly goes about trying to calculate them. Now, Maria, I mentioned that you dug through the methodology and you’re as mystified as I am about what the heck are they measuring?

[00:06:37.910] – Maria

I mean, I dug through the methodology and so far as they provide a whopping, what, three sentences to explain their methodology. So not exactly standing up to rigorous third party scrutiny. The learning one for me is just a lot of their metrics are a little bit silly, but at least compensation, you say, okay, well, compensation is something that we can measure. It’s something that is at least factual on some late basis. And diversity, I agree with you. I think that they should rename diversity to something that is more along the lines of promoting opportunities, providing opportunities for otherwise underrepresented groups. Because in that case, I would agree. Howard is probably the best school in the country for that, hands down. But if you define diversity in terms of having a wide spectrum of people, then it just depends how you choose to define that term. But the Learning Index, I mean, basically it’s based on a survey. So they’re surveying, for example, recent graduates. One of the things that kind of bugs me there are lots of things that bug me, so buckle up. One of the things that bugs me is that 25% of their waiting comes from people who have just graduated in the class of 2022.

[00:07:53.540] – Maria

Right now, it is September of 2022. So even if they just collected this data a week or two ago, these are graduates who have only been out of school for about three months. And I’m sure that they needed more time than that to collect the data. But let’s just give them the benefit of the doubt. Hypothetically speaking, let’s say the graduates were three months after graduation when they took the survey. How can a student who just graduated provide a lot of perspective on how much they learned, how valuable that learning was, how valuable the network is? A lot of these things, other than compensation, it can be very difficult to ask someone. If you would have asked me three months after I graduated, well, what was the learning like? I probably would have had a very different answer than I do today. So that’s a little bit strange. But yes, for the Learning Index, and this is the one that’s a lot of fun to play with. You can either go to the Poets and Quants story about this, or you can go to the Businessweek website, but you start filtering the US. Schools by the learning score, and look, these are great.

[00:08:54.520] – Maria

Again, we’re not trying to diss anyone. I mean, the number one school is William and Mary. College of William and Mary Emory. Georgia. Texas. Dallas. Jindal. So these are all wonderful schools, but how do you judge if you’re self reporting? Yes, I think I had a wonderful learning experience at my business school. I know as a graduate of that business school that my school looks better if I say amazing things about it.

[00:09:20.300] – John

Totally.

[00:09:20.970] – Maria

Therefore, I have a very vague I mean, there’s not even an assemblance of objectivity here, right. The motivations for somebody to say yes, even if they hated it, or even if they didn’t learn squat, they have a motivation as graduates of the program to make their own program look as good as possible. And so who knows how this is being judged, because there is no universal test at the end of business school. Unlike other graduate types of programs, where there might be licensing exams or bar exams or medical board exams. There is no similar exam for MBA. So there really is no way to quantitatively say, well, this school teaches more than other schools, or the graduates of this school are more academically prepared than others. And as much as I dislike the reliance of other school sorry, excuse me, other rankings, emphasis on GMAT scores and GRE scores, at least that’s something that is a standardized academic measure. So at least you can sort of try to guess or try to glean some sort of indirect information about the quality of the students at that school, but simply saying, well, we asked for something called the learning index, and we pretty much just looked around.

[00:10:39.200] – Maria

What are the classes that they offer and what’s the breadth of those classes? So anyway, I will stop talking about the learning index one, but it’s like, hey, if I know that, I’m going to help myself by saying that my school had perfect teachers and I learned so much and it was the best learning experience ever. I’m going to do that even if it’s not true.

[00:10:59.240] – John

That’s true. And let me put even another little sort of underline on this. If you come from a school that isn’t generally top ranked or highly selective. You are probably going to be more inclined to be generous to the school and how you score it knowing that those scores will be used in the ranking than if you come from a Harvard of Stanford or Wharton. Where you have the self confidence to know that your degree is highly desired by a lot of employers. Is recognized for its quality and its standards. And there’s no question about the value of that degree. So that self confidence that you bring to filling those questions out on the survey is going to be reflected in a ranking because you’re less likely to be a total cheerleader for a brand that isn’t Harvard, Stanford, Warden, INSEAD business school, but rather whole. And that’s a real problem. And there’s no way of this ranking is based on survey data from students, alumni, some from employers. And don’t get me started on their employer survey because it’s ridiculous. It’s insane. They basically survey or try to survey everyone who comes to a campus to recruit students.

[00:12:17.510] – John

So if McKinsey goes 20 different campuses, McKinsey gets 20 surveys. And the people who are going to recruit students at Harvard are Harvard alums, okay, and the people who go to recruit students at Dartmouth for McKinsey are Dartmouth Grads, which is the dumbest way to survey employers about their true attitudes about how one program compares to another. But that’s the way business suite does it. There are so many flaws in the methodology and so many mindless decisions by journalists who lack good judgment and lack any knowledge about graduate management education that this thing is a total, complete farce. Now, last year, we covered a pretty controversial dispute. A deputy dean who was in charge of the full time MBA program at Yale School of Management, reverse engineered the Businessweek ranking. He threw all the data that they published into a spreadsheet, did regression analysis and all other kinds of things that I am not qualified to talk about because I’ve never taken the course in statistics. And he discovered that the weights that Businessweek says that it applies to each of these five measures that they rank from compensation of diversity to entrepreneurship and networking and learning were not properly applied because if they were, the rankings would have been substantially different.

[00:13:48.980] – John

And he found that last year 23 of the top 25 schools, at least according to Businessweek, would have had differences in the ranking. Well, guess what? He pulled out his spreadsheet and his analysis again and he applied it to this newest ranking. And he also sees not only similar and significant flaws in the weighting of these different measures. But just as importantly, he says that the methodology was basically constructed to off you skate the errors that Businessweek is making. And one conclusion is that even though Wharton clearly is undervalued at a rank of seven this year by Businessweek, if in fact Businessweek applied the weights that they contend that they applied, wharton would be down to like 19 or 20. Obviously that would be an incredible embarrassment for Bloomberg Businessweek. And so the suggestion here is that the editors put their hands into this black box that they’ve constructed and have manipulated the numbers to some degree so that the results wouldn’t be as embarrassing as they already are. So you can read his analysis on Poets and Quants. It’s on the website along with our own analysis of the data.

[00:15:18.390] – John

Caroline, is there any good at all that comes from this?

[00:15:22.590] – Caroline

On the positive side, I guess these rankings and all of these articles also generate visibility for business education and business schools and MBA programs. So perhaps it does help to attract some attention and provide some useful information to new generation of potential MBA candidates. But I think it’s really a shame that there isn’t more rigor behind this. It is disappointing. And last year when Professor You mentioned had raised his concerns with the analysis behind the ranking, they very satisfactory response at all, right? They just sort of and it looks like they’ll be interesting to see any response this year. But I guess they’re digging in their heels if they have just come up with if they haven’t corrected the errors from last year.

[00:16:26.310] – John

I think he’s getting the silent tree, more or less.

[00:16:30.150] – Caroline

It’s shameful, right? You would hope that the publication that is behind this similar concerns with The Economist and how could an organization and The Economist turn out the rubbish that they turned out here by at least they’ve seen fit to put an end to that. But perhaps Businessweek will go the same way.

[00:16:57.450] – John

I suspect that Businessweek is very vulnerable given these attacks on the credibility we’ve seen the Economists decide, nope, no more for us. We’re out of it. Forbes has not come out with the ranking for three years now, and as far as I know, I don’t see any new ranking that’s forthcoming. The Businessweek ranking now is the second year in a row it’s been severely attacked, and the year before that, you recall, many of the schools withdrew and Business, we had to actually take the hiatus for a year during the Pandemic. That leaves us. News and Financial Times and both. Obviously there are flaws in both of those methodologies, but I think those rankings are generally regarded to be among the more credible MBA rankings out there. The FT given its global nature, and US. News given the rigor of its methodology, based on many base data points that are provided by the schools. Although, as we have discovered there, there’s nothing that prevents the school from cheating, as Temple University did, and led to a prison sentence for its dean. So there is that problem with US. News which does not conduct audits.

[00:18:12.970] – John

The FT does. I wonder, Maria, if you were to construct a new MBA ranking, what would you measure and what do you think could be important and would lead to an authoritative, credible ranking?

[00:18:29.850] – Maria

Well, for starters, I would definitely not ask people to rate themselves. Hey, we’re all above average, right? Like we all began, all the children are above average at our school. First of all, I would get rid of any of that sort of self promotion aspect to it. I think at the end of the day, and I think I’ve mused upon this a little bit before, I think you do forced choice. It’s sort of a survey type of thing where you tell people like, okay, it’s not along a spectrum. Like, here are two options. In between the two, which would you choose? Between Harvard and Stanford, which would you choose? Between Stanford and Wharton, which would you choose? Between Wharton and Yale, which would you choose? Honestly, I know it’s like, perhaps overly simplistic, but ask the students, where would they want to go? Ask the recruiters. If you can only recruit one person from a business school, what school would that be? And also, in that case, I would actually wait the recruiters based upon their eliteness, upon their selectivity. So perhaps there is a program out there that’s like, well, I don’t want to go to Stanford because I know those kids are all going to turn me down, so I’m not going to put Stanford.

[00:19:41.350] – Maria

Even though secretly they wish they could hire Stanford people, they wouldn’t put them because they’re jealous. So I would really just do it as simple as it sounds. Just sort of say and even to faculty, right? If you’re maybe faculty members. I actually have a friend who’s on a faculty at UCLA, Anderson, who was recently considering moving to another school. And there are a lot of decisions that come up to play, come into play when you’re trying to decide what other school to go to, including things like, do I want to live in the city of the other program. So maybe the question to ask faculty and staff members is not where would you most like to teach, but let’s say something like your best friend or the academic that you most admire in your field. Where do you think they should go teaching? If you could say where they should be placed, where would you most put them? Would you put them at some of these others? Would you put them at Halt? Or do you think they would instead go to Wharton? I just think that that’s kind of the because there is no quantifiable way to rate an MBA program, at least no easy way to do so at the very least, then, let’s lean into that subjectivity. But also you can’t vote for yourself.

[00:20:51.370] – John

You know, I like that idea. I like that idea a lot. I think those are really smart, thoughtful ways to ask questions that get a really good answers. I wonder if another way is I’m going to throw this out and then, Caroline, we’re going to come to you, but we’re giving you a little more time to think about this. I wonder if you could say that, look, most people enter MBA programs because they have a vision for what they want to do and even a company or an industry in which they want to work. So if you said. Okay. MBA surveys show that the 50 most popular employers of MBAs are XYZ. And then you did LinkedIn searches on the MBAs who work in those companies in a specific time period for. Let’s say. The last three years. And then you weighed those results by the size of the class of each business school and how many graduates they actually put into the market. Would that be a viable ranking? Because basically what you’re saying there is, look, if you want to go to McKinsey, here are the programs that would most likely get you there.

[00:22:09.290] – John

If you believe that some of the most desirable jobs are at McKinsey, Bane, BCG, JP Morgan, Morgan Stanley, Microsoft, Apple, Google, Amazon and then a select group of private equity firms. Hedge firms, and BC firms. Which may not commonly appear in the top 50 list of organizations that most MBAs want to work for. Would you really be getting that than the schools that most successfully place their students and the jobs that they really want? And is that really what a ranking should show? Because there’s no way to measure in any faithful, accurate, credible way the actual academic experience, which I wish there were, because you can’t trust surveys that go to students or Alums because of the cheerleading factor. And admission statistics are, while somewhat helpful, they don’t really tell you about the quality of education you’re receiving and compensation. Well, obviously the graduates from Harvard Stanford are going to get the most money and they’re going to go into the fields that provide the most money. That doesn’t tell you much about the whole system, but the jobs that you land and the jobs that you want in these highly selective firms that are choosing these people.

[00:23:31.370] – John

Isn’t that basically a great way to measure the marketplace to actually put the feet to the fire of the business schools or is that a cockamamie thing as well?

[00:23:44.160] – Caroline

Caroline well, I do think that the recruitment data is very useful and I think that candidates should definitely pay a lot of attention to that. And I’m always telling candidates to please spend plenty of time combing through those recruitment reports because there is a wealth of information there and it’s very useful and it’s very important to understand if students at the school that you’re thinking about applying to actually getting jobs and making the kind of career change that you’re looking to make. So I do think that some of those indicators that you mentioned could be useful. I think salary can be a useful indicator as well. Of course, it depends what people want to do, right? So with your methodology you could be penalizing schools where they have more people going into, I don’t know, startups. Right. Starting your own company, entrepreneurship. Right. So maybe a school like GSB wouldn’t come out so well in your ranking. Yes, it doesn’t make it any less desirable for certain people or some of the international programs where they may be feeding candidates into more diverse portfolio of companies and some of the top US schools because candidates are going all around the world.

[00:25:09.190] – Caroline

So there is no perfect methodology. But I do think that you’re onto something that the recruitment data and placement statistics and the stories behind that are very important because recruiters, they are buying into these programs, right? It’s not just about some survey or random impressions of one school versus the next. They’re actually hiring people and investing a huge amount of money in recruiting people from some schools and paying them large amounts of money. So they are going to be doing that for a reason and they think that those students are very capable young professionals and can add a lot of value to their organization. So I do think that the choices that recruiters make are very interesting and say a lot about the value of schools.

[00:26:10.610] – John

Yeah, that’s true. You’re right. I mean, you said this before and it bears repeating that one way to really target schools is to look at the employment reports and see if they’re placing students in the companies or industries that you really want to work in. Because if that’s happening, it’s not only the fact that they’re placing them, the fact that they also have alumni who can help you in those industries and companies get in the door and advise you as to how to basically swing the odds in your favor of an employment offer. So that can be very valuable to you. So there you have it. If you want to read up on the latest Businessweek ranking from the Poets and Quants, check it out. We show, of course, the winners and the losers will tell you what is total BS and what isn’t. And we’ll give you the analysis, the latest analysis by the deputy dean who runs the MBA firm at Yale SOM, which is really enlightening and helps to put the whole thing into the proper perspective and makes you think this is more entertainment than it is information. But nonetheless, we’re all junkies for who’s up and who’s down, and we can discount and disregard the actual outcomes, but nonetheless, they’re still somewhat fascinating to read.

[00:27:36.710] – John

Anyway, there you have it. So I want to thank our sponsor, the Gies College of Business. Are you thinking about earning your MBA? With the fully online IMBA from the University of Illinois Gies College of Business, you can earn a degree on your schedule without ever leaving your home. You’ll learn from these top faculty and build a global network of experienced peers at a cost of just $23,000. That’s right, 23k no wonder the IMBA comes with a 96% student satisfaction rate. Apply by October 6 to start classes in January. Learn more at onlinemba.Illinois.edu. I want to thank my co host Maria and Caroline for the insights. If you’ve listened to our other updates on the different rankings, you’ll know that their views are very consistent. We all don’t like them, but we all kind of love to talk about them. This is John Byrne with Poets and Quants. You’ve been listening to Business Casual.