Last week, a small article appeared in the Globe and Mail. If you didn’t manage to see it, the link is here.
The article provides a brief overview of Alberta universities, including Athabasca. It does not provide any in-depth reporting on individual schools. Despite the assertion in the opening paragraph that “choosing the school that best suits you requires going beyond rankings and reputation and considering the unique culture and educational environment of your potential alma mater,” the relatively brief article does not provide much insight into either the culture or educational environment of Alberta universities, including Athabasca. So, what was the purpose of the article?
AU’s Communications Director, John O?Brien, provided a bit of background and context to the Globe and Mail piece.
The section was not designed to be a piece of journalism, according to Mr. O?Brien, even though it gives every appearance of being one. It is what is called “native advertising”, according to Mr. O’Brien, or a special section in a newspaper that features some editorial content wrapped around the featured advertising space, not the other way around. In other words, a section of the newspaper primarily designed to sell print ads and not provide any in-depth content. The advertising format in the print edition of the paper does not translate to the website view, so only the article content is shown. The author, Ash Kelly, is not a Globe and Mail reporter but is an employee of Discourse Media, a Vancouver-based company contracted by the Globe and Mail to produce the section.
The Globe and Mail has since informed us that, contrary to Mr. O?Brien’s comments, the Canadian University Report is a wholly editorial product, not an advertorial one. It is assigned and reported free of advertiser input and is intended as a combination of feature pieces about issues in higher education, service pieces to help students consider issues before they choose a university, and a series of brief profiles based on student surveys, university data, several sources of publicly available information and other sources, giving students some insight into potential choices of schools across the country. Thirteen reporters–both Globe and Mail staffers and freelance–contributed to the full magazine, with Ash Kelly being the main reporter for the school profile section.”
The article was based, in part, on data from an annual survey that Athabasca and other Canadian universities complete every January. What concerns some AU personnel is that the survey is designed and weighted for a traditional bricks-and-mortar university and contains questions that simply do not pertain to Athabasca; for example, ?describe your student residence experience?. In this particular instance, AU was not approached to answer any questions regarding its research projects, and yet the published results contained a section on university research. Subsequently, several AU Faculty and Board members were puzzled by the section’s ?not applicable/not enough data? designations for areas like Athabasca’s research and student experience, given that AU does excellent research and the company has access to Athabasca’s results from annual surveys such as the National Survey of Student Engagement.
O?Brien cautions against reading too much into this advertorial and its portrayal of Athabasca University, especially in relation to other universities. “Nothing in this article was inaccurate, it simply means that it doesn’t have enough data to give a clear picture of what Athabasca is all about.”
This advertorial, that is based on the survey results, highlights the current trend of ranking systems being used to give a picture of universities.
On another front, the Maclean’s magazine annual University Ranking survey results for 2016 have just been published this week. The Maclean’s University survey has been around since 1991 and the special edition is one of the most popular and eagerly anticipated issues for the weekly news magazine. But how much weight should people give the survey results, and are they an accurate portrayal of Canadian universities?
The Maclean’s survey is part of the relatively recent trend of condensing schools? performance and student experience into statistics and then ranking the schools based on the data. University ranking began in 1983, but the trend did not fully take hold until the 1990s, when a proliferation of school ranking systems (ranging from The Fraser Institute?that rates elementary and secondary schools?to a number of league tables for higher education both within North America and abroad) gained popularity. It is difficult to say, however, whether the weighting and importance attached to them has also increased along with the number of ranking systems. Certainly, education at the post-secondary level is now a high-stakes game. One where competition is often fierce for students to be admitted to programs with limited places, especially at elite schools, and the overall cost of post-secondary education has skyrocketed. Universities and colleges must also compete for government and private funding to create opportunities for growth and development. This has contributed to an atmosphere where there is a lot riding on investments into education and the quality of the overall experience. Perhaps these rankings play into the current mentality.
So what is the big deal about university rankings? Are they truly an unbiased, comprehensive method to provide a snapshot of an institution? Certainly each time new results are released they grab the news headlines. Good results invite bragging rights by the institution and the results are often used to promote a school and boosting its stature. Conversely, a low ranking may hinder a school’s reputation and create a label that may take years for it to recover from. On the surface, using algorithms and statistical methodology should give a foolproof result and the authors of these surveys pride themselves on the quality of their statistical research. But because they are presented in ways that focus on certain aspects of the institution and present them in simplified ways, perhaps what these rankings leave out is as important as what they include.
The Maclean’s survey is the most popular and well-known university ranking in Canada. The magazine asserts that it has tweaked and improved the methodology that it uses over the years. It looks at many factors, including student evaluations of teaching staff, the quality and amount of research produced, and student satisfaction. Maclean’s prides itself on including student stories about the institutions profiled in its annual special edition.
But not everyone is a fan of the Maclean’s report. Some universities are recognizing it is primarily designed to sell magazines, not provide comprehensive results from every single college and university across Canada. Some universities begrudgingly participate and others have refused to provide any data to the magazine. Indira Samarasekera, a former president of the University of Alberta, has argued that too many measures, especially those used for reputational rankings, rely on subjective opinions. Ranking systems reduce university findings to one common core and ignore the fact that the evaluation in like-for-like Faculties can vary widely between universities or even between individual classes and teaching faculty when looking at a single school. Rankings such as the Maclean’s report rely heavily on hard data such as the amount of faculty research and gloss over what cannot be measured, even though the unmeasurable may still be crucial to the university experience, such as community engagement. There have been various critiques of ranking systems, including an article by Malcolm Gladwell in The New Yorker : “The Order of Things: What college rankings really tell us”, that highlight the issues and flaws within them. But, ironically, there has not been much scholarly research to evaluate the criteria used and how the information is compiled. In the case of the Maclean’s survey, a lot of the student satisfaction results are taken from online surveys that are self-referring, meaning that students take the survey voluntarily, rather than being a comprehensive and representative sample in all university populations.
The case of Athabasca University’s unique situation in Canada also illustrates how these rankings can present a bias toward certain institutions. Athabasca, by its nature, is somewhat of an educational anomaly in post-secondary education because, although it has a bricks-and-mortar building as its headquarters, its distance-learning model means that it is excluded from a bricks-and-mortar university culture as in the case of traditional universities. It does not have student residences or sports teams or predominantly face-to-face classroom experiences. And yet many surveys, including the one conducted by Maclean’s, still give much merit to these factors. As a result, Athabasca drops behind the pack. With the increase in the number of traditional universities offering online and distance classes and programs in addition to their traditional framework, it remains to be seen whether the surveys adapt to this changing model of education.
When asked by The Voice in a telephone interview about AU’s stance regarding the rankings, AU’s Communications Director, John O?Brien, said that there is a place for these surveys and ranking systems, but he has no idea whether people pay a huge amount of attention to their results as, to his knowledge, this hasn’t been investigated. He admitted it would be nice for these ranking systems to have a broader mandate that incorporates online and distance learning, and for them to “get into the 21st century.”
So while the Maclean’s survey is likely to be eagerly read out of a sense of curiosity as to who got the top spots this year, it should by no means be taken as the final word or most comprehensive method to evaluate Canada’s universities. University rankings seem to be here to stay?for now?but whether they continue to be relevant in light of how education is evolving remains anyone’s guess.
Carla and her family have recently purchased her first telescope for stargazing. However, they have been frustrated by the cantankerous Calgary weather!
This article has been corrected from a previous version which did not include the Globe and Mail’s information about the nature of the Canadian University Report article.
