Prof: Internet is racist because of users, search engines

Dan Jackson
Massachusetts Campus Correspondent

  • Prof. Charlton McIlwain analyzed web traffic and search rankings to determine if users were going out of their way to avoid "nonracial" sites.
  • The internet demonstrates a physical manifestation of structural racism, according to a New York University professor.

    In an opinion piece on The Conversation and re-published in the Billings Gazette, Professor Charlton Mcllwain writes about the inherent racism of the Internet, created by in part by search engines and links between sites.

    "[T]he internet does appear in fact segregated along racial lines."   

    Professor Mcllwain first argues that “the internet does appear in fact segregated along racial lines,” saying his research “demonstrates that websites focusing on racial issues are visited less often, and are less visible in search result rankings than sites with different, or broader, focuses.”

    “This phenomenon is not based on anything that individual website producers do. Rather, it appears to be a product of how users themselves find and share information online, a process mediated mostly by search engines and, increasingly, social media platforms,” he writes.

    His work searches for “online analogues” of systemic racism—that is, racism that applies to “cultures and societies at large”—in which “subtle biases permeate culture and society in ways that yield overwhelming advantages for whites, at the expense of nonwhites.”

    “Specifically, I am trying to determine whether the online environment, one completely constructed by humans, systematically produces advantages and disadvantages along racial lines – whether intentionally or inadvertently,” he writes, conceding that this is “a difficult question to approach.”

    However, he begins by assuming that modern technological systems are born of systemically, structurally racist cultures and society, arguing that this makes it possible for such discrimination to be reflected online.

    First, Professor McIlwain used a link analyzation program called Voson to examine web traffic to what Alexa.com characterizes as the Internet's top 56 African-American sites. He then categorized websites by their meta tags, “designat[ing] as 'racial' websites with meta tags including terms such as 'african american,' 'racism,' 'hispanic,' 'model minority' and 'afro.' Sites without those terms in their metatags" he designated "nonracial." 

    “By using website metatags,” he writes, “I was able to distinguish between racial and nonracial sites (and the segregated traffic between them) based on whether the site's producers themselves define the site's identity in racial terms.”

    Once he classified the sites, he examined the links between them.

    “How many of each type of link the data contained would reveal whether bias influenced website producers' decisions. If there were no bias, the number of links would be proportional to the number of each type of site in the data set. If there were bias, the numbers of links would be disproportionately high or low,” McIlwain declared.

    His examination did not yield significant proof to indicate “that any segregation in people's [I]nternet behavior is caused by web producers.”

    “People who travel the web just clicking links on websites at random would not arrive at racial or nonracial sites substantially more or less than they should based on the number of such sites that exist,” McIlwain said. “But people don't just follow links; they exercise their preferences when navigating the web.”

    Next, McIlwain used metrics site Similarweb to look at how people came to various websites and which sites they navigated to next. He found “significantly greater numbers of clicks between nonracial sites, and fewer numbers of clicks between racial and nonracial sites,” which “indicates that users are going out of their way to visit nonracial sites.”

    Similarly, his data showed nonracial sites received much better rankings in search engines. Though this could be explained by users directly navigating to their preferred site, “[i]t's much more likely that people type a word or phrase into a search engine like Google … To quote a conclusion from search optimization firm Brightedge, 'overwhelmingly, organic search trumps other traffic generators.'”

    “In both of these situations, people and search engines steer traffic in ways that give advantages to nonracial websites and disadvantages to racial sites. This approximates what, in the offline world, is called systemic, structural racism,” he concludes.

    On his NYU faculty page, McIlwain says that his “primary interests focus broadly on issues of race and media, particularly within the social and political arena.”

    However, his recent interests “have turned to the intersections of race and digital media, principally as they relate to three primary questions: to what degree can/has the internet and other forms of digital media use lead to increased political participation, voice and influence for people of color?; in what ways might internet use provide greater access to social, professional and economic mobility for people of color?; and in what tangible ways do forms of racial discrimination, disparate treatment and denial of opportunity take place in online environments?”

    McIlwain also blogs at his personal website, primarily linking to his op-eds in other outlets.

    Follow the author of this article on Twitter: @MrDanJackson





    Dan Jackson

    Dan Jackson

    Massachusetts Campus Correspondent

    Dan Jackson is a Massachusetts Campus Correspondent, and reports liberal bias and abuse on college campuses for Campus Reform. He is a junior at Becker College and currently serves as chapter president of Young Americans for Liberty and Campaign Manager for Central MA.

    More By Dan Jackson

    Latest 20 Articles