Hello all, and welcome to the first issue of our newsletter! Each week, we’ll examine a recent story about “AI” that has some relevance for our organizing project. Though in general we are going to stick to locally-focused news stories, the inspiration for this week’s story, about the fallout from Starbucks Korea’s catastrophic AI-generated marketing campaign, was so wild that I felt compelled to share it. Thanks for reading! - Theodora
I was going to write this week’s newsletter summarizing some of the reporting going on around how unbelievably expensive so-called “frontier” AI models actually cost to run (see, for example, Ed Zitron’s reporting on OpenAI’s financials, or Axios’s report that one company spent five hundred million dollars on Claude in one (1) month).
But then I saw this post on the social network Mastodon:
https://mastodon.social/@johncarlosbaez@mathstodon.xyz
The LinkedIn post he’s referring to goes into a lot of worthwhile detail (though, ironically, it does kind of scan to me as LLM-assisted writing), but in brief: Starbucks Korea ran an AI-generated ad campaign in conjunction with the anniversary of the beginning of the the pro-democratic Gwangju Uprising in 1980 – and the backlash has been, from the perspective of Starbucks Korea, catastrophic.
The Gwangju Uprising protests – frequently referred to as the May 18th Democratization Movement, after the day they began – arose in opposition to the expansion of martial law, on the heels of the previous year’s military coup. In response to the protests, the military dictatorship violently suppressed the protestors, killing hundreds, if not thousands, of people.
I’m hardly an expert on South Korean society and political culture, but it seems pretty clear to me that this might not be the best occasion for Starbucks to launch a themed set of tumblers.
But it wasn’t merely a case of a company trying to profit off of public sentiment: the merchandise wasn’t just opportunistic, but wildly disrespectful. As the Korea Herald reported the day after the campaign, “The coffee chain held a promotion for its Tank Tumblers, with the date appearing alongside phrases such as ‘Tank Day’ and ‘Bang on the Desk.’”
For context, not only were many of the protestors killed with tanks, “Bang on the Desk” refers to an infamous cover-up of an activist’s death in custody under that same dictatorial administration seven years later. As The Diplomat reports, the phrase is “instantly recognizable to any Korean adult.”
The outcry was immediate and intense. Historically, Starbucks is extremely popular in South Korea: the country is Starbucks’s second-biggest national market behind the USA. As the LinkedIn post documents, this popularity has taken a massive, demonstrable hit in the wake of the controversy, leading to the firing of the CEO of Starbucks Korea and an official apology by Chung Yong-jin, the chairman of the Shinsegae Group, which owns a majority stake in the company. Several survivors of the massacre filed criminal complaints against the corporation. A few days ago as of writing this, they shut down every Starbucks in the country for a day in order for their employees to “watch recorded lectures on modern Korean history and engage in ‘social sensitivity’ training.”
How did Starbucks Korea let this happen? For one thing, it seems like (in a pattern unhappily familiar to observers of large American corporations) Starbucks Korea has been increasingly flirting with right-wing signaling. Mitch Shin, reporter for The Diplomat, cites in the article linked above an earlier example of Starbucks Korea marketing ignoring national days of mourning. And Chung Yong-jin, the chairman of the Shinsegae Group, has made his right-wing allegiances explicit in the past, deploying violent anticommunist rhetoric on his social media accounts. Still, as Chung’s apology indicates, everyone involved knows this was a dramatic overstep, even if their goal had been to signal right-wing affinities.
This isn’t a story exclusively about AI (though no story about AI really is), but I think this all points to a few things that are relevant for our purposes at the at the AAUP-NC General Intelligence Project.
On the simplest level, as anyone who’s been exposed to slop-slinging White House social media accounts knows all too well, the right wing has a particular affinity for generative AI. In a literal sense, it is an intrinsically conservative technology, based as it is on recycling pre-existing material; moreover, LLMs are demonstrably capable of spitting out conspiracist and far-right garbage. (The most spectacular example of this might still be the time Elon Musk’s nonconsensual pornography machine and also-ran chatbot “Grok” started calling itself Mecha-Hitler.)
On the institutional level, it’s interesting case study in the insufficiency of so-called “humans in the loop” – the people responsible for checking the outputs of AI. There were definitely humans in this particular loop: the ad campaign seems to have passed through at least four levels of corporate review before being finally approved. Again: how does something like this happen?
On the one hand, AI gives Starbucks Korea and the Shinsegae Group a scapegoat – what the financial analyst and writer Dan Davies calls an “accountability sink.” As Mandy Brown summarizes on her blog, accountability sinks are “structures that absorb or obscure the consequences of a decision such that no one can be held directly accountable for it.” Perhaps all four or five levels of management that approved this campaign knew exactly what they were doing, and they simply flew a little too close to the sun. Perhaps they even knew generative AI was an excuse they could reach for if they were to get in trouble.
But the idea that they overreached - that they misread the situation; that they were divorced, in some sense, from reality - itself ties back the problems we’ve begun to observe with generative AI use. While the research is still in its early stages, the available evidence strongly indicates that people who use AI trust it too much – specifically, that “AI fluency creates a new failure mode: Wrong answers in delivered in flawless prose get accepted.” The appearance of a good answer is enough to make frequent users of AI believe it’s a good answer. (If they even look: according to the LinkedIn post, several of the people responsible for approving the campaign didn’t even open the relevant attachement.)
This points to one of the major flaws with the “human-in-the-loop” model of AI safeguards: by outsourcing their work to an LLM, humans lose the skills necessary to check the LLM’s output. As writer Cory Doctorow put it in 2024: “Presented with conclusions, rather than problems to solve, experts lose the facility and familiarity with how all the factors that need to be weighed to produce a conclusion fit together.”
And this decline in competency is frequently exacerbated by the problem of LLM sycophancy, which arises as a result of frontier AI business models. In order to increase time-on-device, companies like OpenAI (whose chatbots are used by 40% of South Korean smartphone owners, according to a study cited in a Politico article from earlier this year) train their LLMs to be extremely supportive and friendly — “even,” as a paper in Science puts it, “when users [engage] in unethical, illegal, or harmful behaviors.” Anecdotes of executives persuaded of their brilliance by overfriendly chatbots are increasingly frequent. (My favorite is perhaps the videogame publishing company CEO who attempted to get out of a $250 million contract by ignoring his lawyers and running with the advice of ChatGPT; shockingly, the chatbot’s plan did not work.)
And so I don’t think we can blame AI for either the catastrophe of Starbucks Korea’s marketing strategy, nor can we blame it for the callousness and rightward drift of any given corporate culture. (Again, I’m not remotely an expert on South Korea, let alone their corporations.)
But AI isn’t incidental to the story, either, even if the specifics of its role in the campaign won’t ever be clear. Maybe Starbucks Korea is just using generative AI as a post-facto excuse to launder corporate right-wing ambitions. Maybe they really didn’t realize their AI-assisted ad campaign was going to be a total catastrophe. Maybe they didn’t check the AI output at all!
But in all of these cases, AI is implicated. The “humans in the loop” didn’t stop the campaign from moving forward, and the excuse that the campaign was machine-generated wasn’t enough to stop the PR catastrophe. There’s a lesson in here for the executives and administrators attempting to force workers and students to outsource our thinking to AI. It’s up to us to make sure they learn it.
Thanks for reading the AAUP-NC Genuine Intelligence Project newsletter! Subscribe for free to receive new posts and learn about what we’re doing.