They Asked AI To Write A Stand-Up Show

We asked a professional comedian to deliver some jokes written by artificial intelligence on stage. What happened reveals a lot about just how much machines understand the very human sense of humour.

Karen Hobbs was more nervous than usual before this particular gig. A well-known circuit comedian, she’s accustomed to the UK’s often bruising stand-up comedy scene. It’s eclectic, unpredictable and famously short on pity-laughs. Hobbs has tackled some of the most unforgiving rooms in Britain, from major London theatres to the back rooms of rural pubs. She has even triumphed within the dreaded competition circuit, in which a merciless audience votes in a gladiatorial popularity contest for the funniest gags.

But this Thursday night in late June, above the Covent Garden Social Club bar in Central London, Hobbs was about to attempt something totally new. She would take to the stage equipped not with her usual material, but with a stand-up set written for her by the AI platform ChatGPT. Most daunting of all, she would follow three comedians doing their actual, human material.

But when it comes to art, it’s debatable whether or not generative AI, by its nature, can be truly creative. Large language models (LLMs) like ChatGPT work by processing billions of lines of text scraped from the internet and other sources, unpacking the patterns and relationships between words and sentences. Using that data, AI generates responses that are, statistically, the most likely answer to a given prompt. That means these AI tools can only replicate information that already exists in some form, though it can result in never-before-seen combinations of ideas. Whether that counts as creativity is a philosophical question, one currently without a satisfying answer.

To understand whether AI tools can really demonstrate humour, Alison Powell, an associate professor of communications at the London School of Economics who studies AI’s influence on our media, insists that we must first ask ourselves a question: “How do jokes work?” Powell herself cut her teeth in improv comedy, a scene that is arguably more brutal still than the world of stand-up. In improv there is no room for planning, and the comedian has nothing but their instinctive response to an audience prompt.

You might assume comedians have little to worry about if AI is so fundamentally derivative, but there are major secondary risks for creatives. “Comedians should be concerned about data theft and regurgitation, because many of the generative AI tools, especially ChatGPT, are being trained on content on the internet,” Powell says. “This means people’s writing and creativity are acquired from the internet without permission.”

But pilfering jokes in itself isn’t the only worry. As AI improves, so too might its ability to compete. “This would be a concern for a young comedian because if they get better at telling jokes, the models get better,” she says.

OpenAI has been mired in controversy for its alleged use of copyrighted content taken from behind paywalls to train its ChatGPT algorithms. Back in April, eight national US newspapers led by the New York Times banded together to sue the company, who they accused of “purloining millions” of dollars’ worth of intellectual property. (OpenAI did not respond to the BBC’s request for comment, but the company maintains its processing of copywritten material constitutes fair use.)

“One way that AI can tell jokes is to do what any five-year-old does – repeat a successful joke that they have heard, or try to make an obvious variation of it,” says Les Carr, a professor of web science at the University of Southampton, who dabbles in stand-up comedy in his spare time. “So, comedians – who have spent the last five years putting more and more of their content on TikTok, Instagram and YouTube to gain a following – should be concerned about OpenAI and Google and Facebook stealing their work in the same way that authors and artists are.”

For Carr, a central concern is that AI will benefit from the way we tend to use the internet. “Jokes are something that people love to share on the internet or on social media, and so it is very difficult to say where a chatbot joke came from – did it make it up, or did it just repeat it off of ‘/r/Jokes‘ [a comedy forum] on Reddit?”

When Hobbs prompted the AI to generate the set for her, she ran into a strange problem. ChatGPT defaulted to writing in the voice of a male comedian – joking about impatience with its apparently shopping-obsessed girlfriend. When she asked the AI to rewrite the jokes in a female voice, it shifted the shopping-obsessed girlfriend to the first person.

The gags ChatGPT produced for Hobbs lean into crude, lazy stereotypes around millennial women. “My social life, it’s booming. If, by booming, you mean my best friend is a potted plant named Wilson,” the AI wrote. The joke is clichéd one-liner, as the AI meshes the vague idea that young women enjoy pot plants alongside the odd reference to the Tom Hanks film Cast Away – with his volleyball friend “Wilson”.

It’s no surprise that these models struggle to deliver on satisfying builds and punchlines, says Michael Ryan, a masters student at a Stanford University AI expert who has researched AI’s impact on comedy. “A well-done stand-up bit can lead the audience through a funny story all the way to a hilarious punchline,” he says. “The whole time the comedian knew exactly where he or she was going with the joke and brought the audience there.”

ChatGPT and its brethren don’t have the ability to process that kind of information. Unlike a human comedian, AI can’t adapt in real time, at least not the AI tools currently available to the public. “This is not how modern LLMs work,” Ryan says.

However, that could change. Research is already ongoing to give AI a greater understanding of the world around it. “Researchers are already working to perfect audio capabilities AI models which will help with understanding social factors and being able to adapt to an audience, as well as with comedic timing,” Ryan says.

Ryan co-led a major analytics project to test the limits of AI-generated jokes. While his research shed light on its limitations, Ryan believes that the explosive growth of the technology will produce LLMs with an uncanny ability to generate humour in the near future. “I believe we are going to see genuinely funny AI comedy sets in the next few years,” he says.

It may be some time before AI can replicate the on-stage tasks that real world comedians have mastered for centuries. When it comes to writing a single funny joke however, researchers have already made progress.

In 2023, screenwriter Simon Rich wrote an article for Time magazine about his experience using an unreleased OpenAI model called code-davinci-002, developed specifically for creative tasks. Rich collaborated with two other writers on a book of poetry penned by the AI (and later read aloud by Werner Herzog), but not before he asked the machine to spit out some jokes. The results were so good, Rich said, that it gave him nightmares.

Rich fed code-davinci-002 a series of headlines from satirical website The Onion and asked it to produce some satirical headlines of its own. “Budget of new Batman movie swells to $200m as director insists on using real Batman,” the AI wrote in one favourite example. Then there was “Story of woman who rescues shelter dog with severely matted fur will inspire you to open a new tab and visit another website.” Humour is subjective, but the robot jokes went far beyond Rich’s standards for a laugh.

While researchers investigate how to mimic the awareness of context in AI, Karen Hobbs – back in London’s West End – was becoming all too aware of hers. As the host began to warm the audience up, she discovered – much to Hobbs’s horror – that a significant chunk of the crowd on the right-hand side of the room had never been to a comedy gig before. Whatever the AI came out with would be one of their first experiences with the medium.

But perhaps the audience might like what they were about to hear? Drew Gorenz, a PhD student at the University of South California, specialises in digging into the psychology of what exactly makes things humorous. Quite literally, he’s in the business of explaining why the joke is funny.

He set out to pit human-generated jokes against their digital counterparts, and found that, by-and-large, the AI jokes won out when put to more than 200 readers.

Gorenz’s methodology was a little like the online party game Quiplash, where contestants are given a sentence with a gap or an acronym as a prompt, and compete to give the funniest answer. “It is not writing [US comedian] John Mulaney-level jokes, but compared to regular people, its jokes were rated in the top 63rd to 87th percentile depending on the prompt we gave it,” he says.

Gorenz believes that LLMs are “unappreciated” in their ability to write comedy, and that the output of the AI-generated jokes can only be as good as the prompt that the model is given. “Most people, including comedians, would not perform well if asked on the spot by a stranger to ‘say something funny’. The more specific the prompt, the better the answer,” he says. And some models may be better than others. ChatGPT and Google’s Gemini, for example, are built for general-purpose applications. An AI tuned for humour would probably do far better than the mainstream models at churning out jokes, even if it fell behind at other applications.

Yet, Gorenz makes a key distinction between the jokes and the performer themselves, as stand-up comedy audiences “expect authenticity and vulnerability from their comedians” that might be less crucial in another medium. And, for LLMs that are only just getting to grips with the mimicry of language, could this be too tall an order for our AI ghostwriter?

For Powell, the pursuit of digitising comedy is as futile as it is impractical and unethical. Developing generative AI models requires “huge” amounts of energy and expense”, and “it would probably be cheaper, more interesting and produce way more surprises to invest in young comedians, and in cultural production rather than trying to invest the resources necessary to do the work computationally.

“I think that probably a greater benefit would come from investing in human comedians who have many different kinds of ideas that are not statistically similar to ones that have come before, and who are able to be funny in all the different kinds of human cultures and linguistic context.”

As she took to the stage to perform her AI-written set, Hobbs had given the audience fair warning that their reaction could be make-or-break for comedians – at the hands of their new cyber overlords. “If you do laugh through the whole thing, we’ll all be out of jobs!” she said.

Luckily enough there were only irregular bursts of laughter and, mainly when Hobbs herself made sideways looks at the audience, slightly horrified expressions. “I once gave my wife a glue stick instead of lipstick. She still isn’t talking to me,” Hobbs read out to a baffled audience, waiting patiently for a better punchline.

“Dating is like shopping,” the AI wrote in another particularly gendered moment. “You go out looking for what you want and end up with something you don’t need.” Hobbs added moments later: “I’ve literally never felt more stupid in my whole life.” And this gets at a deeper truth. An AI model can go through the motions of constructing jokes. It may even capture the nuance of a good bit, occasionally. But only a human comedian can suffer through the awkwardness of bombing in front of an audience.

For now, AI models haven’t yet figured out this particular secret sauce. Comedians can breathe a sigh of relief that they won’t have to dust off their CVs just yet.

For timely, trusted tech news from global correspondents to your inbox, sign up to the Tech Decoded newsletter, while The Essential List delivers a handpicked selection of features and insights twice a week.

For more science, technology, environment and health stories from the BBC, follow us on Facebook and X.