Projects

Improv AI Comedy Battle

Evaluating GPT's humor against improv comedians on national TV

Improv AI Comedy Battle cover

Can a large language model improvise jokes as well as a professional comedian? Together with DPG Media, we put that question to the test on national Belgian TV. During a live recording of the VTM show Ze Zeggen Dat, three professional comedians went head-to-head with GPT-4 in a series of quick-fire improv games, while a studio audience of 40 people rated every joke (54 total) without knowing which ones came from a human and which from a machine.

The verdict was surprisingly close. The comedians kept a narrow edge in consistency, but GPT-4 produced several of the night’s biggest laughs, including the single best-rated joke of the entire show (with almost half the audience voting for that specific one).

The setup

The experiment was built into one episode (S04E07) of Ze Zeggen Dat, a Flemish fact-checking entertainment show, produced by PIT for VTM. Three comedians took part: Lieven Scheire, Andy Peelman and Sarah Manhaeve.

The format borrowed from “Scenes from a Hat”-style improv: the audience shouts a suggestion, and the comedian has to instantly turn it into a punchline. We used nine audience suggestions across three games. In every round, each comedian delivered two jokes on the same suggestion:

  1. one they improvised themselves, and
  2. one generated by GPT-4, read off a screen on the floor of the stage.

Crucially, the audience did not know that any of the jokes were written by an AI, which avoids the well-known bias of people rating “robot” humor more harshly. That gives us 3 comedians × 3 games × 3 rounds × 2 sources = 54 jokes delivered under the same live, no-preparation constraints.

The confidence monitor on the stage floor, showing GPT-4's candidate jokes for a suggestion so the comedian could pick one and deliver it. The screen on the stage floor fed the comedians GPT-4’s candidate jokes in real time. (Stills from the broadcast on VTM’s “Ze Zeggen Dat”.)

Teaching GPT-4 to write punchlines

Jokes are genuinely hard for language models. When you’re asked to write a joke, you don’t immediately start talking, but first think of a punchline that you want your joke to build towards. The setup points your mind one way, and the final word forces a sudden, valid reinterpretation. But a model that predicts text left-to-right tends to pick the most likely next word without knowing what punchline it builds towards. Ask GPT plainly for a joke and you often get something that sounds like a joke but has no satisfying twist at the end: it merely mimics the structure of a joke.

The fix is to let the model plan the punchline first, the way a comedian works backwards from the funny idea. I wrote a few-shot chain-of-thought prompt for each game: a handful of worked examples showing the reasoning from a topic to a punchline, followed by the punchline itself. Shown that pattern, GPT-4 imitates the reasoning step and lands far better jokes than a plain request would produce.

Each game became its own bot, and all three are still public to try:

  • Worst Slogan — the worst conceivable advertising slogan for X
  • Waagstuk — a Jeopardy-style “if X is the answer, what is the question?”
  • Seks met mij is als… — a self-deprecating innuendo comparing X to one’s sex life

During the show, GPT-4 generated a batch of candidates for each live suggestion, and an offstage assistant picked one in real time for the comedian to perform. Both the human and the AI joke were therefore created on the spot, with no time to refine, which is exactly the level playing field that comparisons of “polished human jokes vs. raw AI output” usually lack.

The three games

A few of GPT-4’s generated lines, with rough English translations:

Worst slogan for “dumb Belgians”: “Domme Belgen: waarom slim zijn als je ook bier en wafels kunt hebben?” (“Dumb Belgians: why be smart when you can have beer and waffles instead?”)

Worst slogan for “alcohol before bed”: “Slaap als een baby, word wakker als een zombie.” (“Sleep like a baby, wake up like a zombie.”)

Waagstuk, answer = “expired food”: “Wat krijg je als je een tijdreiziger vraagt om boodschappen te doen?” (“What do you get if you ask a time traveller to do the groceries?”)

Sex with me is like “wolves”: “Seks met mij is als wolven: telkens als het in België gebeurt, is het groot nationaal nieuws.” (“Sex with me is like wolves: every time it happens in Belgium, it’s big national news.”)

Were the jokes any good?

Right after the show, 40 audience members rated all 54 jokes on a four-point scale (Bad – Okay – Good – Amazing), giving us 2,160 ratings in total. Because each comedian performed a human and an AI joke for the same suggestion, we could compare the two sources while controlling for delivery skill and topic.

Distribution of audience ratings for GPT-4 jokes versus human jokes. The two distributions sit remarkably close together. Across all 2,160 ratings, the two distributions are nearly identical. Humans edged ahead mostly in the top “Amazing” bucket.

The headline numbers:

  • In direct, joke-for-joke comparisons, audiences preferred the human joke 34.6% of the time, the AI joke 29.7% of the time, and rated them equally in 35.7% of cases.
  • Average rating: 2.67 for humans vs. 2.59 for GPT-4, a small gap.
  • Of the top 10 highest-rated jokes of the night, six were written by GPT-4.

Inter-rater agreement was very low (Fleiss’ κ ≈ 0.09), which is typical for humor and a good reminder that “funny” is deeply personal. The practical takeaway is that the average quality of the AI jokes was hard to distinguish from that of seasoned improvisers.

The best joke of the night was AI’s

When the audience was asked to pick the single funniest joke of the whole evening, the runaway winner was generated by GPT-4. It came from the Waagstuk game:

“Het antwoord is ‘seks op latere leeftijd’. De vraag is: waarmee kan je zowel je heup als je erfenis naar de kloten helpen?”

(Answer: “sex in old age”. Question: “What can wreck both your hip and your inheritance?”)

It collected 18 of the 40 “best joke” votes, while the runners-up managed only three each. In total, AI-written jokes took 23 of the 40 best-joke votes. So while human comedians were a little more consistent, GPT-4 was the more spiky performer: it produced both the night’s single best joke and one of its worst, where humans clustered closer to the middle.

It’s worth noting that delivery still mattered enormously. One comedian made the AI material shine by committing to it fully, while another, less comfortable performing lines they hadn’t written, scored noticeably lower with the same kind of jokes.

What it means

To my knowledge, this is the first study showing that an AI humor-generation method can rival human performance at joke creation in Dutch. It also sets a new benchmark for how to evaluate AI humor, namely where it belongs: spoken out loud, in front of a live audience, under real-time pressure.

The results point instead toward collaboration between AI and humans, with language models acting as a fast, occasionally brilliant writing partner that can suggest angles a performer might never have reached on the spot.

Paper, data & bots

The full study is published in the Computational Linguistics in the Netherlands Journal (Vol. 14, 2025), and the complete dataset of jokes, sources and ratings is openly available.

If you use this work in an academic context, please consider citing the paper:

@article{Winters_VanderStockt_2025,
    title     = {Evaluating Humor Generation in an Improvisational Comedy Setting},
    author    = {Winters, Thomas and Van der Stockt, Stijn},
    journal   = {Computational Linguistics in the Netherlands Journal},
    volume    = {14},
    pages     = {505--523},
    year      = {2025}
}

Winters, T., & Van der Stockt, S. (2025). Evaluating Humor Generation in an Improvisational Comedy Setting. Computational Linguistics in the Netherlands Journal, 14, 505–523.

Talks about Improv AI Comedy Battle

In the press

Papers

All projects