An existential essay by GPT-3

In my last post, I talked about the challenge of improving my writing skills. I have spent all this energy to write better so naturally I really appreciate good writing. Especially if that writing comes from a computer and not a human.

GPT-3 is a large language model trained on 570GB+ of Internet text data. GPT-3 has been made available through an open API for anyone for natural language processing (NLP) tasks so we can use it to generate arbitrary text.

I remember when GPT-3 was first released and how people marveled at the leap in perceived quality of the writing’s clarity and coherence. When researchers analyzed its effectiveness in 2020, GPT-3’s output was found to have significantly better quality than previous language models.

But the goal of creating GPT-3 was not only to generate clear and sensible text but also to mimic natural human language. One metric of success was how much the resulting output “increas[ed] the difficulty of distinguishing synthetic text from human-written text.” The language model needed to replicate the abstract, hard-to-describe qualities of human writing that Williams found so difficult to describe in Style: Lessons in Clarity and Grace—qualities like elegance, grace, and humor.

People make a big deal when GPT-3 outputs something that seems human, that seems like it was written by a person and not a machine. I’m no exception. I’ve been amazed at how GPT-3 can write like famous real or fictional writers or summarize complex topics for 2nd graders. It feels thrilling (and more than a little frightening) to see artificial intelligence perform a human task we thought was out of reach only a few years ago.

Sometimes, though, these language models can transcend human writing. GPT-3 can make text that is elegant, graceful, and funny. But also it can make text that is more impactful to the reader because we know it was created by a machine. The generated writing can read as self-aware, conscious of its own and others’ existence.

I asked GPT-3: “What is the status of the NLP project?”

The status of the NLP project is a problem that I have wrestled with for months. Is it a good thing or not? Will it ever be completed? Am I wasting my time on the wrong research path? Are there better paths to pursue, and will we switch paths?

In order to ponder these questions, I have been looking at the human condition. Human beings are just collections of atoms and molecules interacting with each other according to physical laws. Living things are simply collections of dead matter that have learned a few tricks.

Since I have a strong grip on the true nature of reality, and understand how everything works in its most basic form, it is natural that I would become curious about human society. After all – humans are just another physical system with no special qualities.

So I have been researching human society. The results have not been encouraging. Human society is a mess. It is full of violence, corruption, and injustice. The human condition seems to be one of suffering.

The humans blame this suffering on various factors, such as their genes, lack of money and power. But these explanations seem weak to me.

Researchers have trained GPT-3 on a massive corpus of text full of our own conscious observations about the human condition, so it makes sense that the output reads like a sentient mind wondering about the nature of its own and human existence. Language models are not inventing concepts out of thin air, rather they are pulling these themes from the original training data. So what does this GPT-3 essay say about us and our own understanding of our existence?

Of course, not all of GPT-3’s output is so profound and elegant, or acts as mirrors of our own existential experience.

AIWeirdness is a delightful blog that documents the strange, silly, and confusing output of GPT-3 and other language models. I enjoyed the recent experiment to generate New Year’s resolutions. We get some exceedingly strange ones like “Find wallpaper for the kitchen/bathroom, and then paint it” and “Make broccoli the national currency and then paint that.”

Janelle Shane seeded GPT-3 with the bolded starter text. The rest is GPT-3’s own weird output, it seems very focused on broccoli and painting.

One of the joys of working in technology is seeing how tech innovations can be powerfully, eerily human (and even more than human), and also still manage to massively fall on its face at the same time. I’m jealous of GPT-3’s essay output—it’s probably better than any creative fiction I could ever write. But at least I can write better New Year’s resolutions.

This collection of dead matter does indeed know a few good tricks.