ChatGPT, the world's most famous chatbot, was released in late November of last year. The initial reaction was surprise, followed almost immediately by fear about the implications, most notably that it would generate school essays for dishonest students. OpenAI, ChatGPT's parent firm, provided what many users hope will be the antidote to the poison yesterday, almost exactly two months later.
OpenAI's latest creation is a "classifier for indicating AI-written content," and it's as simple to use as one could hope: Copy and paste text into the box, then click "Submit" to see the result. But if you're expecting a direct answer, you'll be disappointed. Instead, it provides the text a classification ranging from "extremely unlikely" to "unlikely," "unclear," "maybe," or "likely AI-generated."
In other words, it's like having one of those vexing conversations with your doctor; you'll never receive a clear answer, thus your doctor will never be technically incorrect.
Fortunately, OpenAI is not concealing the classifier's inaccuracy. "Our classifier is not completely reliable," the tool's introduction page states. We're told that when given a "challenge set" of texts, it gave false positives 9 percent of the time.
However, in Mashable's tests, it proved much less reliable than in select constrained scenarios. In purposefully difficult settings, it almost always returned erroneous answers in both directions, claiming that AI-written text was unlikely to be AI-written and that human-written text was almost certainly AI-written. Perhaps the difference is that we were attempting to deceive it.
Here's how things turned out:
It does not confuse classics with ChatGPT AI outputs.
First, the good news: this tool does a decent job of doing the bare minimum by avoiding mislabeling masterpieces of the English language as AI outputs. We used bits from Shakespeare, the King James Bible, Ulysses, and other works, and it worked perfectly. It deemed them all "extremely unlikely" to have been written by AI. http://sentrateknikaprima.com/
This excerpt from Tennyson's "The Lady of Shallott," for example, was suitably classed.
The lesson is that while this computer may not notice that it has been fed a masterpiece of unparalleled beauty and depth, it does not declare, "I think a robot could have created this."
