Are ChatGPT Essays Detectable: The Truth Behind AI Detectors
ChatGPT Essays Can Be Detected
ChatGPT is a powerful and free AI tool that has changed the way we work. It has also been used by many students to do their homework, especially essays. So, can teachers and professors tell if a student has used ChatGPT to write their essays?
5 Indicators Of ChatGPT Essays
ChatGPT can produce coherent and fluent sentences that sound natural and almost human-like. Here are a few indicators that we have noticed that might tell if a sentence or essay is AI-generated.
Wrong Content
One of the most obvious and serious flaws of ChatGPT it won’t admit that it is wrong or lack information about a certain subject.
Since ChatGPT does not have any reliable sources of information to draw from, it may just make up facts or hallucinate facts for an essay. For example, in an essay about the history of the United States, ChatGPT may write something like this:
The United States of America was founded in 1776 by a group of patriots who fought against British rule. They declared their independence and formed a new nation based on the principles of democracy, freedom, and equality. The first president of the United States was George Washington, who led the country through the Revolutionary War and the Constitution. The United States expanded its territory through various wars and treaties, such as the Louisiana Purchase, the Mexican-American War, and the Alaska Purchase. The United States also faced many challenges and conflicts, such as the Civil War, the Great Depression, and the World Wars.
At first glance, it seems like there’s nothing wrong with this essay but if you read it carefully, you might be able to spot a few mistakes:
- The United States was not founded in 1776, but in 1789, when the Constitution was ratified.
- The patriots did not form a new nation based on democracy, freedom, and equality, but rather a confederation of states with a weak central government.
- George Washington was not the first president of the United States, but the first president under the Constitution. The first president of the United States was John Hanson, who served under the Articles of Confederation.
- The United States did not expand its territory through the Alaska Purchase, but rather through the Alaska Purchase Treaty.
- The United States did not face the World Wars as a single challenge or conflict, but rather as two separate and distinct wars.
Vague Information
Unless specifically prompted to do so, ChatGPT will always give a generic answer to most questions.
You can easily identify this mistake since ChatGPT usually uses vague or generic terms, such as “some people”, “many studies”, “in recent years”, etc., without providing any sources or evidence to back up its claims. For example, ask ChatGPT to write an essay about climate change, without further prompting, we might get something like this:
Climate change is a serious issue that affects everyone on the planet. Some people believe that it is caused by human activities, such as burning fossil fuels, deforestation, and agriculture. Others think that it is a natural phenomenon that has nothing to do with humans. Many studies have shown that climate change has negative impacts on the environment, such as rising sea levels, melting ice caps, extreme weather events, and loss of biodiversity. In recent years, there have been several initiatives to combat climate change, such as renewable energy sources, carbon taxes, and international agreements.
Although there’s nothing wrong with this paragraph, it does not provide any specific or accurate information on the topic. It does not cite any sources or evidence for its claims, nor does it explain how or why climate change affects the environment or what initiatives have been taken to combat it. In short, these kinds of answers bring no value to readers.
Lack Of Coherence
Another weakness of ChatGPT is that it sometimes fails to maintain a consistent and logical flow of thought throughout the essay.
It may jump from one idea to another without proper transitions or connections, or it may repeat or contradict itself within or across paragraphs. It may also lose track of the main argument or thesis statement of the essay and deviate from the topic or purpose of the writing. For example, here’s part of an essay about Social Media written by ChatGPT:
Social media is a popular and influential form of communication in the modern world. It allows people to share their opinions, experiences, and interests with others online. It also enables people to stay connected with their friends, family, and colleagues across different locations and time zones. Social media has many benefits for individuals and society, such as enhancing social interaction, promoting democracy, and spreading information.
However, social media also has some drawbacks that need to be considered. One of them is cyberbullying, which is the use of social media to harass, threaten, or humiliate someone online. Cyberbullying can have serious consequences for the victims, such as depression, anxiety, low self-esteem, and even suicide. Another drawback is privacy invasion, which is the exposure of personal or sensitive information online without consent. Privacy invasion can lead to identity theft, fraud, blackmail, or other crimes.
Social media is not only a tool for communication, but also a platform for entertainment. Many people use social media to watch videos, play games, listen to music, or follow celebrities online. These activities can be fun and enjoyable, but they can also be addictive and distracting. Some people spend too much time on social media and neglect their other responsibilities or interests in life. They may also develop unrealistic expectations or comparisons based on what they see online.
This paragraph lacks clear coherence or structure. It starts with a general introduction to social media, then switches to two drawbacks of social media (cyberbullying and privacy invasion), then switches back to another benefit of social media (entertainment). These points are not related or relevant to each other or to the topic of the essay.
Repetition
If you have read enough paragraphs written by ChatGPT, you’ll see the same phrases, words, or expressions being used again and again.
Here are a few words or phrases that we have noticed:
- Whether or not
- Delve into
- It’s important to remember
- Elevate your…
- However
- In summary
- In conclusion
We are not saying that if an essay has these words they are written by ChatGPT but if they are used in the same way throughout the essay then it might be an indicator that the essay was written by ChatGPT.
Predictable Pattern
This is a more subtle, and hard-to-notice indicator. ChatGPT tends to follow a certain pattern of ending each paragraph with a positive summary, regardless of the tone or content of the essay.
It may use words or phrases such as “in conclusion”, “therefore”, “thus”, “as a result”, etc., to wrap up the paragraph with a final statement that highlights the main point or argument of the paragraph. For example:
Animal rights are the rights of animals to live free from human exploitation, abuse, and harm. Animal rights activists believe that animals are sentient beings that deserve respect and compassion, just like humans. They argue that animals have intrinsic value and dignity that should not be violated by human actions.
However, animal rights are often violated by human activities that cause suffering and death to animals. Some examples are animal testing, factory farming, hunting, fur trade, and circuses. These activities are unethical and cruel because they treat animals as objects or commodities rather than as individuals with feelings and interests.
Therefore, animal rights are important and necessary for a humane and just society. By respecting animal rights, we can protect animals from harm and injustice. We can also improve our own moral character and well-being by being more compassionate and responsible towards other living beings.
This paragraph ends each sub-paragraph with a positive summary that states the main point or argument of the sub-paragraph. However, this pattern does not fit well with the topic of animal rights, which is more controversial and complex than what ChatGPT presents.
These are not all indicators about a ChatGPT-written essay, there might be more depending on the subjects, and context of the essay.
Can Turnitin Detect ChatGPT Essays?
Turnitin is a tool mostly used by teachers, professors, and researchers to detect plagiarism. And yes, Turnitin can also detect ChatGPT Essays. Turnitin claimed that it can identify content from the GPT-3 and GPT-3.5 language models with 98% accuracy.
However, much like any other AI Detectors, these predictions can’t be fully trusted and should only be used as a reference only.
Should Teachers Trust AI Detectors?
AI detectors are useful tools to catch student cheating by using ChatGPT but they should only be used in cases where students just copy-paste the result from ChatGPT.
If students spend some time editing the result, they might easily bypass AI detection tools.
GPT Zero Notification
In addition, there are also other situations when the student’s writing is so good academically that their entire hand-written essay might be flagged as AI-generated.
Therefore, teachers should not trust AI detectors blindly or use them as the sole basis for grading or evaluating student work.
Instead, they should use them as one of many tools in a holistic assessment of student work.
Conclusion
Some of the ways to detect ChatGPT essays are by looking for incorrect, vague, or irrelevant information; lack of coherence or logic; repetition or redundancy; and unnatural or inappropriate essay patterns. Teachers can use these indicators or tools like Turnitin to check students’ work.
However, there’s not yet a 100% reliable way to detect ChatGPT essays yet. In the meantime, teachers and professors should educate their students on how to use ChatGPT responsibly and ethically and designs new homework to promote critical thinking instead of traditional assignments.