Opinion: A generation cheated by AI

Being falsely accused of using AI to cheat wasn’t something I ever expected would happen to me.

When I was in seventh grade, I was selected by my teacher to craft an essay for a writing contest partnered with my middle school. Given there was a chance I would be able to win money through this contest, I was thrilled. In retrospect, I had no idea just how much I would come to love writing these essays after that first year.

In the four years since, I have become a regular participant in the annual writing contest, competing with nearly 150 other students every year. As my writing progressed, so did the payout. What started as a platform to win money quickly became a gratifying experience in which I have grown obsessed with the process of brainstorming, crafting, editing, and perfecting every single essay I have submitted.

This year, I was disqualified.

The contest started out like every other. After the prompt was sent out in September, I spent weeks brainstorming a thesis, finding external sources to cite, and composing then re-composing my essay. Taking a simple prompt and explaining my position on animal captivity, I weaved in the Pope’s Laudato Si, explaining how animal treatment imitates human nature. After weeks of crafting and perfecting, I submitted my essay and settled in for several months of waiting to be notified if I had won.

Since this was my fifth year entering the contest, I thought I knew what to expect. However, the day after submission, I was shocked to learn that the contest board had run my essay through AI detection software and found my essay to be 99% written by AI. I was immediately disqualified. I was shattered, baffled, and wrongly accused with no consideration of my potential to prove my innocence. The AI ​​detection software said I used AI and so that was it. The contest board believed the software to be infallible, and I had no case to make. Furthermore, the board informed me that my essay from the year before was also run through the software and was similarly accused. My dreams of winning this year’s contest were obliterated, simultaneous with my previous performance being illegitimized.

I was given a choice: re-write my essay within 24 hours or be eliminated from the contest. How could I write a new essay in such a short amount of time and have any shot at winning or even expect a different outcome from the AI ​​detection software? The accusation was simply untrue. Feeling defeated but also defiant, I went on a quest to figure out how something like this could happen.

With the rise of artificial intelligence tools like ChatGPT that are programmed with the ability to create content from scratch and write complete essays within seconds, it is no wonder educational organizations are hungry to detect people who have cheated. As a result, dozens of startups claiming to accurately detect these AI cheats spring up to fill the void. Many of these companies claim to have accuracy rates above 95%but it is unclear what the basis of these claims even are. In fact, the company used to screen my essay was previously reported to have found portions of the Bible 100% AI-generated.

I decided to reach out to some of the highly used AI detection companies, including the one that claimed my work was not original. It was no surprise that most of the companies did not respond to my inquiry about how their software works and how false positives occur. In my initial email to the company whose software flagged my essay, I explained my experience with the software, questioning their accuracy claim and attempting to gain insight into the cause of this false accusation. After numerous emails back and forth, all the company could come up with you that maybe my writing was “too formulaic.” Each time I offered to send them my essay to re-scan and provide feedback into the potential flaws in their algorithm, I was met with a repeated claim of a 2% false positive rate. Not once did they seek to gain the information necessary to understand how this could have happened and more importantly showed no interest in improving their software nor validating their claims.

You may think this is just about an essay, but what if my “formulaic” writing is flagged in my college applications or that of my peers? If this can happen to a high school student, can’t this happen to anyone? There may be times when our writing is run through AI detection software, and we will have no knowledge of it. What potential impact could this have on our futures? This experience has made me contemplate if there are more people interested in perpetuating and profiting from AI than there are those trying to protect us from unanticipated harms. While there are a handful of organizations like the Center for Humane Technology focused on how AI impacts people, this seems to be the exception, not the rule. AI has the potential to solve many of the world’s problems, but what if there are just as many or more problems it creates?

I do not think AI is inherently bad, but with AI comes unintended consequences. We should all consider how dangerous AI tools can be when they are inaccurate, unregulated, and leveraged without full knowledge of how they actually work. I have chosen to write my story in hopes of reclaiming the future I did not want AI to decide. While the board finally reversed their decision to disqualify me, the damage was already done. My essay did not win the contest this year, but if this article succeeds at raising awareness of the issues with unregulated AI, perhaps I am the real winner after all.

Author’s note: This article was written without the use of AI.

Related