AI Detection Tools Don't Work Anyways.....
From Abhivardhan, our President
I have been following Dr Rachel Barr for more than an year now, which is why I am sharing this video of hers.
Her posts on neuroscience are always helpful. But this take of hers on LLM slop is something one should go through.
Let me do a breakdown as well as what I understand.
1️⃣ LLM Slops show a tendency of "underdeveloped narratives", which show decoupling of skill-level correlates.
What does this mean?
LLMs, despite their impressive general language abilities, produce "slop" in narrative tasks because they prioritize surface-level coherence over deep, meaningful storytelling or idea-telling. This is evident in their tendency to generate predictable that lack the tension / context-consciousness or diversity of human narratives.
This brings me to point 2.
2️⃣ LLM texts are terrible in reiterating "disproportionately well-developed ideas with lower writing skills", so when you see them in writing - it will give you a narrative illusion that - man, either this text is an AI slop or the idea discussed is too mediocre.
And yes, it affects a lot of non-native speakers including me. I still do strict proof-readings when I use AI-generated summaries. Had a conversation on this with fellow ISAIL members on t
his as well, including Shruti Kakade and others many times before.
3️⃣ AI detection tools, designed to identify whether text is human- or AI-generated, analyse patterns like word choice, sentence structure, or n-gram frequencies. These are rooted in distributional proximity—how closely text resembles patterns in training data labeled as "AI-generated" or "human-written." However:
- Lack of semantic grounding: These tools don’t truly "understand" meaning or intent. They rely on probabilistic models, which can misinterpret nuanced, context-specific, or creative text. For example, a human mimicking AI’s formulaic style might be flagged as AI-generated, while an AI producing highly human-like text might slip through.
- False positives are costly: Even a single false positive—flagging human text as AI—can have serious consequences, like accusing a student of cheating or rejecting a valid job application.
That being said, I think we should not judge people if they are using AI-generated text. But we should definitely ask them - hey - have you even done some proof-reading? Have you checked what are you presenting?
This should be more than enough. I think AI-generated text should still be used, but not bluntly, and poorly.
What do you think?