AI content detectors. Are they worth it? | AI in business #38

Do you remember when a few years ago we got excited about the first AI-generated articles that barely resembled human prose? Today, when AI can write almost poetic essays, many people wonder how to distinguish between the work of a machine and that of a human. And OpenAI, the company behind the success of ChatGPT, has shut down access to its AI detector for good due to its ineffectiveness.

Today, developers of AI content detectors present them as tools to guard authenticity. The question is, are they worth the trust and investment? In this article, we’ll look at how AI content detectors work, why they might go extinct, what challenges they bring, and the ethical dilemmas they pose.

AI content detectors

AI content detectors are based on language models similar to those used to generate AI content. They can be divided into those whose task is to check the origin of images, texts, and music generated with the support of artificial intelligence. Each type of “AI detector” works slightly differently, but none of them can distinguish with absolute certainty between human-created and AI-generated content.

AI-generated image detectors are playing an increasingly important role due to the media’s power to generate fake news. They analyze anomalies, distinctive styles and patterns, and look for signs left behind by models such as DALL-E.

Prominent among the detectors used to identify images is the “AI or Not” tool from Optic, which uses image databases generated by Midjourney, DALL-E and Stable Diffusion. While results are uncertain, it is a step toward developing more precise identification methods in the future.

Source: AI or Not (https://www.aiornot.com/)

Behind the operation of AI detectors that recognize AI-generated texts are advanced algorithms that analyze the structure and word choice of the text, and then recognize AI-specific patterns. They make use of:

• classifiers – an algorithm that classifies text and checks style, tone and grammar. For example, a product description that could fit any product of its type might be classified as an AI creation,
• embeddings (embeddings) – numerical representations of words allow machines to understand the context of their use. It is thanks to them that the program “understands” that a text with a monotonous selection of words can be the work of AI,
• perplexity – which is a measure of the unpredictability of a text. Texts written by humans tend to have higher perplexity, although texts that are inherently simple, utilitarian in typical form, or written by foreigners can be mistakenly classified as AI-generated,
• diversity (burstiness) – this factor describes variability in sentence length and structure. Humans tend to write more varied texts than artificial intelligence.

The above-mentioned elements together are used by AI content detectors to assess whether we are dealing with man-made or machine-made text.

Why use AI content detectors?

AI content detectors work in a variety of fields – from education to marketing and recruitment. Here are the top reasons to have them as a tool to aid in evaluation, but not as definitive proof of whether content has been generated:

• Identification of AI-modified photos depicting well-known people – to detect whether the photo depicts a real situation,
• Preventing disinformation – In the context of fighting disinformation, effective AI content detectors help social media moderators detect spreading false information to identify and eliminate repetitive content generated by bots,
• Limiting the publication of low-value texts – AI content detectors can help publishers reject texts containing generic information generated by ChatGPT, Bing, or Bard after typing a simple query.

However, it is worth remembering that the origin of the text is not the basis for Google’s lowering of a site’s ranking. Google’s Search Center blog states that it is key for Google to “reward quality content regardless of how it is created […]. Automation has long been used to generate useful content, such as sports scores, weather forecasts and transcripts. AI can open up new levels of expression and creativity and be a key tool to support the creation of great web content.”

Unreliability of AI content detectors. Reality or myth?

Although AI content detectors are ubiquitous, their effectiveness can be questionable. The main problems are:

• low efficiency in detecting AI content,
• problems with false positives, as well as
• difficulties in adapting detectors to rapidly diversifying and improving new AI models.

Tests conducted by OpenAI showed that their classifier recognized GPT-generated text only 26% of the time. An interesting example of the unreliability of generators can be seen in an experiment conducted by TechCrunch, which showed that the GPTZero tool correctly identified five out of seven AI-generated texts. While the OpenAI classifier only identified one.

Source: GPTZero (https://gptzero.me/)

In addition, there is a risk of receiving a false positive, that is, identifying text written by a human as AI-generated. For example, the beginning of the second chapter of Miguel de Cervantes’ Don Quixote was marked by the OpenAI detector as most likely written by artificial intelligence.

While errors in the analysis of historical literary texts can be treated as an amusing curiosity, the situation becomes more complicated when we want to use detectors as tools for evaluating texts. The U.S. Constitution was marked by ZeroGPT as 92.15% written by artificial intelligence. And, according to a study published by researchers at Stanford University, 61% of TOEFL essays written by non-native English-speaking students were classified as AI-generated. Unfortunately, there is no data on how high the percentage of texts falsely classified as positive in other languages is.

Another issue is the change of classification on subsequent runs of the detector. This is because it often happens that a detector such as ZeroGPT or Scribbr changes the classification of text fragments, which it marks as AI-generated once and as human-written another time.

Source: Scribbr (https://www.scribbr.com/ai-detector/)

AI image and video detectors are primarily used to identify deepfakes and other AI-generated content that can be used to spread disinformation.

Current detection tools such as Deepware, Illuminarty, and FakeCatcher do not provide test results on their reliability. In the legal context of detecting AI-generated visual material, there are initiatives to add watermarks to AI images. However, this is a very unreliable way – you can just easily download an image without a watermark. Midjourney takes a different approach to watermarking, leaving it up to users to decide whether they want to watermark an image in this way.

Avoiding AI detection. Is it possible and how?

Entrepreneurs should be aware that AI content detectors are not a substitute for human quality assessment and are not always reliable. Their practical maintenance issues may pose considerable difficulties, just as trying to avoid getting your content classified as AI-generated. Especially when the AI is simply a tool in the hands of a professional – that is, it is not “content generated by AI,” but rather “content that was created in collaboration with AI.”

It is relatively simple to add someone to the generated materials so the way they are created is really difficult to detect. If the person who uses generative AI knows what effect to achieve can simply manually tweak the results.

The basic question lies in the reason behind our wants to avoid detection if the content was generated by AI.

• If this is an ethical issue and concerns, for example, the authorship of published scientific research – one is left to rely on the professional ethics of the scientist and the responsible use of AI-based tools.
• If the employer wishes employees to opt out of using AI – there remains a contractual arrangement for the use of generative artificial intelligence.

It also raises the question of whether we want to promote responsible use of AI through bans and detractors (ZeroGPT and GPTZero!), or through an appreciation of transparency, trust-building and honest use of advanced technologies.

Source: ZeroGPT (https://www.zerogpt.com/)

Summary

The answer to the question of whether AI content detectors are worth using is far from clear. AI content detectors are still in development, and their future is difficult to predict. One thing is certain – they will evolve along with the development of AI technology. Advances in AI, including the increasing ability of language models to mimic human writing style, means that AI content detection could become even more complicated. For businesses, this is a sign to follow these developments and not rely solely on tools, but on their assessment of content and its suitability for the purpose for which it was created. And to use the rapidly developing artificial intelligence wisely.

If you like our content, join our busy bees community on Facebook, Twitter, LinkedIn, Instagram, YouTube, Pinterest, TikTok.

AI in business:

1. Threats and opportunities of AI in business (part 1)
2. Threats and opportunities of AI in business (part 2)
3. AI applications in business - overview
4. AI-assisted text chatbots
5. Business NLP today and tomorrow
6. The role of AI in business decision-making
7. Scheduling social media posts. How can AI help?
8. Automated social media posts
9. New services and products operating with AI
10. What are the weaknesses of my business idea? A brainstorming session with ChatGPT
11. Using ChatGPT in business
12. Synthetic actors. Top 3 AI video generators
13. 3 useful AI graphic design tools. Generative AI in business
14. 3 awesome AI writers you must try out today
15. Exploring the power of AI in music creation
16. Navigating new business opportunities with ChatGPT-4
17. AI tools for the manager
18. 6 awesome ChatGTP plugins that will make your life easier
19. 3 grafików AI. Generatywna sztuczna inteligencja dla biznesu
20. What is the future of AI according to McKinsey Global Institute?
21. Artificial intelligence in business - Introduction
22. What is NLP, or natural language processing in business
23. Automatic document processing
24. Google Translate vs DeepL. 5 applications of machine translation for business
25. The operation and business applications of voicebots
26. Virtual assistant technology, or how to talk to AI?
27. What is Business Intelligence?
28. Will artificial intelligence replace business analysts?
29. How can artificial intelligence help with BPM?
30. AI and social media – what do they say about us?
31. Artificial intelligence in content management
32. Creative AI of today and tomorrow
33. Multimodal AI and its applications in business
34. New interactions. How is AI changing the way we operate devices?
35. RPA and APIs in a digital company
36. The future job market and upcoming professions
37. AI in EdTech. 3 examples of companies that used the potential of artificial intelligence
38. Artificial intelligence and the environment. 3 AI solutions to help you build a sustainable business
39. AI content detectors. Are they worth it?
40. ChatGPT vs Bard vs Bing. Which AI chatbot is leading the race?
41. Is chatbot AI a competitor to Google search?
42. Effective ChatGPT Prompts for HR and Recruitment
43. Prompt engineering. What does a prompt engineer do?
44. AI Mockup generator. Top 4 tools
45. AI and what else? Top technology trends for business in 2024
46. AI and business ethics. Why you should invest in ethical solutions
47. Meta AI. What should you know about Facebook and Instagram's AI-supported features?
48. AI regulation. What do you need to know as an entrepreneur?
49. 5 new uses of AI in business
50. AI products and projects - how are they different from others?
51. AI-assisted process automation. Where to start?
52. How do you match an AI solution to a business problem?
53. AI as an expert on your team
54. AI team vs. division of roles
55. How to choose a career field in AI?
56. Is it always worth it to add artificial intelligence to the product development process?
57. AI in HR: How recruitment automation affects HR and team development
58. 6 most interesting AI tools in 2023
59. 6 biggest business mishaps caused by AI
60. What is the company's AI maturity analysis?
61. AI for B2B personalization
62. ChatGPT use cases. 18 examples of how to improve your business with ChatGPT in 2024
63. Microlearning. A quick way to get new skills
64. The most interesting AI implementations in companies in 2024
65. What do artificial intelligence specialists do?
66. What challenges does the AI project bring?
67. Top 8 AI tools for business in 2024
68. AI in CRM. What does AI change in CRM tools?
69. The UE AI Act. How does Europe regulate the use of artificial intelligence
70. Sora. How will realistic videos from OpenAI change business?
71. Top 7 AI website builders
72. No-code tools and AI innovations
73. How much does using AI increase the productivity of your team?
74. How to use ChatGTP for market research?
75. How to broaden the reach of your AI marketing campaign?
76. "We are all developers". How can citizen developers help your company?
77. AI in transportation and logistics
78. What business pain points can AI fix?
79. Artificial intelligence in the media
80. AI in banking and finance. Stripe, Monzo, and Grab
81. AI in the travel industry
82. How AI is fostering the birth of new technologies
83. The revolution of AI in social media
84. AI in e-commerce. Overview of global leaders
85. Top 4 AI image creation tools
86. Top 5 AI tools for data analysis
87. AI strategy in your company - how to build it?
88. Best AI courses – 6 awesome recommendations
89. Optimizing social media listening with AI tools
90. IoT + AI, or how to reduce energy costs in a company
91. AI in logistics. 5 best tools
92. GPT Store – an overview of the most interesting GPTs for business
93. LLM, GPT, RAG... What do AI acronyms mean?
94. AI robots – the future or present of business?
95. What is the cost of implementing AI in a company?
96. How can AI help in a freelancer’s career?
97. Automating work and increasing productivity. A guide to AI for freelancers
98. AI for startups – best tools
99. Building a website with AI
100. OpenAI, Midjourney, Anthropic, Hugging Face. Who is who in the world of AI?
101. Eleven Labs and what else? The most promising AI startups
102. Synthetic data and its importance for the development of your business
103. Top AI search engines. Where to look for AI tools?
104. Video AI. The latest AI video generators
105. AI for managers. How AI can make your job easier
106. What’s new in Google Gemini? Everything you need to know
107. AI in Poland. Companies, meetings, and conferences