ArticlesWriting tools

AI can guess how your message sounds. It can't tell what you actually feel.

The accuracy numbers, why even trained humans disagree with each other, and the new EU law that treats text differently from your face or your voice.

By Samet Durgun · Co-founder of Subtext · 11 min read

In short

AI is good at predicting how a message might come across to a reader, but it has no reliable way of knowing what the writer feels. On fine-grained emotion labels the best specialised models still miss a large share, and trained human labellers often disagree too. In one 2025 study, both LLMs and human strangers fell short of authors’ own reports of their feelings. Under Commission guidelines, the EU ban on emotion inference at work and in schools covers biometric data such as faces and voices, not written text.

If you use a tone checker

  1. Give it the thread, who the reader is and what is at stake, not one line on its own.
  2. Read its verdict as how the words may come across, not as what anyone feels.
  3. Treat any confidence percentage from a general chatbot with caution.

I co-founded Subtext, a tone tool for messages that the post discusses near the end, and say so at the start.

Not sure how your message sounds?

Subtext names the issues and the feeling in your draft, and marks the words.

A tone checker can tell you that a message may come across as cold. It cannot tell you that the person who wrote it is angry. This piece keeps those two claims apart.

I co-build Subtext, an app that helps people read messages and figure out how to reply. So I had a selfish reason to dig through the research on AI tone detection: the benchmarks, the ACL papers, the psychology critiques, and the EU law that now bans part of this field. Weigh my stake accordingly. Every claim below links to its source, so you can check my reading rather than trust it. Near the end, Subtext reads the draft “Good for you.” and shows what it says about a line that can be read two ways.

Tone detection is five different jobs

The phrase sounds like one feature. Under the hood it covers separate problems that get harder as you move down the list.

Sentiment is the easy one. It asks whether a text is positive, negative, or neutral. Every major cloud provider sells it. Google’s API1 returns a score from minus one to plus one along with a magnitude for emotional strength, Amazon Comprehend2 sorts text into positive, negative, neutral, or mixed, and Microsoft’s Azure service3 adds aspect-level opinion mining, so it splits a sentence like “the app is great but support was awful” correctly instead of averaging it into mush.

Emotion detection, the next step, covers anger, joy, fear, sadness, and their many cousins. Harder, as the numbers will show.

Interpersonal tone is what you care about when texting your landlord. Does this sound warm, direct, defensive, accusatory. Researchers were already modelling frustration, formality, and politeness in work email back in 20184, and it worked reasonably well because they treated these as separate dimensions instead of one emotion label.

Then come sarcasm and subtext, which depend on context more than on words. At the bottom sits the writer’s private emotional state, which is the thing most marketing implies and the research least supports.

So when a product page says it detects tone, and most tone checkers never check your tone, my first question is which of these five it means.

The accuracy numbers that never make the landing page

The reference dataset for fine-grained emotion in text is Google’s GoEmotions5: 58,000 Reddit comments that Demszky’s team hand-labelled with 27 emotions plus neutral and presented at ACL in 2020. Its BERT baseline scored a macro F1 of 0.46 across the full label set. Later work6 with stronger models and data tricks pushed that to roughly 0.49 to 0.51. Years of progress, and the best specialised models still miss a large share of fine-grained labels.

Ask a general chatbot to do the same 28-way sort and its benchmark score drops well below the fine-tuned small models, partly because free-form models invent labels outside the taxonomy and don’t calibrate to the test’s thresholds. Which sounds damning until you learn that in one 2025 study7, human judges often sided with GPT-4’s labels over the dataset’s official ones. Sometimes the model disagrees with the benchmark because the benchmark is the weaker party.

Coarse positive versus negative classification scores far higher across the board. The collapse happens exactly where the interesting distinctions live, like annoyance versus anger or disappointment versus disapproval.

Humans disagree about tone too

The part that reframed the whole topic for me is what happens when you check the humans. The GoEmotions paper’s appendix reports agreement between its own human labellers, and for most emotions the Cohen’s kappa sits between roughly 0.1 and 0.5. Gratitude is easy at 0.75. Grief lands at 0.09, close to labelling at random. When trained humans reading the same sentence can’t agree whether it expresses disappointment or disapproval, the benchmark grades a model scoring 0.46 macro F1 against their labels, and that answer key is soft in exactly those places. Kappa and F1 measure different things, so the human figure is no hard ceiling, but it tells you part of the model’s miss belongs to the task.

Cohen’s kappa, GoEmotions’ own human labellers Gratitude: 0.75. Grief: 0.09, close to labelling at random.

Kappa of 1.0 is perfect agreement, 0 is chance. When trained humans can’t agree whether a sentence is disappointment or disapproval, a model’s 0.46 macro F1 is being graded against an answer key that is soft in exactly those places.

It goes deeper. Whose feeling are we even measuring? A dataset called EmoBank8 annotated the same sentences twice, once from the writer’s perspective and once from the reader’s, because those turn out to be different variables. The same researchers found in 2017 that annotating from the writer’s perspective produced better quality overall. And the large SemEval 2025 emotion task9, which covered more than 30 languages from seven language families, deliberately targets perceived emotion, meaning what a general reader would attribute to the text and explicitly not what the writer felt inside.

The sharpest test came in 2025. An ACL study10 asked authors to label their own emotions, then had outside humans and LLMs guess. Both fell short of the authors’ own reports, and the LLMs beat the human strangers almost across the board. Each half of that finding matters. AI is now better than a random person at reading you from text alone, and it still can’t reliably reconstruct what you felt.

One practical consequence shows up in a 2023 comparison in Heliyon11, which ran eight commercial emotion APIs over the same datasets and found them disagreeing with each other, with rankings that shifted between datasets. Any vendor quoting “95 percent accurate emotion detection” without naming the labels, the data, and the language is quoting a number that can’t carry the weight of the claim.

Models ace emotion tests and still miss the person

Two research results look contradictory until you separate what they measure.

In a 2025 study in Communications Psychology12, six models including ChatGPT-4, Gemini 1.5 Flash, Claude 3.5 Haiku, and DeepSeek V3 took five standard emotional intelligence tests, the kind used to assess humans. The models averaged 81 percent correct against the 56 percent human average from the tests’ validation studies. In Nature Human Behaviour13, GPT-4 matched or beat humans on classic theory-of-mind tasks like spotting false beliefs and indirect requests, though it stumbled on recognising faux pas. And in JAMA Internal Medicine14, licensed evaluators read answers to real patient questions and preferred the chatbot’s over the physicians’ 78.6 percent of the time, rating them 9.8 times more likely to be empathetic. I’d keep one caveat in mind. The chatbot’s answers averaged 211 words against the physicians’ 52, and the preference shrank when doctors wrote longer. For people, whether emotional intelligence really matters at work is a separate question.

Meanwhile EmoBench15, which Sabour and colleagues designed to require emotional reasoning rather than pattern recall, still finds a clear gap between models and average humans.

Both findings can be true because knowing about emotions and knowing someone’s emotion are different skills. A model has read more about jealousy, grief, and awkward workplace apologies than any human alive. What it lacks is context about the one person who wrote this specific message.

Four people can type the same words: “Congratulations. I’m happy for you.” One means it. Another resents you. A third is heartbroken and being brave. The fourth is being sarcastic. Every version is identical on the screen, so no text model can separate them, and neither could a human stranger. That information lives outside the words.

Context does most of the work

Take the reply “Fine.”

On its own it could be agreement, disappointment, irritation, indifference, or a door quietly closing. Put it after “You’ve moved the deadline a third time” and the reading narrows fast. Nothing about the word changed. The context did.

Sarcasm research keeps landing on the same conclusion, that even humans often need the surrounding conversation to catch sarcastic intent1617, which is why newer evaluations test models on whole dialogues instead of isolated sentences. The practical rule I’d draw for any tone tool follows directly. Feed it the thread, the relationship, and the stakes. Judging a single message in isolation is asking the model to fail politely.

Tone is cultural, and most models grew up on American English

A 2025 benchmark called BESSTIE18 tested models on sentiment and sarcasm across Australian, British, and Indian English. Performance was consistently worse on Indian English, with sarcasm the hardest case, and models struggled to generalise from one variety to another.

That result matches everyday experience. “Kindly do the needful” reads one way in Indian professional English and another way to a model calibrated on American office norms. After 13 years in Berlin I can add the German version. Software raised on American politeness can read the directness that counts as respect for your time here as cold. Labels like rude, warm, or too direct are cultural judgments wearing an objective costume. A tone tool that ignores who is writing to whom, and in which culture, will confidently misread both sides.

A related bias shows up in content moderation, a different task from sentiment scoring. Sap and colleagues examined widely used hate speech datasets19 and found that trained classifiers labelled tweets written in African American English, sometimes called African American Vernacular English or AAVE, offensive up to twice as often as comparable tweets in other varieties, without annotators ever seeing who wrote them. The gap lives in the toxicity classifier, not the sentiment classifier, so a benchmark like BESSTIE would not surface it. A moderation system trained on that data flags routine AAVE phrasing as offensive more often than the same message worded another way.

Face reading is where emotion AI lost the plot

Everything above concerns text. The face-scanning branch of this industry has a rougher record, and it shapes how regulators now treat the whole category.

In 2019, psychologist Lisa Feldman Barrett and colleagues reviewed over a thousand studies20 and concluded that you cannot reliably infer how someone feels from their facial movements. People scowl while concentrating, smile while anxious, and cry at weddings. The same year, a study ran photos of 400 NBA players through two commercial systems and found both read more negative emotion into Black players’ faces21 than into white players’ at similar expressions.

The industry noticed. Microsoft retired emotion inference from its Face API22 in 2022. HireVue dropped facial analysis from its hiring assessments in 2021 after the digital rights group EPIC filed a complaint with the FTC23. Text tools are a different technology, but they inherit this scepticism, and they earn it whenever they claim to read inner states rather than wording.

The EU drew a line, and text sits outside it

Since February 2, 2025, the EU AI Act’s Article 5(1)(f)24 prohibits AI that infers people’s emotions in workplaces and schools, with narrow exceptions for medical and safety uses like driver fatigue detection. Breaching the prohibited practices carries fines of up to 7 percent of global annual turnover.

The scope matters for anyone building or using these tools. Guidelines from the European Commission tie the ban to biometric data25: faces, voices, keystroke dynamics, body posture. A system inferring emotions from written text is explicitly outside the prohibition26. Analysing how an email might land is legal. Pointing a webcam at employees to score their mood is banned. That line tracks the science fairly well, and for Subtext it translates into a design constraint we would have chosen anyway: text only, no biometrics, no workplace surveillance.

What today’s tools are good for

Grammarly’s tone detector27 is the clearest consumer example, and its framing is the most direct in the market. It analyses word choice, phrasing, punctuation, and capitalization27 to tell you how your message is likely to sound to a reader. Likely to sound. That phrasing is doing real scientific work.

Google, Amazon, Microsoft, and IBM28 design their cloud APIs to process support tickets, reviews, and survey answers in volume. IBM retired its standalone Tone Analyzer in 2023 and folded emotion into its broader language product. These are the right tools for “how do customers feel about our checkout flow” and the wrong tools for “does my text sound passive-aggressive.”

General LLMs are the most capable option for personal messages because they can use context, handle several tones at once, and explain their reading. They are also the worst calibrated, so treat any confidence percentage as a vibe. On the voice side, Hume AI29 raised a 50 million dollar Series B in 2024 to measure vocal modulation, mostly so voice interfaces can respond with fitting pacing and tone. A narrower use, and a more defensible one than claiming to detect hidden feelings.

How this shapes Subtext

Subtext reads the drafts people care about most, like the reply to your boss, your date, or your mother. I let the research settle how that should work.

Subtext lives in the third of the five buckets above, interpersonal tone, which asks whether a message reads warm, direct, defensive, or accusatory to the person receiving it. The output has to be about perception. “This may come across as dismissive” is a claim the evidence supports. “She is angry with you” is a claim it doesn’t, and the ACL first-party study is the receipt. We also let messages carry several tones at once, because real ones do. Warm and defensive is a normal combination, and the newer benchmarks treat emotion as multi-label for exactly this reason.

Evidence beats verdicts. A useful tool points at the words creating an impression, the way “as I already explained” quietly does the heavy lifting in a tense email, so you can decide whether that impression is the one you want. Uncertainty stays visible too. When a message is ambiguous, the ambiguity is often the most accurate finding, and rounding it into a confident label would be fake precision. One real answer shows how that looks in the product. It comes from a single example on a draft made up for it, so read it as a picture of the product, not as support for any study above. On the draft “Good for you.” Subtext marked the whole line and returned the tag Sarcasm, with the feeling Ambiguity beside it. Its note stopped short of a verdict and said the line “can sound dismissive or mocking, depending on context”.

Good for you.

  • Sarcasm: "Good for you." can sound dismissive or mocking, depending on context.
  • Ambiguity: The message could read as warm congratulations or as a curt, sarcastic brush-off.

Short and simple, but "Good for you." can land as a bit cold without context. Here are a few ways to make the warmth clear.

  • Close to yours (Warmth, safe to send): Good for you, I'm happy to hear it. Keeps your short phrase but adds "I'm happy to hear it", which removes any hint of sarcasm.
  • Warmer (Joy, safe to send): That's great, I'm really glad for you. Swaps the ambiguous phrase for a clearly sincere one, so it should land as genuine support.
  • Fuller rewrite (Support, safe to send): Really glad it worked out for you. You deserve it, and I'd love to hear more whenever you feel like sharing. Adds encouragement and an open invitation to share more, though it assumes the news was something positive.

I don't know what the news was, so I kept these general. Want it shorter, warmer, or more casual?

Context helps. Subtext reads the thread when it gets one and can ask who the recipient is, because the same words mean different things to a manager and a situationship. Subtext will also grade a sentence floating in space, the isolated case I warned about above: “Good for you.” above came back with a tag and with a version marked safe to send, “Good for you, I’m happy to hear it.” It did say that it didn’t know what the news was, so it had kept its versions general.

In practice, you paste your draft or screenshot the thread, and Subtext names what your message is likely to signal, points at the words creating that impression, and offers versions that still sound like you. Try Subtext in your browserDownload on the App StoreGet it on Google Play

My read is that this category is smaller than the pitch decks and more useful than the sceptics allow. Mind reading stays out of reach. Reading wording is available today, and most of the damage in written communication starts there anyway. That felt like a good enough reason to build.


Work on one of these tools and think I’ve misread your benchmark? Tell me on LinkedIn.

Samet Durgun is the co-founder of Subtext, an app that catches the emotional tone of your messages and rewrites them in your own voice. He’s based in Berlin.


Sources

Every link above goes to the primary source where one exists, numbered in order of appearance. Two entries are cited without a link because I could not confirm a stable URL for them, and the last two are further reading rather than claims made above.

  1. Google Cloud. Natural Language API basics: sentiment score and magnitude.
  2. AWS. Amazon Comprehend: sentiment analysis.
  3. Microsoft. Azure Language: sentiment analysis and opinion mining.
  4. Chhaya et al. (2018). Frustrated, Polite, or Formal: Quantifying Feelings and Tone in Email. PEOPLES Workshop, NAACL 2018.
  5. Demszky et al. (2020). GoEmotions: A Dataset of Fine-Grained Emotions. ACL 2020.
  6. Wang et al. (2024). Large Language Models on Fine-grained Emotion Detection Dataset with Data Augmentation and Transfer Learning. arXiv.
  7. Rethinking Emotion Annotations in the Era of Large Language Models (2025).
  8. Buechel and Hahn (2017). EmoBank, a corpus annotated from both writer and reader perspectives, and “Readers vs. Writers vs. Texts,” Linguistic Annotation Workshop 2017.
  9. Muhammad et al. (2025). SemEval-2025 Task 11: Bridging the Gap in Text-Based Emotion Detection. SemEval 2025.
  10. Li et al. (2025). Can Third Parties Read Our Emotions? ACL 2025.
  11. Abu-Salih et al. (2023). Emotion detection of social data: APIs comparative study. Heliyon.
  12. Schlegel, Sommer, and Mortillaro (2025). Large language models are proficient in solving and creating emotional intelligence tests. Communications Psychology.
  13. Strachan et al. (2024). Testing theory of mind in large language models and humans. Nature Human Behaviour.
  14. Ayers et al. (2023). Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum. JAMA Internal Medicine.
  15. Sabour et al. (2024). EmoBench: Evaluating the Emotional Intelligence of Large Language Models. ACL 2024.
  16. Feng et al. (2024). Affect Recognition in Conversations Using Large Language Models. SIGDIAL 2024.
  17. Hazarika et al. (2018). CASCADE: Contextual Sarcasm Detection in Online Discussion Forums. COLING 2018.
  18. Srirag et al. (2025). BESSTIE: A Benchmark for Sentiment and Sarcasm Classification for Varieties of English. Findings of ACL 2025.
  19. Sap et al. (2019). The Risk of Racial Bias in Hate Speech Detection. ACL 2019.
  20. Barrett et al. (2019). Emotional Expressions Reconsidered: Challenges to Inferring Emotion From Human Facial Movements. Psychological Science in the Public Interest.
  21. Rhue (2019). Racial Influence on Automated Perceptions of Emotions. SSRN working paper.
  22. Microsoft (2022). Framework for building AI systems responsibly, announcing the retirement of emotion inference in the Face API.
  23. EPIC. In re HireVue, FTC complaint (2019).
  24. EU AI Act, Article 5: Prohibited AI Practices, applicable since February 2, 2025.
  25. Lewis Silkin (2025). Understanding the EU AI Act’s prohibited practices, on the Commission’s February 2025 guidelines.
  26. Future of Privacy Forum (2026). Red Lines under the EU AI Act: the prohibition of emotion recognition in the workplace and education.
  27. Grammarly. Tone detector product page and how the tone detector works.
  28. IBM. Watson Natural Language Understanding.
  29. Hume AI.
  30. Northeastern University (2021). You can’t determine emotion from someone’s facial movements, and neither can AI.
  31. The Guardian (2024). Are you 80% angry and 2% sad? Why “emotional AI” is fraught with problems.