Skip to content
R&D

Pathological Confidence, Out-of-Distribution Degradation, and Neural Machine Translation Artefacts in Discriminative AI Text Classifiers

Commercial transformer-based synthetic text classifiers (specifically Pangram architectures) report near-zero false-positive rates (FPR < 10⁻⁴) and robust cross-lingual generalisation.

Art of FACELESS — Independent R&D Protocol

AoF-RD-2026-088: Pathological Confidence, Out-of-Distribution Density Collapse, and NMT Artefacts in Discriminative AI Classifiers

Lead Investigator: Retired PhD Pharmacologist (AoF Independent R&D)
Methodology: Cross-Lingual Metric Audit & Epistemic Calibration Assessment
Date: November 2026

1. Abstract

Commercial transformer-based synthetic text classifiers (specifically Pangram architectures) report near-zero false-positive rates (FPR < 10⁻⁴) and robust cross-lingual generalisation. Empirical evaluation of a human-authored control text in literary Welsh (Cymraeg) and its subsequent deterministic Neural Machine Translation (NMT) via Google Translate into English yielded degenerate classification states:

P(Synthetic | Text_Welsh) = 1.000 (100% AI)
P(Synthetic | Text_NMT_English) = 1.000 (100% AI)

This paper documents how commercial detection pipelines suffer from severe epistemic overconfidence induced by:

  1. Out-of-Distribution (OOD) density collapse in minority/regional languages;
  2. Structural metric compression under poetic verse constraints;
  3. Low-entropy path regularisation inherent to sequence-to-sequence beam-search decoding in NMT.

2. Mathematical Architecture of the Detector

Pangram and comparable discriminative classifiers deploy fine-tuned bidirectional transformer backbones (e.g., RoBERTa-large derivations) coupled to a pooling layer and a multi-layer perceptron (MLP) classification head.

For an input token sequence x = [t₁, t₂, ..., t_N]:

h = Encoder(x)
z = W₂ · GeLU(W₁ · h + b₁) + b₂
ŷ = σ(z) = 1 / (1 + e⁻ᶻ)

Where:

  • h is the pooled semantic embedding representation.
  • z represents the unbounded scalar logit output for the synthetic class.
  • ŷ ∈ [0, 1] represents the model's raw posterior estimate: P(Synthetic | x).

2.1 The Asymptotic Saturation Fallacy (The "100%" Artefact)

In binary cross-entropy loss, assigning an empirical posterior probability of exactly 1.000 requires infinite loss if the ground truth is negative (human-authored):

lim [ŷ → 1.0]  Loss_BCE(y = 0, ŷ) = -lim [ŷ → 1.0] log(1 - ŷ) = +∞

In rigorous analytical assays, absolute certainty is unachievable due to intrinsic measurement variance and Bayesian uncertainty bounds. Outputting 100% indicates that the logit z exceeded an internal clipping threshold (z ≥ z_clip, where σ(z) ≈ 1 - ε, with ε < 10⁻⁶). The interface rounds this extreme asymptotic tail to an integer (100%), replacing calibrated Bayesian uncertainty with unearned marketing authority.


3. Mechanisms of Failure

3.1 Out-of-Distribution (OOD) Density Collapse in Welsh

Let D_train represent the multi-lingual training distribution. Standard training corpora exhibit vast linguistic asymmetry:

Ratio(Tokens_English / Tokens_Welsh) > 1,000 : 1

Discriminative classifiers do not examine semantic intent; they estimate whether the input x resides within the expected density manifold of human prose. In sparse, low-resource feature spaces:

∇_x log p(x | Human) → undefined / erratic

Because generative LLMs have limited Welsh pre-training data, synthetic Welsh text is restricted to highly regular, textbook grammatical morphology and canonical mutation rules. When a fluent human author writes structured, disciplined Welsh verse, the token transition probabilities mirror this low-variance profile. The detector's severely under-parameterised manifold for Welsh mistakenly maps genuine human stylistic discipline directly into the synthetic class boundary.

3.2 Verse-Induced Metric Distortion

Classifiers rely on two core statistical proxies across running text:

  • Perplexity (PPL): The geometric mean of inverse conditional token likelihoods:
    PPL(x) = exp( - (1/N) · Σ log P(t_i | t₁, ..., t_{i-1}) )
  • Burstiness: The standard deviation of sentence-level perplexity divided by the mean perplexity:
    Burstiness = σ(PPL) / μ(PPL)

Poetry breaks these assumptions:

  1. Truncated Context: Short poetic lineation fractures standard 512-token sliding self-attention contexts.
  2. Metric / Rhyme Regularisation: Poetic cadences create cyclical, low-entropy token transitions rather than the wide, erratic variance typical of human prose.

The detector reads formal poetic restraint not as human craft, but as mechanical generation.

3.3 Neural Machine Translation (NMT) as an Entropy Funnel

Subjecting the Welsh original to Google Translate generated an English output that also scored 100% synthetic. This is an unavoidable mathematical outcome of NMT sequence decoding.

Google Translate uses an encoder-decoder transformer operating under Beam Search decoding with length penalisation:

ŷ_NMT = argmax_y Σ log P(y_t | y_<t, x_Welsh)

Human literary translation introduces idiosyncratic, high-perplexity lexical choices. In contrast, Beam Search actively prunes low-probability paths, optimising exclusively for:

  • High collocation frequency (standardised multi-word expressions);
  • Minimum conditional entropy paths;
  • Canonical syntactic structures.
[Human Thought in Welsh]
       │
       ▼ (High structural density / low training corpus representation)
[Google NMT Engine]
       │
       ▼ (Beam-search pruning strips lexical entropy and personal voice)
[Smoothed English Output]
       │
       ▼ (Near-zero burstiness / uniform low-perplexity token chains)
[Pangram Detector]
       │
       ▼
Result: 100% AI Flag (Model identifies the NMT conduit, not provenance)

4. Diagnostic Conclusions

  1. Epistemic Overconfidence: Publishing a binary 100% certainty metric on out-of-distribution, minority-language, or short-form texts violates standard diagnostic error budgeting.
  2. Translation Invalidation: Subjecting translated literature to statistical detectors penalises international and minority-language writers. The tool cannot distinguish between an LLM hallucinating text and an NMT pipeline translating human thought.
  3. Protocol Verdict: Commercial detection scores on regional languages and translated verse do not reflect authorship provenance; they merely register the presence of non-English syntactic structures and algorithmic sequence-smoothing.

Empirical evaluations demonstrate that commercial AI text detectors consistently produce false positives when applied to constrained linguistic structures, exhibiting significant error rates against non-native writing and non-standard prose (Liang et al., 2023; Weber-Wulff et al., 2023).

References

Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023). GPT detectors are biased against non-native English writers. Patterns, 4(7), 100779. https://doi.org/10.1016/j.patter.2023.100779 Cited by: 1465

Weber-Wulff, D., Anohina-Naumeca, A., Bjelobaba, S., Foltýnek, T., Guerrero-Dib, J., Popoola, O., Šigut, P., & Waddington, L. (2023). Testing of detection tools for AI-generated text. International Journal for Educational Integrity, 19(1). https://doi.org/10.1007/s40979-023-00146-z Cited by: 1358

Comments

Latest

The Gatekeepers’ Crusade
AI

The Gatekeepers’ Crusade

Legacy media executives and best-selling authors are entirely free to boycott whatever technology they wish from the comfort of their advances and agency contracts. What they do not get to do is pretend they speak for the rest of us.

Members Public
The Receipt
AI

The Receipt

TXN: /the-receipt Art of FACELESS · punk publishing The Receipt What Your Outrage Actually Costs the Grid Harm = energy × uses That's the missing word from the card. You found it. Now let's add up what it costs, all of it, for all of us. HARM = ENERGY

Members Public

© 2026 Art of FACELESS. All rights reserved.
The Hollow Circuit®, Hyperstition Architecture®, The Veylon Protocol®, and Cognitive Colonisation® are proprietary intellectual property of Art of FACELESS.