This study separates ranking failures from poor thresholding across ten audited English and Bengali corpora and seven detector models in order to diagnose why fake-news detectors fail across datasets. It finds a label-coding contradiction in WELFake and a decrease in mean ROC-AUC from 0.914 in-domain to 0.567 cross-corpus. The authors discover that ranking deficits account for 70% of lost macro-F1, with miscalibrated operating points accounting for the remaining portion. In cases where standard EM prior adaptation fails, flagging new items at the source base rate recovers 81% of threshold loss without target labels (a rule that also fixes zero-shot language models). Lastly, although Bengali clickbait spreads across publishers, corpus-invariance and specialized models suffer when corpus and label are entangled, so the only representation change that helps is pooling training data.

researchgate.net/profile/Md-Rejaul-Korim-Sadi/publication/417272703_Rank_or_threshold_Why_fake-news_detectors_fail_on_new_corpora/links/6ae64bb1dc41aa6ac30fddde/Rank-or-threshold-Why-fake-news-detectors-fail-on-new-corpora.pdf

By author

Leave a Reply

Your email address will not be published. Required fields are marked *