Research Article 2026-04-22 under-review v1

Multi-Layer Semantic Alignment for LLM Text Summarization Faithfulness Detection: A Safety Verification Framework for Chinese Historical and Cultural Domain

W
Wenhui LIU Jinan University
Y
Yanjun ZHANG Jinan University

Abstract

Hallucination in large language model-generated summaries poses a critical safety challenge for knowledge-intensive domains such as Chinese history and culture. Existing faithfulness detection methods, primarily designed for English news summarization, treat the task as binary classification, failing to distinguish factual errors from externally verifiable unsupported additions. They also rely on large-scale annotated data and neural resources, limiting applicability in domain-specific, resource-constrained settings. To address these limitations, this paper proposes a multi-layer semantic alignment framework for hallucination safety detection in LLM-generated summaries of Chinese historical and cultural texts. The framework formalizes the task as three-way classification—Faithful, Fact Error, and Unsupported Info—and constructs an interpretable four-layer feature representation encoding lexical, syntactic, semantic, and logical correspondences between source text and summary. The framework operates without neural model dependencies, making it suitable for low-resource environments. Experiments on a newly constructed benchmark dataset of expert-verified samples spanning multiple sub-domains demonstrate that the proposed four-layer configuration substantially and consistently outperforms single-layer baselines. Ablation studies confirm that the four layers provide complementary information, with syntactic features contributing most to overall detection performance. Feature distribution analysis reveals systematic inter-class differences across all layers. This paper makes three contributions: constructing the first three-class faithfulness detection benchmark for the Chinese historical and cultural domain; proposing an interpretable multi-layer alignment feature set and validating its complementarity; and providing practical guidance for domain-specific hallucination safety detection through structured classifier comparison and feature analysis.

Citation Information

@article{wenhuiliu2026,
  title={Multi-Layer Semantic Alignment for LLM Text Summarization Faithfulness Detection: A Safety Verification Framework for Chinese Historical and Cultural Domain},
  author={Wenhui LIU and Yanjun ZHANG},
  journal={International Journal of Computational Intelligence Systems},
  year={2026},
  doi={https://doi.org/10.21203/rs.3.rs-9131850/v1}
}
Back to Top
Home
Paper List
Submit
0.027298s