Author Identifier

Jianxin Li: https://orcid.org/0000-0002-9059-330X

Document Type

Journal Article

Publication Title

Knowledge-Based Systems

Volume

306

Publisher

Elsevier

School

School of Business and Law

RAS ID

77080

Funders

National Natural Science Foundation of China (62272106, 62072113, and 62172243)

Comments

Fan, Y., Ren, T., Huang, C., Zheng, B., Jing, Y., He, Z., ... & Li, J. (2024). A confidence-based knowledge integration framework for cross-domain table question answering. Knowledge-Based Systems, 306. https://doi.org/10.1016/j.knosys.2024.112718

Abstract

Recent advancements in TableQA leverage sequence-to-sequence (Seq2seq) deep learning models to accurately respond to natural language queries. These models achieve this by converting the queries into SQL queries, using information drawn from one or more tables. However, Seq2seq models often produce uncertain (low-confidence) predictions when distributing probability mass across multiple outputs during a decoding step, frequently yielding translation errors. To tackle this problem, we present CKIF, a confidence-based knowledge integration framework that uses a two-stage deep-learning-based ranking technique to mitigate the low-confidence problem commonly associated with Seq2seq models for TableQA. The core idea of CKIF is to introduce a flexible framework that seamlessly integrates with any existing Seq2seq translation models to enhance their performance. Specifically, by inspecting the probability values in each decoding step, CKIF first masks out each low-confidence prediction from the predicted outcome of an underlying Seq2seq model. Subsequently, CKIF integrates prior knowledge of query language to generalize masked-out queries, enabling the generation of all possible queries and their corresponding NL expressions. Finally, a two-stage deep-learning ranking approach is developed to evaluate the semantic similarity of NL expressions to a given NL question, hence determining the best-matching result. Extensive experiments are conducted to investigate CKIF by applying it to five state-of-the-art Seq2seq models using a widely used public benchmark. The experimental results indicate that CKIF consistently enhances the performance of all the Seq2seq models, demonstrating its effectiveness for better supporting TableQA.

DOI

10.1016/j.knosys.2024.112718

Creative Commons License

Creative Commons Attribution 4.0 License
This work is licensed under a Creative Commons Attribution 4.0 License.

Download

Share

 
COinS