Alice Kwak, Jacob Israelsen, Clayton Morrison, Derek E. Bambauer, and Mihai Surdeanu, of the University of Arizona, have made available for download their article, Validity Assessment of Legal Will Statements as Natural Language Inference , published in Findings of the Association for Computational Linguistics: EMNLP 2022. The abstract is as follows:
This work introduces a natural language inference (NLI) dataset that focuses on the validity of statements in legal wills. This dataset is unique because: (a) each entailment decision requires three inputs: the statement from the will, the law, and the conditions that hold at the time of the testator’s death; and (b) the included texts are longer than the ones in current NLI datasets. We trained eight neural NLI models in this dataset. All the models achieve more than 80% macro F1 and accuracy, which indicates that neural approaches can handle this task reasonably well. However, group accuracy, a stricter evaluation measure that is calculated with a group of positive and negative examples generated from the same statement as a unit, is in mid 80s at best, which suggests that the models’ understanding of the task remains superficial. Further ablative analyses and explanation experiments indicate that all three text segments are used for prediction, but some decisions rely on semantically irrelevant tokens. This indicates that overfitting on these longer texts likely happens, and that additional research is required for this task to be solved.
To see the full article, click: Validity Assessment of Legal Will Statements as Natural Language Inference by Alice Kwak, Jacob Israelsen, Clayton Morrison, Derek E. Bambauer, and Mihai Surdeanu
Posted by Kaitlyn Bare, Associate Editor, Wealth Strategies Journal.
Wealth Strategies Journal 