Hagen, Torah
Dissertation topic:
"Improving the Representation of Semantic Relations and Semantic
Change using Language Models with Knowledge Graphs in
Low-Resource Scenarios."
Contact address at the University of Würzburg:
Chair of Digital Humanities and Modern German Literature
and Modern German Literature
Campus Hubland Nord
Emil-Hilb-Weg 23
97074 Würzburg
First supervisor: Prof Dr Fotis Jannidis
Second supervisors:
Dr Barbara McGillivray (King's College London)
Class in the Graduate School: "Digital Humanities"
Doctorate in the Graduate School from WS 2024/25.
Abstract:
The dissertation analyses the enrichment of pre-trained language models and word embeddings with knowledge graphs. Typically, the relationships between words in a language model are based "only" on distributional semantics; through training on a large text corpus. Frequently co-occurring words are closer to each other in the vector space, and in this way, word associations can be mapped in the model. However, not all meaningful word relations are frequently expressed in the form of continuous text and may therefore be inadequately represented. Structured knowledge in the form of graphs can help to reinforce word relations, in particular when there is not enough training data in the form of continuous text available to create a stable language model.
To be precise, this work will investigate how the injection of knowledge graphs can improve the representation of semantic relations that exist between words in language models, in particular word similarity, word association, and lexical entailment. Another kind of word relation, semantic change, which can be interpreted as the relation of a word to itself, will be highlighted as a special low-resource use case by first creating and then utilising a historical knowledge graph for the injection process.
