Speaker
Description
Complex digital lexicographical resources often feature rich data structures that are difficult for non-specialist users to query. This paper presents a pilot study on an LLM-driven natural-language interface for the Lehnwortportal Deutsch, a graph-based database of German loanwords in other languages. Rather than translating user questions directly into the database query language, the proposed architecture maps natural-language requests to the JSON configuration format already used by the portal’s visual query builder. This intermediate representation makes it possible for users to inspect and modify query builder configurations generated from their questions. We discuss the advantages of this approach and present a pilot study that evaluated it on a set of 310 German questions that, due to lack of real user questions, had to be AI-generated using several different strategies to cover a large range of usage scenarios as well as available querying options. Using one-shot prompting, a state-of-the-art commercial-grade LLM produced valid JSON in almost all cases; 260 outputs were rated perfect and 23 acceptable by human assessment under a low reasoning budget. A high-budget rerun improved most initially bad cases, though with impractical latency for real-time use. The paper also discusses methodological challenges in generating and evaluating synthetic query sets, handling unsupported user requests, and extending the approach through larger datasets, comparing multiple language models, varying output parameters, and using fine-tuning as an alternative strategy.