Monday: "Introduction to ML/AI"
Michal Hradiš (Brno), What is AI? Overview lecture
Claudius Krause (MLA²S, ÖAW), Machine Learning at the ÖAW
Tuesday: Vibe Coding & „Promptotyping”
Christopher Pollin (DH Craft Graz)
This workshop introduces current approaches to AI-assisted software development. Starting with Vibe Coding and Context Engineering, participants will learn the Promptotyping methodology, which enables researchers without programming skills to develop their own data-driven tools and interfaces. Through hands-on exercises, participants will learn to construct with (Frontier-)LLMs and iteratively build functioning research artifacts (tools, models, workflows).
Dr. Christopher Pollin is the founder of Digital Humanities Craft OG and an independent researcher specializing in applied generative AI. He received his PhD from the University of Graz on digital modeling of historical information and is developing the Promptotyping methodology for the systematic creation of research tools using LLMs. He teaches Digital Humanities, Generative AI and Prompt Engineering at several Austrian universities.
Wednesday: "Ethics and AI"
Bernd Wachmann (ÖAW), The ÖAW AI Strategy
Open EuroLLM (to be confirmed)
Laura Ganglgruber (University of Vienna), AI policy at the University
Charlotte Spencer-Smith (CMC), Governing Machines: From Moderation to Alignment
Bias in ML (to be confirmed)
Thursday: "From idea to final project"
Stavrina Dimosthenous (University of Manchester), Data/Code Management, reproducibility
Felix Schmitt (ÖAW/CLIP), Introduction to CLIP
Friday: "Explainability"
Pia Sommerauer (Vrije Universiteit Amsterdam), "Evaluation and Interpretability of NLP systems"
(Large) language models achieve impressive results on various NLP tasks. As such, they have become attractive tools for many humanities fields that study research questions based on (large amounts of) texts. LLMs are, however, based on vast amounts of internet texts; they still rely on learning (possibly complex and sophisticated) associations between words and their contexts. It is difficult to tell to what degree language models can reflect a human-like understanding of semantics and how specific information is reflected.
Language models underly many current state-of-the-art systems for various NLP applications, such as automatic detection of offensive language, automatic sentiment analysis, and many others. Even though such systems may achieve high performance on standard test sets, it remains difficult to predict how they will perform in a specific use case (e.g. texts of a different style or genre or written in a different time period). Therefore, it is particularly important to understand how systems are likely to behave; what mistakes they might make, and where their strengths lie.
In this session, we will take a critical look at the strengths and weaknesses of (L)LLMs and consider how LLM-based systems can be evaluated. In particular, we will look at approaches that go beyond standardly computed scores but try to understand what phenomena models can and cannot handle. In a practical, hands-on session, we will build a challenge dataset using a behavioral testing approach that can be used to analyze a system (and, if time allows, analyze a system).
