"Explainability und Language Models" by Benjamin Roth (University of Vienna)
Overview about XAI Methods in Language Models (feature-based, instance-based, mechanistic explainability, and self-explanations).
"Evaluation and Interpretability of NLP systems" by Pia Sommerauer (Vrije Universiteit Amsterdam)
(Large) language models achieve impressive results on various NLP tasks. As such, they have become attractive tools for many humanities fields that study research questions based on (large amounts of) texts. LLMs are, however, based on vast amounts of internet texts; they still rely on learning (possibly complex and sophisticated) associations between words and their contexts. It is difficult to tell to what degree language models can reflect a human-like understanding of semantics and how specific information is reflected.
Language models underly many current state-of-the-art systems for various NLP applications, such as automatic detection of offensive language, automatic sentiment analysis, and many others. Even though such systems may achieve high performance on standard test sets, it remains difficult to predict how they will perform in a specific use case (e.g. texts of a different style or genre or written in a different time period). Therefore, it is particularly important to understand how systems are likely to behave; what mistakes they might make, and where their strengths lie.
In this session, we will take a critical look at the strengths and weaknesses of (L)LLMs and consider how LLM-based systems can be evaluated. In particular, we will look at approaches that go beyond standardly computed scores but try to understand what phenomena models can and cannot handle. In a practical, hands-on session, we will build a challenge dataset using a behavioral testing approach that can be used to analyze a system (and, if time allows, analyze a system).
Data Management and Handling
A Crash-Course on the Lifecycle of Scientific Data by Massimiliano Carloni (ACDH-CH), Seta Štuhec (ACDH-CH), and Stefan Resch (ACDH-CH)
Machine learning typically involves a lot of trial and error, experimentation, and alternative solutions. It also encompasses a large number of components and artefacts, including datasets, configuration files, pre-trained artefacts, and evaluation metrics. Organising and keeping track of all this can be a daunting task. These lectures aim to give a quick introduction to the basics of data (and model) management, based on the workflows and experiences developed in the research data repository ARCHE of the Austrian Academy of Sciences. The lectures will provide information on what (research) data and metadata are; the data lifecycle; why data management (and a data management plan) is so important; what organisational principles we can use to keep better track of our own data and models; what potential copyright and licensing considerations we should take into account. It will also focus on the so-called FAIR (Findable, Accessible, Interoperable, Reusable) Data Principles and give guidance on how to implement them in ML and AI research when organising, preserving, and disseminating your data and/or models. In addition to this, Stefan Resch will present VELD ("Versioned Executable Logic and Data"), a concept that combines a design pattern, a reference implementation, and a platform to standardize the creation of flexible and reproducible data/code/workflows.
Open Source and Licensing: Publish your code – it’s good enough by Piotr Majdak (ARI)
Scientists often view their data (such as code, research data, or graphics) as suboptimal, but usually it is "good enough" to benefit both research and the scientific community. Publishing data promotes transparency, enables peer review, and fosters improvements. One burden in publishing data is the choice of the license. In this talk, we briefly recap the development of open source and approach that concept from the perspectives of user and developer. We then look into various licensing models and go through a step-by-step instruction list to publish the code and data. Finally, we analyze three example open-source projects to show the potential pitfalls.
ÖAW Datathek by Herwig Stöger (Verlag der ÖAW)
DATATHEK is the new generic research data repository of the Austrian Academy of Sciences. It provides you with the simple and convenient option of publishing your research data in a way that is visible and citable in the long term, respecting the FAIR data prinicples. It has been developed under the umbrella of the EU SONICOM project with Acoustics Research Institute of the Academy.
Neuro-Symbolic AI by Emanuel Sallinger (TU Vienna)
Neuro-symbolic AI brings together the two big families of AI: on the one hand statistical AI models such as (deep) neural network-based AI, on the other hand symbolic AI methods such as logic-based AI. Traditionally divided, in recent times neuro-symbolic AI has seen rapid growth.
In this course, we are going to cover - for researchers from either area of AI - the principles, techniques and applications of neuro-symbolic AI. We are particularly focusing on graph-based and knowledge graph-based methods, were one can see neuro-symbolic techniques come alive, bringing together:
* knowledge graph embeddings as ML-techniques particularly suited to graph structures
* graph neural networks utilizing the graph structure itself
* logic-based techniques for very complex reasoning tasks
Apart from the unique aspects of neuro-symbolic AI, we are also going to make connections to the other courses of the school, including to data management, natural language processing (such as the use of knowledge graphs and neuro-symbolic AI to aid large language models) and geometric deep learning.
Geometric Deep Learning by Michael Bronstein (AITHYRA and University of Oxford)
"Symmetry, as wide or as narrow as you may define its meaning, is one idea by which man through the ages has tried to comprehend and create order, beauty, and perfection.” This poetic definition comes from the great mathematician Hermann Weyl, credited with laying the foundation of our modern theory of the universe. Symmetry was a crucial element of the Erlangen Programme in the late 19th century that revolutionised geometry and created novel branches of mathematics. Today, symmetry comes to our help in bringing a geometric unification of deep learning. These lectures will present a common mathematical framework, referred to as "Geometric deep learning", to study the most successful network architectures, giving a constructive procedure to build future machine learning in a principled way that could be applied in new domains such as biology, medicine, and drug design. We will explore geometric ideas underpinning the recent successes in protein design that were awarded the 2024 Nobel Prize in Chemistry and show the current cutting edge of the field.
