TALES School II: Hands-on workshop on machine learning applications to astrophysics

Hackathon - Projects & Material

General agenda:

08:30-09:30 Breakfast
09:30-11:00 Session 1/Project work
11:00-11:30 Coffee break
11:30-13:30 Session 2/Project work
13:30-14:30 Lunch break
14:30-16:00 Session 3/Project work
16:00-16:30 Coffee break
16:30-18:00 Session 4/Project work
19:30-20:30 Dinner

Labs and working spaces are available 24/7 for project work.

---

Project Title: Looking for Anomalous AGN Activity in ZTF DR5 Light Curves

Project Leader: Paula Sánchez-Sáez

 

Sánchez-Sáez et al. (2021, AJ, 162, 206) searched ZTF Data Release 5 for changing-state AGN using a two-stage method: a Variational Recurrent Autoencoder (VRAE) that compresses each g-band light curve into a 16-dimensional latent space, followed by an Isolation Forest run on those latent features. Sources were flagged as anomalous by high reconstruction error or by a low Isolation Forest score, yielding 8,809 anomalies — mostly artifacts, but including 75 promising changing-state AGN candidates alongside atypical blazars, misclassified variable stars, and flaring NLS1s.

This data challenge reuses the same data products, but drops the changing-state focus in favor of class-agnostic anomaly detection: finding and characterizing anomalies of any kind across the AGN / QSO / Blazar population, including  flares, periodic signals, extreme variability, contaminants such as eclipsing binaries, cataclysmic variables, supernovae and tidal disruption events, photometric artifacts, or anything else unexpected.

The challenge

How to surface the most interesting anomalies is open, the approach is up to each team. Some directions worth considering:

•      What does the data look like? Exploring the variability features and the latent space (with UMAP, t-SNE, or anything else) is a natural starting point.

•      Which detection method? Isolation Forest, LOF or k-NN distances, One-Class SVM, VRAE reconstruction error, density-based methods, deep one-class approaches, or something entirely different.

•      Which representation? The latent features, the variability features, a combination, or a new representation built from the light curves themselves.

•      What did you actually find? Inspecting the top candidates, classifying the anomaly types, and cross-matching against other catalogs such as SIMBAD, TNS and Gaia turns a ranked list into a result.

Teams are encouraged to define their own notion of what makes an anomaly interesting and to justify it. Guidance and support will be available throughout, but the choice of strategy is yours.

The data

Each source in the sample comes with:

Light curves	ZTF DR5 g-band photometry: time, magnitude, magnitude error
Variability features	A set of variability features typically used for AGN classification, as computed by the ALeRCE broker (Sánchez-Sáez et al. 2021a). including amplitudes, structure function parameters, periodicity statistics, colors, autocorrelation parameters, and others
Latent-space features	16-dimensional latent mean vector from the VRAE modeling of the light curves (Sánchez-Sáez et al. 2021b)
Spectral properties	Redshifts, bolometric luminosity, black hole mass and Eddington ratio, where available from SDSS
AGN class	Classification according to the Milliquas catalog (AGN, QSO or Blazar)

 

The data come in two samples:

•      Balanced sample: 24,755 sources, balanced in physical properties (black hole mass, bolometric luminosity) and cadence (number of epochs). This is the sample used by Sánchez-Sáez et al. (2021b) to train the VRAE. This sample is already available.

•      Full sample: 230,464 sources, all AGN with data available in ZTF DR5. This is the population to be mined for anomalies.

 

Acquired skills and knowledge

Anomaly detection · time-domain analysis · light-curve features · deep learning and machine learning.

 

Suggested software: Python 3, scikit-learn, TensorFlow / Keras or PyTorch.

 

References and data

• Sánchez-Sáez et al. 2021b, AJ, 162, 206: paper that present the VRAE model
• Sánchez-Sáez et al. 2021a, AJ, 161, 141: ALeRCE broker light curve classifier (with the variability features description)
• Portillo et al. 2020, AJ, 160, 45: a good reference to understand how variational autoencoders work.
• VAE modeling code (GitHub): Software used by Sánchez-Sáez et al. 2021b 
• Balanced data folder:  catalogs and light curves for the balanced sample. The full sample will be provided during the hackathon.  

 

Publications using these data should acknowledge the following articles  Sánchez-Sáez et al. 2021b, AJ, 162, 206 · Sánchez-Sáez et al. 2021a, AJ, 161, 141

---

Project Title: Measuring the tIme delays among the light curves of lensed quasar images

Project Leader: Paolo Bonfini

 

Quasars are variable objects across all wavelengths. In the optical (but also in UV and X-rays) the quasar accretion disc can vary significantly (by some fraction of a magnitude or more) over periods of days to years, producing a light curve that can be observed in various filters. A common model that can describe this variability in the optical is a Damped Random Walk.

 

A galaxy intervening along the line-of-sight to a quasar can gravitationally lens it, producing 2 or 4 images. Each quasar image contains the same variable signal but shifted in magnitude by a constant gravitationally induced magnification by the lensing galaxy and in time due to the different length of the path the light travels along (both different for each image). This introduced time-delay, Δt, in signal arrival times between quasar images is an important quantity for cosmology and can be used to measure the Hubble constant, H0. If a precision of ~1 day can be obtained for Δt in many lensed quasars,  then, combined with some other model components, the corresponding precision in H0 will be ~1%. This is a very precise measurement that can shed light into the Hubble tension, i.e. the significant discrepancy in H0 measurements between various methods that can potentially hint to the existence of new physics.

 

 

The Challenge: 

Measuring Δt from light curves presents the following challenges:

• Taking into account irregularly sampled data with season gaps. In particular with respect to LSST, this means that the observing strategy in each filter leads to differently sampled  light curves. However, the time-delay in each band should be the same because the gravitational deflection of light is achromatic. 
• The signal-to-noise ratio for highly magnified images will be high, but the opposite will happen for low magnification images.
• Microlensing can be thought of as a second-order lensing effect that is produced by individual stellar mass objects (stars, remnants, etc) within the lensing galaxy. This can further (de)magnify the quasar in a time-dependent way but it does not introduce any time-delay (although see the reference to Tie & Kochanek 2018 below). This is possible because the angular size of the accretion disc happens to match the Einstein radius, which can be thought of as the characteristic scale for lensing, of stellar mass objects within the lens.

 

 

The data:

The objective is to measure the time delta, Δt, among light curves from the 3 following datasets:

 

Simulated light curves with known Δt	Synthetic light curves created to mimic an observing strategy, signal-to-noise ratio, and filter, either for LSST or ZTF or both. These will be provided individually for each image of a lensed quasar. This will be the benchmark to develop the algorithms on.
Simulated light curves with unknown Δt	The same as above, but without giving away the true Δt to the challenge participants. Measuring it is the real challenge of this Hackathon.
Existing light curves with measured Δt	There are <50 lensed quasars with publicly available data from the literature, mostly from the COSMOGRAIL program that observed them with the Euler telescope in Chile. A good amount of these quasars have well-measured time-delays. Using this dataset challenge participants can compare the performance of their algorithms to the state-of-the-art.
Existing light curves without good Δt measurements	Some of the known and observed lensed quasars do not have a measured Δt. This can be for a number of reasons (problematic season gaps, intrinsic variability, etc). This is the real-life challenge. If you manage to robustly measure a time-delay for these systems then you could publish your result!

 

 

Acquired skills and knowledge:

Handling time series with gaps, correlating time series, understanding the basics of gravitational lensing, microlensing, the Hubble tension, time delay cosmography, quasar structure and variability

 

Suggested code for measuring time delays:

• https://gitlab.com/cosmograil/PyCS3
• https://github.com/duxfrederic/TDCOSMO_XVII_time_delays (this is based on the previous method, but is easier to use)
• pyPETAL https://pypetal.readthedocs.io/en/latest/index.html 

 

References

• See Di Valentino et al. 2025 for a review on the Hubble tension.
• See Birrer et al. (2024) for a review on time-delay cosmography with quasars and Suyu et al. (2024) for lensed supernovae (very similar problem).
• See Vernardos et al. (2024) for a review on microlensing.
• See Tie & Kochanek (2018) for the microlensing time delay.