Speaker
Description
My presentation addresses the technical challenges of applying automated text recognition (ATR) to demographic data preserved in late Ottoman census registers. It builds on the experience we gained in the LOOP research project—Late Ottoman Palestinians. Two issues are central. First, the registers are handwritten in Ottoman Turkish, an under-resourced historical language for which effective pre-trained ATR models remain scarce. Second, the data consists of handwritten entries embedded in tabular forms, many with empty cells and others containing multi-line content that challenges standard layout assumptions. Existing table-segmentation methods perform poorly on such material, resulting in unreliable region detection and reduced transcription accuracy.
To address these challenges, we rethought segmentation entirely. Instead of attempting table recognition, we trained the segmentation model to draw a single line spanning the full width of each table row. During text recognition, vertical column boundaries are allowed to be recognized as pipe characters. In post-processing, the table is reconstructed from the text lines with the pipe character as a separator, as if from a CSV file. This approach achieved an overall text recognition rate of 94%, with an error rate of under 2% for years of birth. A further challenge emerged in segmentation: while line placement can be trained, the associated masks that define the text recognition area cannot. Due to empty cells and multi-line entries, these masks are often positioned too low. To address this, we export the segmented files and apply a custom Python script that generates new masks aligned to the full height of each table row. The corrected files are then re-imported for text recognition.
I speak from the perspective of end users of digital humanities tools. Rather than proposing new algorithms, we combine established platforms and models and extend them through Python scripts to construct a robust processing pipeline. Since few components functioned “out-of-the-box” for this material, the process required iterative experimentation and, at several points, deliberately “outside-the-box” technical decisions.