Background and Motivation

Large-scale sequencing initiatives such as the Earth BioGenome Project aim to sequence the genomes of all taxonomically classified eukaryotic species [1], generating vast amounts of raw genomic data in a short period of time. Subsequent analyses of these genomes require efficient and precise gene predictions, i.e., accurate predictions of the genes’ locations as well as their underlying exon-intron structure. This task is particularly complex in eukaryotic genomes, as protein-coding exons are strongly underrepresented in genomic data, impeding efforts to pinpoint the exact positions of splice sites, i.e., the borders between exons and introns [2].

While integrating experimental evidence such as RNA sequencing yields the most accurate predictions, such evidence is not available for many species and is costly to obtain. Ab initio tools, like Tiberius, thus derive gene structures solely from the raw DNA sequence. Tiberius combines a deep neural network with a hidden Markov model encoding biological knowledge and is among the most accurate ab initio gene predictors [2, 3].

Compared to evidence-based annotation tools such as BRAKER3, Tiberius is significantly faster, albeit not yet as accurate. Although both tools predict individual exons similarly well, Tiberius generally achieves lower F1 scores at the level of complete genes and transcripts, especially for non-mammalian vertebrates [3]. It is thus desirable to further improve Tiberius’ gene structure predictions.

The Idea: Window Loss to Reward Coherent Gene Structures

In my bachelor’s thesis, I investigated whether this can be achieved by training Tiberius with a new loss function, the Window Loss, replacing Tiberius’ CCE-F1 loss, which combines categorical cross-entropy (CCE) with the estimated F1-score for exon classes (e.g., internal exon positions and splice sites in each of the three reading frames) [2].

While the CCE-F1-loss is a position-wise loss, evaluating class predictions nucleotide by nucleotide [2], the Window Loss assesses the entire state path within a window of the DNA sequence. Therefore, it specifically rewards the model for assigning a high probability to the complete, correct gene structure.

Methods

I implemented the Window Loss and integrated it into Tiberius’s training pipeline. I then tested applying the Window Loss to a) the full DNA sequence and b) short windows around splice sites that a pretrained model currently predicts imprecisely, a form of hard example mining [4] (see Fig. 1). The models were trained on 66 vertebrate genomes and compared to a reference model trained with the CCE-F1 loss. Their mean locus F1 scores on nine held-out vertebrate test species were used to investigate whether either approach improves on the reference model.

Findings and Conclusion

The results of the training experiments are mixed. When applied to the full sequence, the Window Loss deteriorates both sensitivity and precision. Used for fine-tuning on hard examples, however, it improved the mean locus F1 score over the reference model for all test species, particularly for non-mammalian vertebrates. These improvements appeared during the first epochs of fine-tuning but did not persist as training continued. This suggests that the Window Loss may be a useful fine-tuning approach for structurally difficult regions. Future work should confirm the robustness of these improvements, explore alternative definitions of hard windows, and investigate how the model can retain previously attained knowledge.

References

[1] Lewin, H. A., Richards, S., Lieberman Aiden, E., Allende, M. L., Archibald, J. M., Bálint, M., ... & Zhang, G. (2022). The earth BioGenome project 2020: Starting the clock. Proceedings of the National Academy of Sciences, 119(4), e2115635118. https://doi.org/10.1073/pnas.2115635118

[2] Gabriel, L., Becker, F., Hoff, K. J., & Stanke, M. (2024). Tiberius: end-to-end deep learning with an HMM for gene prediction. Bioinformatics, 40(12), btae685. https://doi.org/10.1093/bioinformatics/btae685

[3] Gabriel, L., Brůna, T., Kaur, A., Krishnan, A., Ortmann, F., Salamov, A., ... & Hoff, K. J. (2026). Accurate ab initio gene prediction in eukaryotes with Tiberius in multiple clades. bioRxiv. https://doi.org/10.64898/2026.04.24.720536

[4] Shrivastava, A., Gupta, A., & Girshick, R. (2016). Training region-based object detectors with online hard example mining. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 761-769). https://doi.org/10.48550/arXiv.1604.03540