Julian | Researcher (AI in sciences)

Abstract Bioacoustic research, vital for promoting conservation and understanding animal behaviour and ecology, faces a monumental challenge: analysing vast datasets where animal vocalizations are rare. While deep learning techniques are becoming standard, adapting them to bioacoustics remains difficult. We address this challenge with animal2vec, an interpretable large transformer model and a self‐supervised training scheme tailored for sparse and unbalanced bioacoustic data. It learns from unlabelled audio and then refines its understanding with labelled data. Furthermore, we introduce and publicly release MeerKAT: Meer kat K alahari A udio T ranscripts, a dataset of meerkat ( Suricata suricatta ) vocalizations with millisecond‐resolution annotations, the largest labelled dataset on a non‐human terrestrial mammal currently available. Our model sets a baseline on the MeerKAT corpus, outperforming other transformer models, and improves on existing methods on the publicly available NIPS4Bplus birdsong dataset. Moreover, animal2vec performs well even with limited labelled data (few‐shot learning). animal2vec and MeerKAT provide a new reference point for bioacoustic research, enabling scientists to analyse large amounts of data even with scarce ground truth information.

Single-shot coherent diffraction imaging of isolated nanosized particles has seen remarkable success in recent years, yielding in-situ measurements with ultra-high spatial and temporal resolution. The progress of high-repetition-rate sources for intense X-ray pulses has further enabled recording datasets containing millions of diffraction images, which are needed for the structure determination of specimens with greater structural variety and dynamic experiments. The size of the datasets, however, represents a monumental problem for their analysis. Here, we present an automatized approach for finding semantic similarities in coherent diffraction images without relying on human expert labeling. By introducing the concept of projection learning, we extend self-supervised contrastive learning to the context of coherent diffraction imaging and achieve a dimensionality reduction producing semantically meaningful embeddings that align with physical intuition. The method yields substantial improvements compared to previous approaches, paving the way toward real-time and large-scale analysis of coherent diffraction experiments at X-ray free-electron lasers.

Intense short-wavelength pulses from free-electron lasers and high-harmonic-generation sources enable diffractive imaging of individual nanosized objects with a single x-ray laser shot. The enormous data sets with up to several million diffraction patterns present a severe problem for data analysis because of the high dimensionality of imaging data. Feature recognition and selection is a crucial step to reduce the dimensionality. Usually, custom-made algorithms are developed at a considerable effort to approximate the particular features connected to an individual specimen, but because they face different experimental conditions, these approaches do not generalize well. On the other hand, deep neural networks are the principal instrument for today’s revolution in automated image recognition, a development that has not been adapted to its full potential for data analysis in science. We recently published [Langbehn et al., Phys. Rev. Lett. 121, 255301 (2018)PRLTAO0031-900710.1103/PhysRevLett.121.255301] the application of a deep neural network as a feature extractor for wide-angle diffraction images of helium nanodroplets. Here we present the setup, our modifications, and the training process of the deep neural network for diffraction image classification and its systematic bench marking. We find that deep neural networks significantly outperform previous attempts for sorting and classifying complex diffraction patterns and are a significant improvement for the much-needed assistance during postprocessing of large amounts of experimental coherent diffraction imaging data.

Coherent diffractive imaging of individual free nanoparticles has opened routes for the in situ analysis of their transient structural, optical, and electronic properties. So far, single-shot single-particle diffraction was assumed to be feasible only at extreme ultraviolet and X-ray free-electron lasers, restricting this research field to large-scale facilities. Here we demonstrate single-shot imaging of isolated helium nanodroplets using extreme ultraviolet pulses from a femtosecond-laser-driven high harmonic source. We obtain bright wide-angle scattering patterns, that allow us to uniquely identify hitherto unresolved prolate shapes of superfluid helium droplets. Our results mark the advent of single-shot gas-phase nanoscopy with lab-based short-wavelength pulses and pave the way to ultrafast coherent diffractive imaging with phase-controlled multicolor fields and attosecond pulses.Diffraction imaging studies of free individual nanoparticles have so far been restricted to XUV and X-ray free - electron laser facilities. Here the authors demonstrate the possibility of using table-top XUV laser sources to image prolate shapes of superfluid helium droplets.