Efficient data retrieval of arbitrary patterns in time series under long pulse conditions and massive databases

20 Nov 2025, 09:15
30m
Presentation Big data (incl. smart data retrieval) Thursday Morning 1

Speaker

Jose Manuel Hostalet Wandosell

Description

Fusion devices such as ITER are expected to generate over one million sensor signals per discharge, each lasting 30 minutes or more. Identifying known patterns within these long time series is essential for understanding plasma behavior, but it poses significant computational challenges. In this work, we present a method for efficient pattern search in large-scale time series using FAISS, a high-performance library for similarity search. The technique involves a sliding window over each signal, transforming each window into a feature vector normalized to lie on the unit hypersphere. These vectors are stored in a FAISS index using cosine distance as the similarity metric. Once indexed, the system enables rapid retrieval of similar segments by comparing a known reference pattern against the database and selecting those below a predefined distance threshold. This approach has shown excellent performance, identifying target patterns within seconds across discharges longer than 30 minutes. The technique offers a practical and scalable solution for fast pattern retrieval in the massive, high-dimensional datasets expected from ITER and future fusion experiments.

Primary authors

Co-author

Presentation materials