Resilient Time-Series Data Infrastructure for Fusion Experiments using High-Availability TimescaleDB and Grafana

20 Nov 2025, 11:15
30m
Presentation Big data (incl. smart data retrieval) Thursday Morning 2

Speaker

Prem Kumar (Institute for Plasma Research, Gandhinagar, Gujarat, India)

Description

In modern fusion research, uninterrupted access to historical data is critical for thorough post-discharge analysis, precise experiment control, and robust machine health monitoring. The ability to efficiently retrieve, visualize, and analyze large volumes of time-series data plays a central role in improving operational reliability and supporting data-driven decision-making.

This work presents a fault-tolerant, scalable architecture for time-series data collection, storage, and visualization, specifically designed to address the demanding requirements of fusion experiments. The platform is built on TimescaleDB, deployed in a high-availability PostgreSQL configuration using Patroni for automated failover, HAProxy for connection routing, and Keepalived for virtual IP failover management. This setup ensures continuous availability and transparent recovery during node failures. The entire infrastructure is fully containerized using Docker and orchestrated with Docker Compose, with future extensibility towards Kubernetes for improved scalability and cluster management.

Grafana dashboards are integrated to provide dynamic and intuitive visualization of historical data streams, supporting interactive analysis across different timescales and sensor channels. Benchmarking results demonstrate up to a 97% improvement in query latency for historical data retrieval with HA TimescaleDB compared to standalone PostgreSQL, highlighting its efficiency for short-interval queries and diagnostic workloads. In addition, the architecture supports seamless failover, load-balanced querying, and container-native service discovery, making it suitable for long-duration experimental campaigns where reliability is paramount.

Overall, this architecture provides a reproducible and extensible foundation for resilient data infrastructure in fusion research. It not only addresses immediate challenges of data availability and performance but also establishes a pathway for integration with advanced features such as anomaly detection, predictive analytics, and natural language querying, enabling intelligent diagnostic capabilities for next-generation fusion devices.

Primary author

Prem Kumar (Institute for Plasma Research, Gandhinagar, Gujarat, India)

Co-author

Mrs Kirti Mahajan (Institute for Plasma Research, Gandhinagar, Gujarat, India)

Presentation materials