Engineering a Scalable, FAIR Data Infrastructure for Resource-Efficient Research

          @misc{ scivideos_oai:cds.cern.ch:3025557,
            doi = {},
            url = {https://videos.cern.ch/record/3025557},
            author = {},
            keywords = {},
            language = {en},
            title = {Engineering a Scalable, FAIR Data Infrastructure for Resource-Efficient Research},
            publisher = {},
            year = {2026},
            month = {may},
            note = {oai:cds.cern.ch:3025557 see, \url{https://scivideos.org/cern-cds/3025557}}
          }
          

Abstract

The Department of Aeronautics at Imperial College London is addressing the environmental and accessibility challenges of modern research by deploying a scalable repository architecture. This system integrates a custom InvenioRDM interface with Ceph object storage to manage massive computational datasets in alignment with FAIR principles. By leveraging software-defined storage on commodity hardware, the department avoids carbon-intensive "forklift upgrades," allowing for sustainable, incremental capacity growth. The infrastructure features a self-healing, S3-compatible backend designed to eliminate "dark data" through domain-specific metadata curation. To reduce energy consumption associated with unnecessary network egress, the platform supports flexible retrieval, enabling researchers to inspect granular data subsets rather than downloading entire multi-terabyte files. Ultimately, this ecosystem prevents redundant, energy-heavy re-computations by transforming primary data into a permanent, reusable asset. The proposed talk details the lifecycle of this transition from ad-hoc management to an integrated, environmentally conscious research framework.

00:00:00 Slide 1
00:00:01 Slide 2
00:00:32 Slide 3
00:01:41 Slide 4
00:02:48 Slide 5
00:04:28 Slide 6
00:06:12 Slide 7
00:08:01 Slide 8
00:08:50 Slide 9
00:09:38 Slide 10
00:10:23 Slide 11
00:10:43 Slide 12
00:11:13 Slide 13
00:13:31 Slide 14
00:14:44 Slide 15
00:16:34 Slide 16