NWB Workshops and Hackathons

Back to the projects list

Implement Exdir as an alternative backend to HDF5

Key Investigators

• Mikkel Elle Lepperød (CINPLA, University of Oslo, Norway)

Project Description

Exdir is a hirarchical data structure similar to HDF5 which operates on the filsystem instead of a HDF5 file. It stores data in binary files using numpy (npy files) and metadata in YAML files. Having Exdir as a backend in NWB would enable users to operate directly on the filesystem. This facilitate human readability of metadata, possibility to use version control systems such as git and share parts of a large NWB file.

Objectives

1. Objective A. Implement object and region references in Exdir.
2. Objective B. Implement a working version of Exdir in hdmf.

Approach and Plan

1. Implement a lookup table for all objects in a exdir file with uuid for each object.
2. Store references as ids.
3. Implement References and RegionReferences API in exdir.
4. Switch h5py imports with exdir in hdmf.

Progress and Next Steps

Materials

Background and References

• Source code: https://github.com/CINPLA/exdir
• Documentation: https://exdir.readthedocs.io/en/latest/