Edit this page

NWB Workshops and Hackathons

:rewind: Back to the projects list

Integrate Zarr I/O with HDMF and PyNWB

Key Investigators

Project Description

Integrate Zarr as alternative (python-only) backend for HDMF and PyNWB


Approach and Plan

Progress and Next Steps



Installing the current draft backend

conda create -n zarr4nwb python=3.6
conda activate zarr4nwb
pip install zarr
git clone https://github.com/NeurodataWithoutBorders/pynwb.git
cd pynwb/
git checkout add/zarrio
python setup.py develop
git clone https://github.com/kangdh/hdmf.git
cd hdmf/
git checkout 1.0.3-zarr
python setup.py develop

Convert test Allen ICEphys file

from pynwb import NWBHDF5IO, NWBZarrIO
import os
infile = "H19."
outfile = "test_zarr_" + os.path.basename(infile)
h5r = NWBHDF5IO(infile , 'r', load_namespaces=False)
f = h5r.read()
zw = NWBZarrIO(outfile,
zw.write(f, cache_spec=True)
zr = NWBZarrIO(outfile, 'r')
zf = zr.read()

To create a test file: 1) copy the above code to a python file, 2) download the Allen pre-release file, and run the script, e.g.:

curl -O http://download.alleninstitute.org/informatics-archive/prerelease/H19.
python convert_to_zarr.py
ls test_zarr_H19.

With chunking enabled this should result in:

find test_zarr_H19. | wc -l
>>>    3168
du -hs test_zarr_H19.
>>>    166M

Background and References