NWB Workshops and Hackathons

Back to the projects list

Convert and add NWB2 datasets to DANDI

Key Investigators

• Satrajit (satra) Ghosh (MIT)
• Yaroslav (yarik) Halchenko (Dartmouth)
• Ben Dichter (Ben Dichter, LLC)
• Michael (mike) Grauer (Kitware, LLC)

Project Description

There are several public NWB datasets out there, but they vary in their conformance to NWB and their readability via PyNWB. Thie project aims to make public datasets conformant to NWB 2.0 and findable/accessible through the DANDI archive.

Objectives

1. Extend PyNWB to determine what’s missing to be NWB 2.0 compliant.
2. Convert current DANDI datasets to 2.0
3. Determine the necessary pieces for a DANDI dataset (dandiset).

Approach and Plan

1. Verify that all files are matching with source files

• http://datasets.datalad.org/?dir=/labs/buzsaki
• http://datasets.datalad.org/?dir=/labs/churchland
• http://datasets.datalad.org/?dir=/labs/svoboda
• http://datasets.datalad.org/?dir=/allen-brain-observatory

1. Create a list of non-compliant files and determine what makes these non-compliant.
2. Make files compliant while tracking the conversion.
3. Create dataset_description.jsonld using Metadata descriptor keys (see below). Discuss metadata harmonization and terms necessary.
4. Automate creation of samples.tsv/jsonld from individual NWB files
5. Package the dataset creation, validation, upload, and download process into dandi-cli
6. Add Metadata editing GUI to DANDI.

Progress and Next Steps

Materials

Background and References

• Metadata
• PyNWB
• DANDI cli
• DANDI gui
• DANDI hub
• Dandisets