NWB Workshops and Hackathons

NWB Extensions Infrastructure

Key Investigators

Ryan Ly (LBNL) @rly

Helping hands

Yaroslav O. Halchenko (Dartmouth, DANDI) @yarikoptic John T. Wodder (Dartmouth, DANDI) @jwodder

Project Description

NWB supports user-created extensions to the standard to add support for data types from new technologies and lab- or experiment-specific data types. Many of these extensions are published in the NDX Catalog. It would be nice if PyNWB users had a way to easily install extensions that are found in the catalog.

Objectives

Publish catalog records for AIBS extensions that were not cached with the file.
Establish a CI testing across versions of hdmf+PyNWB and extensions to ensure
In PyNWB, update the error message for when an extension is not found. It should tell users to search for the extension in the NDX Catalog or run a command line tool to download and install the extension.
Create a command line tool to search for a given extension in the NDX Catalog, download it, and install it.

Approach and Plan

2. CI

Cover following testing scenarios
- save/load cycle resulting in identical data structure
  - have helper to produce “saved” .nwb files with clear “provenance” records to test against
  - each extension should provide interface to produce “an example (lean but representative)” file
- ability to load NWB files saved by prior versions of hdmf/pynwb(/extension)
- ability to work with an extension version(s) in the declared “compatibility” range
- absent side-effects from having other extensions loaded
Have tests regularly run and update status
Have (subset?) of testing across extensions ran for PyNWB PRs
Borrow ideas/setup from possible existing setups:
- https://github.com/datalad/datalad-extensions/ - github/github-actions driven dashboard for DataLad extensions
- @yarikoptic had some related setup in elderly PyMVPA to ensure that h5save’d files could be later h5loaded

Progress and Next Steps

Materials

Background and References

NDX Catalog: https://nwb-extensions.github.io/