Back to the projects list
NWB Extensions Infrastructure
Ryan Ly (LBNL) @rly
Yaroslav O. Halchenko (Dartmouth, DANDI) @yarikoptic
John T. Wodder (Dartmouth, DANDI) @jwodder
NWB supports user-created extensions to the standard to add support for data types from new technologies and lab- or experiment-specific data types. Many of these extensions are published in the NDX Catalog. It would be nice if PyNWB users had a way to easily install extensions that are found in the catalog.
- Publish catalog records for AIBS extensions that were not cached with the file.
- Establish a CI testing across versions of hdmf+PyNWB and extensions to ensure
- In PyNWB, update the error message for when an extension is not found. It should tell users to search for the extension in the NDX Catalog or run a command line tool to download and install the extension.
- Create a command line tool to search for a given extension in the NDX Catalog, download it, and install it.
Approach and Plan
- Cover following testing scenarios
- save/load cycle resulting in identical data structure
- have helper to produce “saved” .nwb files with clear “provenance” records to test against
- each extension should provide interface to produce “an example (lean but representative)” file
- ability to load NWB files saved by prior versions of hdmf/pynwb(/extension)
- ability to work with an extension version(s) in the declared “compatibility” range
- absent side-effects from having other extensions loaded
- Have tests regularly run and update status
- Have (subset?) of testing across extensions ran for PyNWB PRs
- Borrow ideas/setup from possible existing setups:
- https://github.com/datalad/datalad-extensions/ - github/github-actions driven dashboard for DataLad extensions
@yarikoptic had some related setup in elderly PyMVPA to ensure that h5save’d files could be later h5loaded
Progress and Next Steps
Background and References
NDX Catalog: https://nwb-extensions.github.io/