Implementing a new Neurodata Type

Table of Contents

• How to Implement a RegisteredType
  • Example: Implementing a new type
• How the Type Registry in RegisteredType Works
• How to Use the RegisteredType Registry
  • Example: Using the type registry
• How to Use the DEFINE_FIELD macro

New neurodata_types typically inherit from at least either Container or Data, or a more specialized type of the two. In any case, all classes that represent a neurodata_type defined in the schema should be implemented as a subtype of RegisteredType.

How to Implement a RegisteredType

To implement a subclass of RegisteredType, follow these steps:

1. Include the RegisteredType.hpp header file in your subclass header file.

   #include "nwb/RegisteredType.hpp"

2. Define your subclass by inheriting from RegisteredType. Ensure that your subclass implements a constructor with the arguments (const std::string& path, std::shared_ptr<IO::BaseIO> io), as the "create" method expects this constructor signature.

   class MySubClass : public AQNWB::NWB::RegisteredType {
   public:
       MySubClass(const std::string& path, std::shared_ptr<IO::BaseIO> io)
           : RegisteredType(path, io) {}
    
       // Implement any additional methods or overrides here
   };

3. Use the REGISTER_SUBCLASS macro to register your subclass. This should usually appear in the header (hpp) file as part of the class definition:

   REGISTER_SUBCLASS(MySubClass, "my-namespace")

4. In the corresponding source (cpp) file, initialize the static member to trigger the registration using the REGISTER_SUBCLASS_IMPL macro:

   #include "MySubClass.h"
    
   // Initialize the static member to trigger registration
   REGISTER_SUBCLASS_IMPL(MySubClass)

5. To define getter methods for lazy read access to datasets and attributes that belong to our type, we can use the DEFINE_FIELD macro. This macro creates a standard method for retrieving a ReadDataWrapper for lazy reading for the field:

   DEFINE_FIELD(getData, DatasetField, float, "data", The main data)

Warning

To ensure proper function on read, the name of the class should match the name of the neurodata_type as defined in the schema. Similarly, "my-namespace" should match the name of the namespace in the schema (e.g., "core", "hdmf-common"). In this way we can look up the corresponding class for an object in a file based on the neurodata_type and namespace attributes stored in the file.

Note

A special version of the REGISTER_SUBCLASS macro, called REGISTER_SUBCLASS_WITH_TYPENAME, allows setting the typename explicitly as a third argument. This is for the special case where we want to implement a class for a modified type that does not have its own neurodata_type in the NWB schema. An example is ElectrodesTable in NWB <v2.7, which did not have an assigned neurodata_type, but was implemented as a regular DynamicTable. To allow us to define a class ElectrodeTable to help with writing the table we can then use REGISTER_SUBCLASS_WITH_TYPENAME(ElectrodeTable, "core", "DynamicTable") in the ElectrodesTable class. This ensures that the neurodata_type attribute is set correctly to DynamicTable on write instead of ElectrodesTable. However, on read this will by default not use the ElectrodesTable class but the regular DynamicTable class since that is what the schema is indicating to use. In the registry, the class will still be registered under the core::ElectrodesTable key, but with "DynamicTable" as the typename value and the ElectrodesTable.getTypeName automatic override returning the indicated typename instead of the classname.

DEFINE_FIELD creates templated, non-virtual read functions. This means if we want to "redefine" a field in a child class by calling DEFINE_FIELD again, then the function will be "hidden" instead of "override". This is important to remember when casting a pointer to a base type, as in this case the implementation from the base type will be used since the function created by DEFINE_FIELD is not virtual.

Example: Implementing a new type

MySubClass.hpp

#pragma once
#include "RegisteredType.hpp"
 
class MySubClass : public AQNWB::NWB::RegisteredType
{
public:
    MySubClass(const std::string& path, std::shared_ptr<IO::BaseIO> io)
        : RegisteredType(path, io) {}
 
    DEFINE_FIELD(getData, DatasetField, float, "data", The main data)
 
    REGISTER_SUBCLASS(MySubClass, "my-namespace")
};

MySubClass.cpp

#include "MySubClass.h"
 
// Initialize the static member to trigger registration
REGISTER_SUBCLASS_IMPL(MySubClass)

How the Type Registry in RegisteredType Works

The type registry in RegisteredType allows for dynamic creation of registered subclasses by name. Here is how it works:

1. Registry Storage:
   • The registry is implemented using 1) an std::unordered_set to store subclass names (which can be accessed via getRegistry()) and 2) an std::unordered_map to store factory functions for creating instances of the subclasses (which can be accessed via getFactoryMap()). The factory methods are the required constructor that uses the io and path as input.
   • These are defined as static members within the RegisteredType class.
2. Registration:
   • The registerSubclass method is used to add a subclass name and its corresponding factory function to the registry.
   • This method is called via the REGISTER_SUBCLASS macro, which defines a static method (registerSubclass()) and static member (registered_) to trigger the registration when the subclass is loaded.
3. Dynamic Creation:
   • The create method is used to create an instance of a registered subclass by name.
   • This method looks up the subclass name in the registry and calls the corresponding factory function to create an instance.
4. Automatic Registration:
   • The REGISTER_SUBCLASS_IMPL macro initializes the static member (registered_), which triggers the registerSubclass method and ensures that the subclass is registered when the program starts.
5. Class Name and Namespace Retrieval:
   • The getTypeName and getNamespace return the string name of the class and namespace, respectively. The REGISTER_SUBCLASS macro implements an automatic override of the methods to ensure the appropriate type and namespace string are returned. These methods should, hence, not be manually overridden by subclasses, to ensure consistency in type identification.

How to Use the RegisteredType Registry

The RegisteredType registry allows for dynamic creation and management of registered subclasses. Here is how you can use it:

1. Creating Instances Dynamically:
   • Use the create method to create an instance of a registered subclass by name.
   • This method takes the subclass name, path, and a shared pointer to the IO object as arguments. This illustrates how we can read a specific typed object in an NWB file.

         // Create an instance of an TimeSeries in a file.
         auto instance =
             AQNWB::NWB::RegisteredType::create("core::TimeSeries", dataPath, io);
         REQUIRE(instance != nullptr);

2. Retrieving Registered Subclass Names:
   • Use the getRegistry method to retrieve the set of registered subclass names.

         // Retrieve and print registered subclass names
         const auto& registry = AQNWB::NWB::RegisteredType::getRegistry();
         std::cout << "Registered subclasses:" << std::endl;
         for (const auto& subclassName : registry) {
           std::cout << " - " << subclassName << std::endl;
         }

3. Retrieving the Factory Map:
   • Use the getFactoryMap method to retrieve the map of factory functions for creating instances of registered subclasses.

         // Retrieve and print factory map
         const auto& factoryMap = AQNWB::NWB::RegisteredType::getFactoryMap();
         std::cout << "Factory functions for registered subclasses:" << std::endl;
         for (const auto& pair : factoryMap) {
           std::cout << " - " << pair.first << std::endl;
         }

Example: Using the type registry

#include "io/BaseIO.hpp"
#include "io/hdf5/HDF5IO.hpp"
#include "io/hdf5/HDF5RecordingData.hpp"
#include "nwb/RegisteredType.hpp"
#include "nwb/base/TimeSeries.hpp"
#include "testUtils.hpp"
 
using namespace AQNWB::NWB;
 
TEST_CASE("RegisterType Example", "[base]")
{
  SECTION("Example to illustrate how the RegisterType registry is working")
  {
    // [example_RegisterType_setup_file]
    // Mock data
    SizeType numSamples = 10;
    std::string dataPath = "/tsdata";
    std::vector<SizeType> dataShape = {numSamples};
    std::vector<SizeType> positionOffset = {0};
    BaseDataType dataType = BaseDataType::F32;
    std::vector<float> data = getMockData1D(numSamples);
    std::vector<double> timestamps = getMockTimestamps(numSamples, 1);
 
    std::string filename = getTestFilePath("testRegisteredTypeExample.h5");
    std::shared_ptr<BaseIO> io = std::make_unique<IO::HDF5::HDF5IO>(filename);
    io->open();
    NWB::TimeSeries ts = NWB::TimeSeries(dataPath, io);
    ts.initialize(dataType, "unit");
 
    // Write data to file
    Status writeStatus =
        ts.writeData(dataShape, positionOffset, data.data(), timestamps.data());
    REQUIRE(writeStatus == Status::Success);
    io->flush();
    // [example_RegisterType_setup_file]
 
    // [example_RegisterType_get_type_instance]
    // Create an instance of an TimeSeries in a file.
    auto instance =
        AQNWB::NWB::RegisteredType::create("core::TimeSeries", dataPath, io);
    REQUIRE(instance != nullptr);
    // [example_RegisterType_get_type_instance]
 
    // [example_RegisterType_get_registered_names]
    // Retrieve and print registered subclass names
    const auto& registry = AQNWB::NWB::RegisteredType::getRegistry();
    std::cout << "Registered subclasses:" << std::endl;
    for (const auto& subclassName : registry) {
      std::cout << " - " << subclassName << std::endl;
    }
    // [example_RegisterType_get_registered_names]
 
    // [example_RegisterType_get_registered_factories]
    // Retrieve and print factory map
    const auto& factoryMap = AQNWB::NWB::RegisteredType::getFactoryMap();
    std::cout << "Factory functions for registered subclasses:" << std::endl;
    for (const auto& pair : factoryMap) {
      std::cout << " - " << pair.first << std::endl;
    }
    // [example_RegisterType_get_registered_factories]
  }
}
 

How to Use the DEFINE_FIELD macro

The DEFINE_FIELD macro takes the following main inputs:

• name: The name of the function to generate.
• storageObjectType : One of either DatasetField or AttributeField to define the type of storage object used to store the field.
• default_type : The default data type to use. If not known, we can use std::any.
• fieldPath : Literal string with the relative path to the field within the schema of the respective neurodata_type. This is automatically being expanded at runtime to the full path.
• description : Description of the field to include in the docstring for the docs

All of these inputs are required. A typical example will look as follows:

DEFINE_FIELD(getData, DatasetField, float, "data", The main data)

The compiler will then expand this definition to create a new method called getData that will return a ReadDataWrapper for lazy reading for the field. The corresponding expanded function will look something like:

template<typename VTYPE = float>
inline std::unique_ptr<IO::ReadDataWrapper<DatasetField, VTYPE>> getData() const
{
    return std::make_unique<IO::ReadDataWrapper<DatasetField, VTYPE>>(
         m_io,
         AQNWB::mergePaths(m_path, fieldPath));
}

See Reading data for an example of how to use such methods (e.g., TimeSeries::readData ) for reading data fields from a file.