Model Types¶

Defines Model Types, which are containers for storables.

class dman.model.modelclasses.Config(auto_clean: bool = True)[source]¶

Configuration for model types.

This class has a global instance that can be accessed as follows:
>>> dman.model.modelclasses.config.auto_clean = True
>>> dman.params.model.auto_clean = True  # equivalent

Parameters:: auto_clean (bool, optional) – Automatically clean files associated with items that were removed from the container.

dman.model.modelclasses.get_record(obj, key: str, default=<dataclasses._MISSING_TYPE object>)[source]¶

Get the record for a field in a modelclass.

Parameters:

obj – The modelclass
key (str) – Key of the field
default (optional) – Default value. Defaults to MISSING.

dman.model.modelclasses.set_record(self, key: str, value: Record)[source]¶

Set the record for a field in a modelclass.

Parameters:

obj – The modelclass
key (str) – Key of the field
value (record) – The value to set.

dman.model.modelclasses.record_fields(obj) → dict[source]¶: Get all the records associated with storable fields in a modelclass.

dman.model.modelclasses.unused_fields(obj) → list[source]¶: Get all the records that are no longer used since they were overwritten.

dman.model.modelclasses.serializefield(*, default=<dataclasses._MISSING_TYPE object>, default_factory=<dataclasses._MISSING_TYPE object>, init: bool = True, repr: bool = True, hash: bool = False, compare: bool = True, metadata=None, pre: ~typing.Optional[~typing.Callable[[~typing.Any], ~typing.Any]] = None) → Field[source]¶

Return an object to identify serializable modelclass fields.: All arguments of the field method from dataclasses are provided. Moreover, record specific options are provided

Parameters:

default (optional) – is the default value of the field.
default_factory (optional) – is a 0-argument function called to initialize.
init (bool, optional) – Include in the class’s __init__(). Defaults to True.
repr (bool, optional) – Include in the object’s repr(). Defaults to True.
hash (bool, optional) – Include in the object’s hash(). Defaults to False.
compare (bool, optional) – Include in comparison functions. Defaults to True.
metadata (_type_, optional) – Additional information. Defaults to None.
pre (Callable[[Any], Any], optional) – Call method on field before setting. Defaults to None.

dman.model.modelclasses.is_serializable_field(fld: Field)[source]¶: Check if a field is explicitly registered as a serializable field.

dman.model.modelclasses.recordfield(*, default=<dataclasses._MISSING_TYPE object>, default_factory=<dataclasses._MISSING_TYPE object>, init: bool = True, repr: bool = False, hash: bool = False, compare: bool = False, metadata=None, stem: str = <dman.core.path._Auto object>, suffix: str = <dman.core.path._Auto object>, name: str = <dman.core.path._Auto object>, subdir: ~os.PathLike = <dman.core.path._Auto object>, preload: str = False, pre: ~typing.Optional[~typing.Callable[[~typing.Any], ~typing.Any]] = None) → Field[source]¶

Return an object to identify storable modelclass fields.: All arguments of the field method from dataclasses are provided. Moreover, record specific options are provided

Parameters:

default (optional) – The default value of the field.. Defaults to MISSING.
default_factory (optional) – A 0-argument function called to initialize. Defaults to MISSING.
init (bool, optional) – Include in the class’s __init__(). Defaults to True.
repr (bool, optional) – Include in the object’s repr(). Defaults to False.
hash (bool, optional) – Include in the object’s hash(). Defaults to False.
compare (bool, optional) – Include in comparison functions. Defaults to False.
metadata (_type_, optional) – Additional information. Defaults to None.
stem (str, optional) – The stem of the file. Defaults to AUTO.
suffix (str, optional) – The suffix or extension of the file (e.g. '.json'). Defaults to AUTO.
name (str, optional) – The full name of the file. Defaults to AUTO.
subdir (os.PathLike, optional) – The subdirectory in which to store te file. Defaults to AUTO.
preload (str, optional) – When True the file will be loaded during deserialization. Defaults to False.
pre (Callable[[Any], Any], optional) – Call method on field before setting. Defaults to None.

dman.model.modelclasses.register_preset(tp: Type, pre: Callable[[Any], Any])[source]¶

Register a preset method pre for a type tp.

When a field with that type is defined in a modelclass, pre is called before setting the field.

Example

>>> register_preset(barray, lambda arg:
>>>    arg.view(barray) if isinstance(arg, np.ndarray) else arg
>>> )

dman.model.modelclasses.modelclass(cls=None, /, *, name: Optional[str] = None, init=True, repr=True, eq=True, order=False, unsafe_hash=False, frozen=False, storable: bool = False, compact: bool = False, store_by_field: bool = False, cluster: bool = False, subdir: str = '', template: Optional[Any] = None, **kwargs) → Callable[[Type[_T]], Type[_T]][source]¶

Convert a class to a modelclass.

Returns the same class as was passed in, with dunder methods added based on the fields defined in the class. The class is automatically made serializable by adding __serialize__ and __deserialize__.

If an attribute __no_serialize__ is added, then the field names listed as string within will not be included in the serialization.

The arguments of the dataclass decorator are provided and some additional arguments are also available.

Parameters:

cls – Class to convert.
name (str, optional) – Name of the serializable (and storable if required). Defaults to class name.
init (bool, optional) – Add an __init__ method. Defaults to True.
repr (bool, optional) – Add a __repr__ method. Defaults to True.
eq (bool, optional) – Add a __eq__ method. Defaults to True.
order (bool, optional) – Add rich comparison methods. Defaults to False.
unsafe_hash (bool, optional) – Add a __hash__ method. Defaults to False.
frozen (bool, optional) – Fields may not be assigned after instance creation. Defaults to False.
storable (bool, optional) – Make the class storable with a __write__ and __read__. Defaults to False.
compact (bool, optional) – Do not include serializable types during serialization, making the result more compact. Defaults to False.
store_by_field (bool, optional) – The stem of files is determined by the field name. Defaults to False.
cluster (bool, optional) – Each file is stored in a subfolder determined by the field name. Defaults to False.
subdir (str, optional) – Store the files in a common subfolder. Defaults to “”.
template (Any, optional) – Template for serialization. Defaults to None.

dman.model.modelclasses.is_modelclass(cls)[source]¶: Check if a class is a modelclass.

dman.model.modelclasses.is_model(cls)[source]¶: Check if the provided class is a model type.

class dman.model.modelclasses.mlist(iterable: Optional[Iterable] = None, subdir: PathLike = '', preload: bool = False)[source]¶: Serializable Model List.

class dman.model.modelclasses.smlist(iterable: Optional[Iterable] = None, subdir: PathLike = '', preload: bool = False)[source]¶: Storable Model List.

class dman.model.modelclasses.mdict(*, subdir: PathLike = '', preload: bool = False, store_by_key: bool = False, store_subdir: bool = False)[source]¶: Serializable Model Dictionary.

class dman.model.modelclasses.smdict(*, subdir: PathLike = '', preload: bool = False, store_by_key: bool = False, store_subdir: bool = False)[source]¶: Storable Model Dictionary.

class dman.model.modelclasses.mruns(iterable: Optional[Iterable] = None, stem: str = 'run', subdir: PathLike = '', preload: bool = False, store_subdir: bool = True)[source]¶: Serializable Model Runs.

class dman.model.modelclasses.smruns(iterable: Optional[Iterable] = None, stem: str = 'run', subdir: PathLike = '', preload: bool = False, store_subdir: bool = True)[source]¶: Storable Model Runs.

dman.model.modelclasses.mlist_factory(subdir: PathLike = '', preload: bool = False)[source]¶: Factory for Model List. Useful in combination with modelclass fields.

dman.model.modelclasses.smlist_factory(subdir: PathLike = '', preload: bool = False)[source]¶: Factory for Storable Model List. Useful in combination with modelclass fields.

dman.model.modelclasses.mdict_factory(subdir: PathLike = '', preload: bool = False, store_by_key: bool = False, store_subdir: bool = False)[source]¶: Factory for Model Dict. Useful in combination with modelclass fields.

dman.model.modelclasses.smdict_factory(subdir: PathLike = '', preload: bool = False, store_by_key: bool = False, store_subdir: bool = False)[source]¶: Factory for Storable Model Dict. Useful in combination with modelclass fields.

dman.model.modelclasses.mruns_factory(stem: str = 'run', subdir: PathLike = '', preload: bool = False, store_subdir: bool = True)[source]¶: Factory for Model Runs. Useful in combination with modelclass fields.

dman.model.modelclasses.smruns_factory(stem: Optional[str] = None, subdir: PathLike = '', preload: bool = False, store_subdir: bool = True)[source]¶: Factory for Storable Model Runs. Useful in combination with modelclass fields.