Logging¶

We illustrate basic dman logging functionality.

When using dman you might have seen certain warning messages pop up, originating from the logger. This logger can be configured using dman.log.config().

import dman

dman.log.config(level=dman.log.INFO)
dman.log.info("This is some info.")

[01/04/23 10:12:24] INFO     This is some info.           example5_logging.py:15

The syntax is similar to that of logging.basicConfig().

dman.log.config(*, level=None, filename: Optional[str] = None, filemode: str = 'a', stream=None, format: Optional[str] = None, datefmt: Optional[str] = None, handlers: Optional[List[Handler]] = None, force: bool = False, **kwargs)

Configure the dman logger.

This function does nothing if the dman logger already has handlers configured, unless if the force keyword argument is set to true.

The default behavior is to create a DmanHandler using the default_handler() method. Any keyword arguments not specified below are passed to that method.

Parameters:

level (_type_, optional) – Set the logger level to the specified level.
filename (str, optional) – Specifies that a FileHandler be created, using the specified filename, rather than a StreamHandler.
filemode (str, optional) – Specifies the mode to open the file, if filename is specified (if filemode is unspecified, it defaults to ‘a’).
stream (optional) – Use the specified stream to initialize the StreamHandler. Note that this argument is incompatible with ‘filename’ - if both are present, ‘stream’ is ignored.
format (str, optional) – Use the specified format string for the handler.
datefmt (str, optional) – Use the specified date/time format.
handlers (List[backend.Handler], optional) – If specified, this should be an iterable of already created handlers, which will be added to the root handler. Any handler in the list which does not have a formatter assigned will be assigned the formatter created in this function.
force (bool, optional) – If this keyword is specified as true, any existing handlers attached to the root logger are removed and closed, before carrying out the configuration as specified by the other arguments.. Defaults to False.

Note that you could specify a stream created using open(filename, mode) rather than passing the filename and mode in. However, it should be remembered that StreamHandler does not close its stream (since it may be using sys.stdout or sys.stderr), whereas FileHandler closes its stream when the handler is closed.

By default we use the RichHandler from rich. If you want to use more standard logging you can use the following configuration

dman.log.config(
    level=dman.log.INFO, use_rich=False, datefmt="%Y-%m-%d %H:%M:%S", force=True
)
dman.log.backend.basicConfig
dman.log.info("This is some info.")

2023-01-04 10:12:24 INFO: This is some info.

The dman logger also supports some additional functionality.

dman.log.config(level=dman.log.INFO, force=True, show_path=False)
with dman.log.layer("example", "layer", prefix="owner"):
    dman.log.info("Indented", label="example")
    dman.log.io("This is an io command", label="example")
    dman.log.emphasize("This is an emphasized command", label="example")

[01/04/23 10:12:24] INFO     <layer owner=example>
                    INFO        [example]: Indented
                    INFO        [example]: This is an io command
                    INFO        [example]: This is an emphasized command
                    INFO     <end layer owner=example>

It is also used extensively during saving, loading and other internal functionality.

_ = dman.save("person", {"name": "Adam", "age": 25, "position": [23, 12]})

INFO     <saving key=person>
INFO        [mount]: Creating empty directory
         ".dman/cache/examples:fundamentals:example5_logging
         /person".
INFO        [save]: saving dict with key "person" to
         ".dman/cache/examples:fundamentals:example5_logging
         /person/person.json".
INFO        <serializing type=dict(3)>
INFO          <serializing type=list(2)>
INFO          <end serializing type=list(2)>
INFO        <end serializing type=dict(3)>
INFO        [save]: finished saving dict with key "person"
         to
         ".dman/cache/examples:fundamentals:example5_logging
         /person/person.json".
INFO     <end saving key=person>

Since these logs can be quite verbose when you set the level to INFO, it can be useful to save the file somewhere. In fact you can do so within the dman file tree.

@dman.modelclass
class Experiment:
    value: int
    log: dman.FileTarget = dman.recordfield(
        name="dman.log", default_factory=dman.FileTarget
    )


exp = Experiment(25)
dman.log.config(
    level=dman.log.INFO, stream=exp.log, force=True, console_style={"width": 160}
)
dman.clean("experiment", generator="")
dman.save("experiment", exp, generator="")
dman.tui.walk_directory(
    dman.mount("experiment", generator=""),
    show_content=True,
    console=dman.tui.Console(width=180),
)

📂 .dman/experiment
┣━━ 📄 dman.log (2.4 kB)
┃    ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
┃     [01/04/23 10:12:24] INFO     <loading key=experiment>                                                                                          repository.py:294
┃                         INFO        [load]: file not available at ".dman/experiment/experiment.json", using default                                repository.py:296
┃                         INFO     <end loading key=experiment>                                                                                      repository.py:294
┃                         INFO     <saving key=experiment>                                                                                           repository.py:217
┃                         INFO        [mount]: Creating empty directory ".dman/experiment".                                                                path.py:418
┃                         INFO        [save]: saving Experiment with key "experiment" to ".dman/experiment/experiment.json".                         repository.py:220
┃                         INFO        <serializing type=Experiment>                                                                               serializables.py:626
┃                         INFO          [modelclass]: serializing modelclass with fields ['value', 'log'].                                         modelclasses.py:499
┃                         INFO          [modelclass]: serializing value of type: "int"                                                             modelclasses.py:521
┃                         INFO          [modelclass]: serializing log of type: "Record"                                                            modelclasses.py:521
┃                         INFO          <serializing type=Record>                                                                                 serializables.py:626
┃                         INFO          <end serializing type=Record>                                                                             serializables.py:626
┃                         INFO        <end serializing type=Experiment>                                                                           serializables.py:626
┃                         INFO        [save]: finished saving Experiment with key "experiment" to ".dman/experiment/experiment.json".                repository.py:227
┃                         INFO     <end saving key=experiment>                                                                                       repository.py:217
┃
┃    ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
┗━━ 📄 experiment.json (285 bytes)
     ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
      {
        "_ser__type": "Experiment",
        "_ser__content": {
          "value": 25,
          "log": {
            "_ser__type": "_ser__record",
            "_ser__content": {
              "target": "dman.log",
              "sto_type": "_storable__stream"
            }
          }
        }
      }
     ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Total running time of the script: ( 0 minutes 0.056 seconds)

Gallery generated by Sphinx-Gallery