Skip to content

Trainers

Reference information for the Trainers API.

eva.core.trainers.Trainer

Bases: Trainer

Core trainer class.

This is an extended version of lightning's core trainer class.

For the input arguments, refer to ::class::lightning.pytorch.Trainer.

Parameters:

Name Type Description Default
args Any

Positional arguments of ::class::lightning.pytorch.Trainer.

 ()
default_root_dir str

The default root directory to store the output logs. Unlike in ::class::lightning.pytorch.Trainer, this path would be the prioritized destination point.

 'logs'
n_runs int

The amount of runs (fit and evaluate) to perform in an evaluation session.

 1
kwargs Any

Kew-word arguments of ::class::lightning.pytorch.Trainer.

 {}
Source code in src/eva/core/trainers/trainer.py 
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
@argparse._defaults_from_env_vars
def __init__(
    self,
    *args: Any,
    default_root_dir: str = "logs",
    n_runs: int = 1,
    **kwargs: Any,
) -> None:
    """Initializes the trainer.

    For the input arguments, refer to ::class::`lightning.pytorch.Trainer`.

    Args:
        args: Positional arguments of ::class::`lightning.pytorch.Trainer`.
        default_root_dir: The default root directory to store the output logs.
            Unlike in ::class::`lightning.pytorch.Trainer`, this path would be the
            prioritized destination point.
        n_runs: The amount of runs (fit and evaluate) to perform in an evaluation session.
        kwargs: Kew-word arguments of ::class::`lightning.pytorch.Trainer`.
    """
    super().__init__(*args, default_root_dir=default_root_dir, **kwargs)

    self._n_runs = n_runs

    self._session_id: str = _logging.generate_session_id()
    self._log_dir: str = self.default_log_dir

    self.setup_log_dirs()

default_log_dir: str property

Returns the default log directory.

setup_log_dirs

Setups the logging directory of the trainer and experimental loggers in-place.

Parameters:

Name Type Description Default
subdirectory str

Whether to append a subdirectory to the output log.

 ''
Source code in src/eva/core/trainers/trainer.py 
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
def setup_log_dirs(self, subdirectory: str = "") -> None:
    """Setups the logging directory of the trainer and experimental loggers in-place.

    Args:
        subdirectory: Whether to append a subdirectory to the output log.
    """
    self._log_dir = os.path.join(self.default_root_dir, self._session_id, subdirectory)

    enabled_loggers = []
    if isinstance(self.loggers, list) and len(self.loggers) > 0:
        for logger in self.loggers:
            if isinstance(logger, (pl_loggers.CSVLogger, pl_loggers.TensorBoardLogger)):
                if not cloud_io._is_local_file_protocol(self.default_root_dir):
                    loguru.logger.warning(
                        f"Skipped {type(logger).__name__} as remote storage is not supported."
                    )
                    continue
                else:
                    logger._root_dir = self.default_root_dir
                    logger._name = self._session_id
                    logger._version = subdirectory
            enabled_loggers.append(logger)

    self._loggers = enabled_loggers or [eva_loggers.DummyLogger(self._log_dir)]

run_evaluation_session

Runs an evaluation session out-of-place.

It performs an evaluation run (fit and evaluate) the model self._n_run times. Note that the input base_model would not be modified, so the weights of the input model will remain as they are.

Parameters:

Name Type Description Default
model ModelModule

The base model module to evaluate.

 required
datamodule DataModule

The data module.

 required
Source code in src/eva/core/trainers/trainer.py 
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
def run_evaluation_session(
    self,
    model: modules.ModelModule,
    datamodule: datamodules.DataModule,
) -> None:
    """Runs an evaluation session out-of-place.

    It performs an evaluation run (fit and evaluate) the model
    `self._n_run` times. Note that the input `base_model` would
    not be modified, so the weights of the input model will remain
    as they are.

    Args:
        model: The base model module to evaluate.
        datamodule: The data module.
    """
    functional.run_evaluation_session(
        base_trainer=self,
        base_model=model,
        datamodule=datamodule,
        n_runs=self._n_runs,
        verbose=self._n_runs > 1,
    )