src.trainer

Module Contents

Classes

Trainer

The Trainer class orchestrates the training, logging, checkpointing, and evaluation (rollout) of a meta-optimization agent on a set of problems. It manages reproducibility, parallelization modes, and integration with logging tools such as TensorBoard.

API

class src.trainer.Trainer(config, user_agent, user_optimizer, user_datasets)

Bases: object

The Trainer class orchestrates the training, logging, checkpointing, and evaluation (rollout) of a meta-optimization agent on a set of problems. It manages reproducibility, parallelization modes, and integration with logging tools such as TensorBoard.

Initialization

Introduction

Initializes the trainer with configuration, agent, optimizer, and datasets. Sets up random seeds for reproducibility, determines the problem dimensionality, and configures parallel training mode based on agent and device.

Args:

• config (object): Configuration object containing training parameters and settings.

  • The Attributes needed for the Trainer are the following:

    • train_parallel_mode (str): Mode for parallel training, can be ‘subproc’, ‘thread’, or ‘dummy’. Default is dummy.

    • seed (int): Random seed for reproducibility across runs.Default is 3849.

    • run_time (str): Run identifier used for naming output files and directories.

    • train_problem (str): Type of training problem, e.g., ‘cec’, ‘bbob’, etc.

    • train_difficulty (str): Difficulty level of training, e.g., ‘easy’, ‘medium’, ‘hard’.

    • device (str): Computing device, can be ‘cpu’ or a CUDA device.Default is cpu.

    • train_batch_size (int): Batch size for training, determines number of environments per iteration.Default is 1.

    • train_mode (str): Training mode, can be “single” or “multi”.Default is single.

    • end_mode (str): Ending condition mode, can be “step” or “epoch”.Default is epoch.

    • max_epoch (int): Maximum number of training epochs when end_mode is “epoch”.

    • save_interval (int): Interval for saving checkpoints (in epochs or steps).

    • agent_save_dir (str): Directory path for saving agent models.

    • log_dir (str): Directory path for saving logs.

    • no_tb (bool): Whether to disable TensorBoard logging, default is False.

• user_agent (object): The agent to be trained.

• user_optimizer (torch.optim.Optimizer): Optimizer for training the agent.

• user_datasets (tuple): Tuple containing the training and testing datasets.

Raises:

• UserWarning: If sub-process parallel training mode is selected with certain agents on CUDA devices, a warning is issued and the mode is changed to ‘dummy’.

save_log(epochs, steps, cost, returns, normalizer)

Introduction

Saves training logs including epochs, steps, costs, returns, and normalizer values for each problem in the training set. The logs are saved as NumPy arrays in a structured directory based on agent class and runtime.

Args:

• epochs (list or np.ndarray): List or array of epoch indices.

• steps (list or np.ndarray): List or array of step counts corresponding to each epoch.

• cost (dict): Dictionary mapping problem names to lists of cost values per epoch.

• returns (list or np.ndarray): List or array of return values per step.

• normalizer (dict): Dictionary mapping problem names to lists of normalizer values per epoch.

Returns:

• None

Notes:

• Creates the log directory if it does not exist.

• Pads cost and normalizer lists with their last value if they are shorter than the number of epochs.

• Saves logs as .npy files for later analysis.

save_class(file_name, saving_class)

Introduction

Saves a given class instance to a specified directory as a pickle (.pkl) file.

Args:

• dir (str): The directory path where the file will be saved. If the directory does not exist, it will be created.

• file_name (str): The name of the file (without extension) to save the class instance as.

• saving_class (object): The class instance or object to be serialized and saved.

Returns:

• None

Raises:

• OSError: If the directory cannot be created or the file cannot be written.

• pickle.PicklingError: If the object cannot be pickled.

train()

Trains the agent using the specified training configuration and dataset.

This method orchestrates the training process, including setting up the training environment, managing epochs, logging progress, and saving checkpoints. It supports different training modes (“single” and “multi”) and integrates with TensorBoard for logging.

Attributes: -self.config (object): Configuration object containing training parameters. -self.train_set (object): Dataset object containing the training problems. -self.agent (object): The agent to be trained. -self.optimizer (object): Optimizer used for training.

Workflow: 1. Initializes TensorBoard logger if enabled. 2. Configures batch size and training mode based on the configuration. 3. Iteratively trains the agent for each epoch until the stopping condition is met. 4. Logs training progress using tqdm and TensorBoard. 5. Saves checkpoints at specified intervals. 6. Handles random seed management for reproducibility.

Returns: None