src.environment.problem.SOO.NE.evox_ne

Module Contents

Classes

MLP

NE_Problem

Introduction This problem set is based on the neuroevolution interfaces in EvoX. The goal is to optimize the parameters of neural network-based RL agents for a series of Robotic Control tasks. We pre-define 11 control tasks (e.g., swimmer, ant, walker2D etc.), and 6 MLP structures with 0~5 hidden layers. The combinations of task & network structure result in 66 problem instances, which feature extremely high-dimensional problems (>=1000D).

Data

envs
model_depth

API

class src.environment.problem.SOO.NE.evox_ne.MLP(state_dim, action_dim, hidden_layer_num)

Bases: torch.nn.Module

Initialization

forward(state)

src.environment.problem.SOO.NE.evox_ne.envs

None

src.environment.problem.SOO.NE.evox_ne.model_depth

[0, 1, 2, 3, 4, 5]

class src.environment.problem.SOO.NE.evox_ne.NE_Problem(env_name, model_depth, seed)

Bases: src.environment.problem.basic_problem.Basic_Problem

Introduction

This problem set is based on the neuroevolution interfaces in EvoX. The goal is to optimize the parameters of neural network-based RL agents for a series of Robotic Control tasks. We pre-define 11 control tasks (e.g., swimmer, ant, walker2D etc.), and 6 MLP structures with 0~5 hidden layers. The combinations of task & network structure result in 66 problem instances, which feature extremely high-dimensional problems (>=1000D).

Original paper

“EvoX: A distributed GPU-accelerated framework for scalable evolutionary computation.” IEEE Transactions on Evolutionary Computation (2024).

Official Implementation

NE

License

None

Initialization

Introduction

Initializes the environment and neural network model for a single-objective optimization (SOO) problem using neuroevolution.

Args:

• env_name (str): The name of the environment to be used.

• model_depth (int): The number of layers or depth of the neural network model.

• seed (int): The random seed for reproducibility.

Attributes:

• env_state_dim (int): Dimension of the environment’s state space.

• env_action_dim (int): Dimension of the environment’s action space.

• nn_model (MLP): The neural network model used for policy representation.

• dim (int): Total number of parameters in the neural network model.

• ub (float): Upper bound for parameter initialization.

• lb (float): Lower bound for parameter initialization.

• seed (int): Random seed for reproducibility.

• env_name (str): Name of the environment.

• model_depth (int): Depth of the neural network model.

• optimum (Any): Placeholder for the optimum solution (default: None).

• init (bool): Flag indicating if initialization is complete (default: False).

reset()

Introduction

Resets the state of the object by initializing the neural network model, adapter, evaluator, and a timer variable.

Args:

None

Returns:

None

Raises:

• RuntimeError: If CUDA is not available or the model cannot be moved to CUDA.

• ImportError: If evox.utils.ParamsAndVector cannot be imported.

__str__()

Introduction

Returns a string representation of the environment, combining its name and model depth.

Returns:

• str: A string in the format “{env_name}-{model_depth}” representing the environment.

func(x)

Introduction

Evaluates a batch of neural network parameter sets in a Brax-based neuroevolution environment and returns their fitness scores.

Args:

• x (numpy.ndarray): A batch of neural network parameters with shape (batch_size, num_params).

Returns:

• numpy.ndarray: An array of fitness scores for each parameter set in the batch, where higher values indicate better performance.

Raises:

• AssertionError: If the last dimension of x does not match the expected problem dimension (self.dim).

Notes:

• The function initializes or updates the evaluator (BraxProblem) if necessary, based on the population size.

• Handles NaN and infinite rewards by assigning a large negative value.

• Converts rewards to a minimization objective by subtracting them from a large constant (1e5).