src.environment.optimizer.rldeafl_optimizer

Module Contents

Classes

select_mutation
select_crossover
RLDEAFL_Optimizer	Introduction: RLDEAFL_Optimizer is a reinforcement learning-based Differential Evolution with Adaptive Feature Learning optimizer. It is designed to solve continuous optimization problems by adaptively selecting mutation and crossover operators using reinforcement learning strategies. The optimizer maintains a population of candidate solutions, applies evolutionary operators, and tracks the best solution found during the optimization process.
Basic_mutation	This class represents a basic mutation. Methods:
Basic_crossover
DE1
DE2
DE3
DE4
DE5
DE6
DE7
DE8
DE9
DE10
DE11
DE12
DE13
DE14
CR1
CR2
CR3

Data

crossover_operators
mutation_operators

API

src.environment.optimizer.rldeafl_optimizer.crossover_operators

[‘CR1’, ‘CR2’, ‘CR3’]

src.environment.optimizer.rldeafl_optimizer.mutation_operators

[‘DE1’, ‘DE2’, ‘DE3’, ‘DE4’, ‘DE5’, ‘DE6’, ‘DE7’, ‘DE8’, ‘DE9’, ‘DE10’, ‘DE11’, ‘DE12’, ‘DE13’, ‘DE1…

class src.environment.optimizer.rldeafl_optimizer.select_mutation(rng)

Initialization

Introduction

Initializes the optimizer with a given random number generator and sets up mutation operators.

Args:

• rng: A random number generator instance used for stochastic operations.

Built-in Attributes:

• self.rng: Stores the provided random number generator.

• self.mutation_operators (list): List of mutation operator names to be used.

• self.operators (dict): Dictionary mapping operator names to their instantiated objects.

• self.n_operator (int): Number of mutation operators.

Notes:

• The mutation_operators variable is expected to be defined elsewhere in the class or module.

• Each operator is instantiated using its name and the provided random number generator.

select_mutation_operator(mutation_operator)

Introduction

Selects and returns the mutation operator class based on the provided mutation operator key.

Args:

• mutation_operator (str or int): The key or index used to identify the desired mutation operator.

Returns:

• type: The class corresponding to the selected mutation operator.

Raises:

• KeyError: If the provided mutation_operator does not exist in the mutation_operators mapping.

class src.environment.optimizer.rldeafl_optimizer.select_crossover(rng)

Initialization

Introduction

Initializes the optimizer with a random number generator and sets up crossover operators.

Args:

• rng: A random number generator instance used for stochastic operations.

Attributes:

• self.rng: Stores the provided random number generator.

• self.crossover_operators: A list or collection of crossover operator names.

• self.operators: A dictionary mapping operator names to their instantiated objects.

• self.n_operator: The number of crossover operators available.

Notes:

Assumes that crossover_operators is defined in the scope and that each operator name corresponds to a callable class or function.

select_crossover_operator(crossover_operator)

Introduction

Selects and returns the crossover operator class based on the provided operator key.

Args:

• crossover_operator (str): The key or identifier for the desired crossover operator.

Returns:

• type: The class corresponding to the selected crossover operator.

Raises:

• KeyError: If the provided crossover_operator is not found in the available operators.

class src.environment.optimizer.rldeafl_optimizer.RLDEAFL_Optimizer(config)

Bases: src.environment.optimizer.learnable_optimizer.Learnable_Optimizer

Introduction:

RLDEAFL_Optimizer is a reinforcement learning-based Differential Evolution with Adaptive Feature Learning optimizer. It is designed to solve continuous optimization problems by adaptively selecting mutation and crossover operators using reinforcement learning strategies. The optimizer maintains a population of candidate solutions, applies evolutionary operators, and tracks the best solution found during the optimization process.

Paper:

Reinforcement Learning-based Self-adaptive Differential Evolution through Automated Landscape Feature Learning

Implementation:

RLDEAFL

Initialization

Introduction

Initializes the RLDEAFLOptimizer with the provided configuration, setting up internal parameters for mutation, crossover, population size, and logging.

Args:

• config (object): Configuration object containing optimizer settings such as maxFEs and log_interval.

  • The Attribute needed for the RLDEAFL_Optimizer:

    • maxFEs (int): Maximum number of function evaluations.

    • log_interval (int): Interval for logging progress.

    • n_logpoint (int): Number of log points to record.

    • full_meta_data (bool): Flag indicating whether to use full meta data.

Built-in Attribute:

• self.__config: Stores the configuration object.

• self.__mu_operator (int): Number of mutation operators. Default is 14.

• self.__cr_operator (int): Number of crossover operators. Default is 3.

• self.__n_mutation (int): Number of mutation strategies. Default is 3.

• self.__n_crossover (int): Number of crossover strategies. Default is 2.

• self.__NP (int): Population size. Default is 100.

• self.__reward_ratio (int): Reward ratio for operator selection. Default is 1.

• self.__mu_selector: Placeholder for mutation operator selector.Default is None.

• self.__cr_selector: Placeholder for crossover operator selector.Default is None.

• self.log_index: Index for logging.Default is None.

Returns:

• None

__str__()

Returns a string representation of the RLDEAFL_Optimizer class.

Returns:

str: The name of the optimizer, "RLDEAFL_Optimizer".

get_costs(position, problem)

Introduction

Evaluates the cost(s) of a given position or set of positions for a specified optimization problem, applying boundary constraints and adjusting for the problem’s optimum if available.

Args:

• position (np.ndarray): The position(s) to be evaluated, typically in normalized [0, 1] space.

• problem (object): The optimization problem instance, expected to have lb, ub, eval, and optimum attributes.

Built-in Attribute:

• self.fes (int): Increments the function evaluation counter by the number of positions evaluated.

Returns:

• cost (float or np.ndarray): The evaluated cost(s) for the given position(s), optionally shifted by the problem’s optimum.

Raises:

• AttributeError: If the problem object does not have the required attributes (lb, ub, eval, optimum).

observe()

Introduction

Observes the current state of the optimizer by normalizing the current vector, recording the current fitness, and tracking the progress as a fraction of function evaluations.

Returns:

• np.ndarray: A 2D array where each row contains the normalized current vector, its corresponding fitness value, and the normalized number of function evaluations.

Notes:

• The normalization of the current vector assumes the lower and upper bounds are 0 and 1, respectively.

• The function evaluation step (fes) is normalized by the maximum allowed function evaluations (max_fes).

init_population(problem)

Introduction

Initializes the population for the RLDEAF optimizer, setting up the initial candidate solutions, their fitness values, and relevant optimizer state variables.

Args:

• problem (object): An optimization problem instance that must have dim, ub, and lb attributes, representing the dimensionality, upper bounds, and lower bounds of the search space, respectively.

Built-in Attribute:

• self.__dim (int): Dimensionality of the problem.

• self.__mu_selector, self.__cr_selector: Mutation and crossover selector functions, initialized if not already set.

• self.fes (int): Function evaluation counter, reset to zero.

• self.__archive (np.ndarray): Archive of solutions, initialized as empty.

• self.current_vector (np.ndarray): Current population of candidate solutions.

• self.current_fitness (np.ndarray): Fitness values of the current population.

• self.gbest_val (float): Best fitness value found in the current population.

• self.__gbest_index (int): Index of the best individual in the current population.

• self.__gbest_vector (np.ndarray): Best solution vector found in the current population.

• self.log_index (int): Logging index, initialized to 1.

• self.cost (list): List of best fitness values per generation.

• self.__init_gbest (float): Initial best fitness value.

• self.meta_X (list, optional): List of population vectors for meta-data logging (if enabled).

• self.meta_Cost (list, optional): List of population fitness values for meta-data logging (if enabled).

Returns:

• object: The result of self.observe(), typically an observation or summary of the initialized population state.

Raises:

• None explicitly, but may raise exceptions if problem is not properly defined or if array operations fail.

__update_archive(old_id)

Introduction

Updates the archive of solution vectors by either appending a new vector or replacing an existing one, depending on the archive’s current size.

Args:

• old_id (int): The index of the vector in current_vector to be added to the archive.

Built-in Attribute:

• self.__archive (np.ndarray): The archive of solution vectors.

• self.__NP (int): The maximum allowed size of the archive.

• self.current_vector (np.ndarray): The current population of solution vectors.

• self.__dim (int): The dimensionality of each solution vector.

• self.rng (np.random.Generator): Random number generator for selecting replacement indices.

Returns:

• None

Raises:

• None

update(action, problem)

Introduction

Updates the optimizer’s population based on the provided actions, applies mutation and crossover operators, evaluates new solutions, updates the archive, and tracks the best solution found so far.

Args:

• action (np.ndarray): An array representing the actions to be taken, including mutation and crossover operator indices and their parameters for each individual in the population.

• problem (object): The optimization problem instance, which should provide methods for evaluating solutions and contain problem-specific attributes such as bounds and optimum.

Returns:

• observation (np.ndarray): The current observation/state after the update.

• reward (float): The reward computed based on the improvement in the global best value.

• is_done (bool): A flag indicating whether the optimization process has reached its termination condition.

• info (dict): An empty dictionary reserved for additional information (for compatibility).

Raises:

• None explicitly, but may raise exceptions if the action array is malformed or if operator selection fails.

class src.environment.optimizer.rldeafl_optimizer.Basic_mutation(rng)

This class represents a basic mutation. Methods:

• get_parameters_numbers: Returns the number of parameters.

• mutation: Performs the mutation.

Initialization

mutation(group, cost, indexs, parameters, archive)

construct_random_indices(pop_size, indexs, x_num)

construct_extra_random_indices(pop_size, indexs, x_num, extra)

construct_pbest(group, p)

class src.environment.optimizer.rldeafl_optimizer.Basic_crossover(rng)

Initialization

crossover(x, v, parameters, group, cost, archive)

class src.environment.optimizer.rldeafl_optimizer.DE1(rng)

Bases: src.environment.optimizer.rldeafl_optimizer.Basic_mutation

mutation(group, cost, indexs, parameters, archive)

class src.environment.optimizer.rldeafl_optimizer.DE2(rng)

Bases: src.environment.optimizer.rldeafl_optimizer.Basic_mutation

mutation(group, cost, indexs, parameters, archive)

class src.environment.optimizer.rldeafl_optimizer.DE3(rng)

Bases: src.environment.optimizer.rldeafl_optimizer.Basic_mutation

mutation(group, cost, indexs, parameters, archive)

class src.environment.optimizer.rldeafl_optimizer.DE4(rng)

Bases: src.environment.optimizer.rldeafl_optimizer.Basic_mutation

mutation(group, cost, indexs, parameters, archive)

class src.environment.optimizer.rldeafl_optimizer.DE5(rng)

Bases: src.environment.optimizer.rldeafl_optimizer.Basic_mutation

mutation(group, cost, indexs, parameters, archive)

class src.environment.optimizer.rldeafl_optimizer.DE6(rng)

Bases: src.environment.optimizer.rldeafl_optimizer.Basic_mutation

mutation(group, cost, indexs, parameters, archive)

class src.environment.optimizer.rldeafl_optimizer.DE7(rng)

Bases: src.environment.optimizer.rldeafl_optimizer.Basic_mutation

mutation(group, cost, indexs, parameters, archive)

class src.environment.optimizer.rldeafl_optimizer.DE8(rng)

Bases: src.environment.optimizer.rldeafl_optimizer.Basic_mutation

mutation(group, cost, indexs, parameters, archive)

class src.environment.optimizer.rldeafl_optimizer.DE9(rng)

Bases: src.environment.optimizer.rldeafl_optimizer.Basic_mutation

EuclideanDistance(x, y)

cal_R_d(group)

construct_r(group, indexs)

mutation(group, cost, indexs, parameters, archive)

class src.environment.optimizer.rldeafl_optimizer.DE10(rng)

Bases: src.environment.optimizer.rldeafl_optimizer.Basic_mutation

Topograph(group, cost)

generate a kNN matrix indicating the nearest neighbors of each individual

mutation(group, cost, indexs, parameters, archive)

class src.environment.optimizer.rldeafl_optimizer.DE11(rng)

Bases: src.environment.optimizer.rldeafl_optimizer.Basic_mutation

mutation(group, cost, indexs, parameters, archive)

class src.environment.optimizer.rldeafl_optimizer.DE12(rng)

Bases: src.environment.optimizer.rldeafl_optimizer.Basic_mutation

mutation(group, cost, indexs, parameters, archive)

class src.environment.optimizer.rldeafl_optimizer.DE13(rng)

Bases: src.environment.optimizer.rldeafl_optimizer.Basic_mutation

mutation(group, cost, indexs, parameters, archive)

class src.environment.optimizer.rldeafl_optimizer.DE14(rng)

Bases: src.environment.optimizer.rldeafl_optimizer.Basic_mutation

mutation(group, cost, indexs, parameters, archive)

class src.environment.optimizer.rldeafl_optimizer.CR1(rng)

Bases: src.environment.optimizer.rldeafl_optimizer.Basic_crossover

Initialization

crossover(x, v, parameters, group, cost, archive)

class src.environment.optimizer.rldeafl_optimizer.CR2(rng)

Bases: src.environment.optimizer.rldeafl_optimizer.Basic_crossover

Initialization

crossover(x, v, parameters, group, cost, archive)

class src.environment.optimizer.rldeafl_optimizer.CR3(rng)

Bases: src.environment.optimizer.rldeafl_optimizer.Basic_crossover

Initialization

crossover(x, v, parameters, group, cost, archive)