src.environment.optimizer.qlpso_optimizer

Module Contents

Classes

QLPSO_Optimizer

Introduction

QLPSO is a problem-free PSO which integrates a reinforcement learning method.

Functions

cal_reward

clipping

API

src.environment.optimizer.qlpso_optimizer.cal_reward(f_old, f_new, d_old, d_new)[source]
class src.environment.optimizer.qlpso_optimizer.QLPSO_Optimizer(config)[source]

Bases: src.environment.optimizer.learnable_optimizer.Learnable_Optimizer

Introduction

QLPSO is a problem-free PSO which integrates a reinforcement learning method.

Original paper

“A reinforcement learning-based communication topology in particle swarm optimization.” Neural Computing and Applications (2020).

Initialization

Introduction

Initializes the QLPSO optimizer with the provided configuration, setting up hyperparameters and internal state variables required for optimization.

Args:

  • config (object): Config object containing optimizer parameters.

    • The Attributes needed for QLPSO are the following:

      • maxFEs: Maximum function evaluations allowed.

      • n_logpoint: Number of log points for cost history.

      • log_interval: Interval for logging progress

      • full_meta_data: Flag to enable/disable full metadata logging

Built-in Attributes:

  • self.__config: Configuration object containing optimizer parameters.

  • self.__C: Cognitive coefficient for PSO. Default is 1.49618.

  • self.__W: Inertia weight for PSO. Default is 0.729844.

  • self.__NP: Population size for PSO. Default is 30.

  • self.solution_pointer: Pointer to the current solution in the population.

  • self.__population: Current population of solutions.

  • self.__pbest: Personal best positions of the population.

  • self.__velocity: Current velocities of the population.

  • self.__cost: Current costs of the population.

  • self.__gbest_cost: Global best cost found so far.

  • self.__diversity: Diversity of the population.

  • self.__state: State of each individual in the population.

  • self.fes: Function evaluation count.

  • self.cost: List of costs that need to be maintained by every backbone optimizer.

  • self.log_index: Logging index for tracking progress.

__cal_diversity()[source]

Introduction

Calculates the diversity of the current population in the optimizer. The diversity is computed as the mean Euclidean distance of each individual in the population from the population mean.

Returns:

  • float: The average Euclidean distance representing the diversity of the population.

__cal_velocity(action)[source]

Introduction

Calculates the updated velocity vector for a particle in the QLPSO (Quantum-behaved Learning Particle Swarm Optimization) algorithm based on the selected neighborhood size determined by the action parameter.

Args:

  • action (int): Determines the neighborhood size for local best selection. Possible values are:

    • 0: Neighborhood size 4

    • 1: Neighborhood size 8

    • 2: Neighborhood size 16

    • 3: Neighborhood size 30

Returns:

  • np.ndarray: The updated velocity vector for the current particle.

Notes:

  • The method uses the current particle’s position, velocity, personal best, and the best position found in the selected neighborhood to compute the new velocity.

  • The random number generator (self.rng.rand()) is used for stochastic updates.

init_population(problem)[source]

Introduction

Initializes the population and related attributes for the QLPSO optimizer based on the provided optimization problem.

Args:

  • problem: An object representing the optimization problem, which must have attributes ub (upper bounds), lb (lower bounds), optimum (optional known optimum), and an eval method for evaluating solutions.

Built-in Attributes:

  • self.__dim: Dimension of the problem.

  • self.__population: Current population of solutions.

  • self.__pbest: Personal best positions of the population.

  • self.__velocity: Current velocities of the population.

  • self.__cost: Current costs of the population.

  • self.__gbest_cost: Global best cost found so far.

  • self.__diversity: Diversity of the population.

  • self.__state: State of each individual in the population.

  • self.fes: Function evaluation count.

  • self.cost: List of costs that need to be maintained by every backbone optimizer.

  • self.log_index: Logging index for tracking progress.Default is 1.

  • self.cost: List of costs that need to be maintained by every backbone optimizer.

  • self.__state: State of each individual in the population.

  • self.meta_X: List to store population positions for metadata logging.

  • self.meta_Cost: List to store population costs for metadata logging.

  • self.meta_tmp_x: Temporary list to store population positions for metadata logging.

  • self.meta_tmp_cost: Temporary list to store population costs for metadata logging.

Returns:

  • int: The state value of the solution pointer after initialization.

Notes:

  • If the problem’s optimum is provided, the cost is offset by this value.

  • If full_meta_data is enabled in the configuration, additional metadata is stored for analysis.

update(action, problem)[source]

Introduction

Updates the state of the optimizer by applying the given action to the current solution, evaluating the new solution, updating rewards, and managing logging and metadata.

Args:

  • action (Any): The action to be applied to the current solution, typically representing a velocity or direction in the search space.

  • problem (object): The optimization problem instance, which must provide lb, ub, eval(), and optimum attributes/methods.

Returns:

  • state (Any): The updated state after applying the action.

  • reward (float): The calculated reward based on the change in cost and diversity.

  • is_done (bool): Whether the optimization episode should be terminated.

  • info (dict): Additional information (currently empty).

Notes:

  • The method updates internal population, velocity, cost, diversity, and logging information.

  • Handles boundary control and personal/global best updates.

  • Supports optional metadata logging if configured.

  • Ensures the cost log is filled up to the required number of log points at the end of an episode.

src.environment.optimizer.qlpso_optimizer.clipping(x: Union[numpy.ndarray, Iterable], lb: Union[numpy.ndarray, Iterable, int, float, None], ub: Union[numpy.ndarray, Iterable, int, float, None]) → numpy.ndarray[source]