src.environment.optimizer.psorlns_optimizer

Module Contents

Classes

PSORLNS_Optimizer

PSORLNS_Optimizer

An implementation of a learnable optimizer that combines Particle Swarm Optimization (PSO) with Reinforcement Learning-based Neighborhood Search (RLNS). This optimizer maintains a population of particles and updates their positions and velocities based on both PSO dynamics and adaptive neighborhood strategies, aiming to solve optimization problems efficiently.

API

class src.environment.optimizer.psorlns_optimizer.PSORLNS_Optimizer(config)[source]

Bases: src.environment.optimizer.learnable_optimizer.Learnable_Optimizer

PSORLNS_Optimizer

An implementation of a learnable optimizer that combines Particle Swarm Optimization (PSO) with Reinforcement Learning-based Neighborhood Search (RLNS). This optimizer maintains a population of particles and updates their positions and velocities based on both PSO dynamics and adaptive neighborhood strategies, aiming to solve optimization problems efficiently.

Initialization

Introduction

Initializes the PSORLNS optimizer with the provided configuration and sets default parameters for the optimization process.

Args:

  • config (object): Config object containing parameters for the optimizer.

    • The Attributes needed for the DEDDQN_Optimizer are the following:

      • n_logpoint (int): Number of log points for the optimizer.

      • full_meta_data (bool): Flag to indicate if full meta-data should be collected.

Attributes:

  • w (float): Inertia weight for the optimizer.Default is 1.

  • c1 (float): Cognitive coefficient.Default is 1.49445.

  • c2 (float): Social coefficient.Default is 1.49445.

  • ps (int): Population size.Default is 100.

  • TT2 (float): Threshold for selecting worse or better solutions.Default is 0.8.

  • neighbor_num (list of int): List of neighbor counts for local search.Default is [5, 10, 20, 30, 40].

  • fes (Any): Function evaluation counter (initialized as None).

  • cost (Any): Cost value (initialized as None).

  • pr (Any): Probability rate (initialized as None).

  • sr (Any): Success rate (initialized as None).

  • log_index (Any): Logging index (initialized as None).

  • log_interval (Any): Logging interval (initialized as None).

  • archive (Any): Archive for storing solutions (initialized as None).

  • archive_val (Any): Archive for storing solution values (initialized as None).

__str__()[source]

Introduction

Returns a string representation of the PSORLNS_Optimizer instance.

Returns:

  • str: The name of the optimizer, “PSORLNS_Optimizer”.

cal_pr_sr(problem)[source]

Introduction

Calculates the Peak Ratio (PR) and Success Rate (SR) at multiple accuracy levels for the current solution archive with respect to the given optimization problem.

Args:

  • problem: The optimization problem object, which provides the attributes nopt (number of global optima) and the method how_many_goptima for evaluating solutions.

Returns:

  • raw_PR (np.ndarray): Array of shape (5,) containing the peak ratio at each accuracy level.

  • raw_SR (np.ndarray): Array of shape (5,) containing the success rate at each accuracy level.

Notes:

  • The method uses five predefined accuracy thresholds: 1e-1, 1e-2, 1e-3, 1e-4, and 1e-5.

  • The solution archive (self.archive) and their corresponding values (self.archive_val) are used to determine which solutions meet each accuracy level.

  • The peak ratio is the fraction of global optima found at each accuracy level.

  • The success rate is set to 1 if all global optima are found at a given accuracy level, otherwise 0.

initialize_particles(problem)[source]

Introduction

Initializes the particles for the PSO (Particle Swarm Optimization) algorithm by generating random positions and velocities within the problem’s bounds, and computes initial costs and best values.

Args:

  • problem: The optimization problem object, which has attributes lb (lower bounds), ub (upper bounds), and be compatible with the get_costs method.

Returns:

  • None: Updates the self.particles attribute with initialized positions, velocities, costs, and best values.

Side Effects:

  • Sets self.particles to a dictionary containing:

    • ‘current_position’: Initial positions of all particles.

    • ‘c_cost’: Initial costs of all particles.

    • ‘pbest_position’: Initial personal best positions.

    • ‘pbest’: Initial personal best costs.

    • ‘gbest_position’: Initial global best position.

    • ‘gbest_val’: Initial global best cost.

    • ‘velocity’: Initial velocities of all particles.

    • ‘pop_dist’: Pairwise distance matrix of all particles.

init_population(problem)[source]

Introduction

Initializes the population of particles for the PSO-RLNS optimizer, sets up problem-specific parameters, and prepares logging and meta-data structures.

Args:

  • problem (object): The optimization problem, which has attributes such as maxfes, dim, ub, and lb.

Built-in Attributes:

  • self.max_fes: Maximum function evaluations allowed for the problem.

  • self.log_interval: Interval for logging progress.

  • self.dim: Dimensionality of the problem.

  • self.fes: Function evaluation counter. Default is 0.

  • self.max_velocity: Maximum velocity for particles, calculated based on the problem’s bounds.

  • self.w: Inertia weight for the optimizer, default is 1.

  • self.archive: Archive for storing solutions, initialized as an empty array.

  • self.archive_val: Archive for storing solution values, initialized as an empty array.

  • self.max_dist: Maximum distance between particles, calculated based on the problem’s bounds.

  • self.eps: Threshold for neighborhood size, default is 0.1.

  • self.log_index: Index for logging progress, initialized to 1.

  • self.cost: List to store the best cost values during optimization.

  • self.pr: List to store the peak ratio values during optimization.

  • self.sr: List to store the success rate values during optimization.

  • self.meta_X: List to store the positions of particles for meta-data collection (if enabled).

  • self.meta_Cost: List to store the costs of particles for meta-data collection (if enabled).

  • self.meta_Pr: List to store the peak ratio values for meta-data collection (if enabled).

  • self.meta_Sr: List to store the success rate values for meta-data collection (if enabled).

Returns:

  • np.ndarray: The initial observed state of the population, as returned by the observe() method.

Notes:

  • Resets or initializes several optimizer attributes, including function evaluation counters, velocity limits, and logging intervals.

  • Initializes the particle population and their associated costs and statistics.

  • Optionally stores meta-data if configured to do so.

get_costs(position, problem)[source]

Introduction

Calculates the cost values for a given set of positions by evaluating them on the provided problem and adjusting by the problem’s optimum value. Also updates the function evaluation count.

Args:

  • position (np.ndarray): An array of candidate solutions to be evaluated.

  • problem (object): An optimization problem instance with eval and optimum attributes.

Built-in Attribute:

  • self.fes (int): Counter for the number of function evaluations, incremented by the batch size.

Returns:

  • np.ndarray: The computed cost values for each position, adjusted by the problem’s optimum.

Raises:

  • AttributeError: If problem does not have eval or optimum attributes.

observe()[source]

Introduction

Computes a normalized state representation based on the neighborhood structure of particles in the optimizer.

Args:

None

Returns:

  • np.ndarray: A (ps, 1) array where each entry represents the normalized count of neighbors for each particle.

Raises:

None

cal_reward(current_cost, parent_cost)[source]

Introduction

Calculates the reward for each solution in the population based on the comparison between current and parent costs.

Args:

  • current_cost (np.ndarray): An array of current costs for each solution in the population.

  • parent_cost (np.ndarray): An array of parent costs for each solution in the population.

Returns:

  • np.ndarray: An array of rewards where each element is 1 if the current cost is less than the parent cost, -1 if greater, and 0 otherwise.

Notes:

  • The length of current_cost and parent_cost should match the population size (self.ps).

update(action, problem)[source]

Introduction

Updates the state of the PSO-RLNS optimizer for one iteration, applying particle swarm optimization and local neighborhood search strategies based on the provided actions and problem definition.

Args:

  • action (np.ndarray): An array of actions (typically integer indices) specifying the neighborhood size or strategy for each particle.

  • problem (object): An object representing the optimization problem, which must provide lower and upper bounds (lb, ub) and a cost evaluation method.

Returns:

  • next_state (np.ndarray): The observed state of the population after the update (shape: [ps, 9]).

  • reward (np.ndarray): The calculated reward for the current step, based on the improvement in cost.

  • is_end (np.ndarray): A boolean array indicating whether the termination condition has been met for each particle.

  • info (dict): Additional information (currently empty, reserved for future use).

Notes:

  • The method updates particle positions and velocities using a combination of PSO and RLNS-inspired rules.

  • It maintains and updates the best-known positions and costs for each particle and the global best.

  • Logging and meta-data collection are performed if enabled in the configuration.

  • The function assumes that the optimizer’s state (e.g., self.particles, self.fes, etc.) is managed externally and updated in-place.