src.environment.optimizer.madac_optimizer¶
Module Contents¶
Classes¶
Functions¶
Introduction¶Safely extends or appends items to a given list, handling various input types such as lists, NumPy arrays, or single elements. |
|
Introduction¶Normalizes the objectives of each solution within specified minimum and maximum bounds. If bounds are not provided, they are computed from the input solutions. |
|
Introduction¶Calculates the Chebyshev (Tchebycheff) fitness value for a solution with multiple minimized objectives. |
Data¶
API¶
- src.environment.optimizer.madac_optimizer.safe_extend(lst, items)[source]¶
Introduction¶
Safely extends or appends items to a given list, handling various input types such as lists, NumPy arrays, or single elements.
Args:¶
lst (list): The list to which items will be added.
items (Any): The items to add. Can be None, a list, a NumPy ndarray, or a single element.
Returns:¶
None
Raises:¶
None
- class src.environment.optimizer.madac_optimizer.MADAC_Optimizer(config)[source]¶
Bases:
src.environment.optimizer.learnable_optimizer.Learnable_Optimizer- init_population(problem)[source]¶
Introduction¶
Initializes the population and related attributes for the multi-objective optimization process. This method sets up the initial population, objective values, reference points, evaluation budget, and various indicators required for the optimizer. It also initializes adaptive weights, static parameters, and operator objects, and updates internal metadata and state information.
Args:¶
problem (object): An instance representing the optimization problem, which must provide attributes such as
n_obj(number of objectives),dim(number of variables),lb(lower bounds),ub(upper bounds), and methods likeeval()andget_ref_set().
Built-in Attribute:¶
self.problem: Stores the optimization problem instance.
self.n_obj: Number of objectives.
self.n_var: Number of decision variables.
self.weights: Weight vectors for decomposition.
self.neighborhoods: Neighborhood structure for the population.
self.population_size: Number of individuals in the population.
self.population: Initial population of solutions.
self.population_obj: Objective values of the initial population.
self.done: Flag indicating if the optimization is complete.Default is False.
self.fes: Number of function evaluations used.
self.moead_generation: Current generation counter.Default is 0.
self.episode_limit: Maximum number of generations/episodes.
self.archive_maximum: Maximum objective values in the population.
self.archive_minimum: Minimum objective values in the population.
self.ideal_point: Current ideal point in the objective space.
self.problem_ref_points: Reference points for the problem.
self.operators: Operators object for evolutionary operations.
self.initial_igd: Initial Inverted Generational Distance indicator.
self.last_igd: Last computed IGD value.
self.best_igd: Best IGD value found so far.
self.initial_hv: Initial Hypervolume indicator.
self.last_hv: Last computed HV value.
self.best_hv: Best HV value found so far.
self.metadata: Dictionary for storing metadata such as solutions and costs.
Returns:¶
object: The current state of the optimizer after initialization, as returned by
self.get_state().
Raises:¶
None directly, but may raise exceptions if the problem instance does not provide required attributes or methods, or if array operations fail due to shape mismatches.
- init_adaptive_weights()[source]¶
Introduction¶
Initializes the adaptive weights and related parameters for the optimizer. This includes setting up elite populations, update rates, and adaptive intervals used during the optimization process.
Built-in Attribute:¶
self.EP (list): Stores the elite population.Default is an empty list.
self.EP_obj (list): Stores the objective values of the elite population.Default is an empty list.
self.EP_MaxSize (int): Maximum size of the elite population, set to 1.5 times the population size.
self.rate_update_weight (float): The rate at which weights are updated.Default is 0.05.
self.nus (int): Maximum number of subproblems to be adjusted, calculated from the update rate and population size.
self.wag (int): Adaptive iteration interval, based on a percentage of the episode limit.
self.adaptive_cooling_time (int): Cooling time for adaptive updates, set to the adaptive interval.
self.adaptive_end (int): The function evaluation step at which adaptation ends, set to 90% of the maximum function evaluations.
Returns:¶
None
- init_static()[source]¶
Introduction¶
Initializes static attributes for tracking optimization progress, statistics, and historical metrics within the optimizer.
Built-in Attribute:¶
last_bonus (float): Stores the last computed bonus value. Default is 0.
stag_count (int): Counts the number of consecutive iterations without improvement. Default is 0.
stag_count_max (float): Maximum allowed stagnation count, set as one-tenth of the episode limit.Default is
self.episode_limit / 10.hv_his (list): History of hypervolume metric values.Default is an empty list.
hv_last5 (tianshou.utils.MovAvg): Moving average of the last 5 hypervolume values.
nds_ratio_his (list): History of non-dominated solution ratio values.Default is an empty list.
nds_ratio_last5 (tianshou.utils.MovAvg): Moving average of the last 5 non-dominated solution ratio values.
ava_dist_his (list): History of average distance metric values.Default is an empty list.
ava_dist_last5 (tianshou.utils.MovAvg): Moving average of the last 5 average distance values.
hv_running (tianshou.utils.RunningMeanStd): Running mean and standard deviation for hypervolume.
nds_ratio_running (tianshou.utils.RunningMeanStd): Running mean and standard deviation for non-dominated solution ratio.
ava_dist_running (tianshou.utils.RunningMeanStd): Running mean and standard deviation for average distance.
reward_his (list): History of reward values.Default is an empty list.
obs_his (list): History of observation values.Default is an empty list.
igd_his (list): History of Inverted Generational Distance (IGD) metric values.Default is an empty list.
Returns:¶
None
- get_neighborhoods()[source]¶
Introduction¶
Computes the neighborhoods for each weight vector in the optimizer by finding the closest weight vectors.
Returns:¶
List[List[int]]: A list where each element contains the indices of the nearest neighbors for the corresponding weight vector.
Details:¶
For each weight vector in
self.weights, this method sorts all other weight vectors based on their proximity (usingself.moead_sort_weights) and selects the closest ones up toself.moead_neighborhood_maxsize. The result is a list of neighborhoods, where each neighborhood is represented by a list of indices.
- get_weights(n_obj)[source]¶
Introduction¶
Generates a set of weight vectors based on the number of objectives (
n_obj) for use in multi-objective optimization.Args:¶
n_obj (int): The number of objectives for which to generate weight vectors.
Returns:¶
np.ndarray: An array of weight vectors suitable for the specified number of objectives.
Notes:¶
For specific values of
n_obj(2, 3, 5, 7, 8, 10), predefined boundary weights are generated usingnormal_boundary_weightswith tailored parameters.For other values, random weights are generated using
random_weightsand the optimizer’s population size.
- get_igd()[source]¶
Introduction¶
Calculates the Inverted Generational Distance (IGD) for the current population and updates the IGD history.
Args:¶
None
Built-in Attribute:¶
self.problem_ref_points: Reference set of points for IGD calculation.
self.population_obj: Objective values of the current population.
self.igd_his: List storing the history of IGD values.
Returns:¶
float: The calculated IGD value for the current population.
Raises:¶
Any exception raised by the
InvertedGenerationalDistanceor itscalculatemethod.
- get_hv(n_samples=100000.0)[source]¶
Introduction¶
Computes the hypervolume (HV) indicator for the current population of solutions in a multi-objective optimization problem. The hypervolume measures the volume in the objective space dominated by the population and bounded by a reference point. For problems with 3 or fewer objectives, the exact HV is calculated. For problems with more than 3 objectives, the HV is estimated using a Monte Carlo sampling approach.
Args:¶
n_samples (int, optional): Number of Monte Carlo samples to use for HV estimation when the number of objectives is greater than 3. Defaults to 1e5.
Returns:¶
float: The computed or estimated hypervolume value for the current population.
Raises:¶
AssertionError: If the normalized population objective values are not within the expected bounds during Monte Carlo estimation.
- get_reward(value)[source]¶
Introduction¶
Calculates the reward based on the provided value (typically IGD), the reward type, and historical performance metrics.
Args:¶
value (float): The current value (e.g., IGD) to evaluate for reward calculation.
Returns:¶
float: The computed reward according to the selected reward type and historical values.
Raises:¶
ValueError: If an invalid reward type is specified in
self.reward_type.
- get_state()[source]¶
Introduction¶
Constructs and returns the current state observation for all agents in the optimizer environment. The state is represented as a 22-dimensional feature vector containing normalized problem parameters, progress indicators, and various statistical metrics related to the optimization process.
Returns:¶
list of numpy.ndarray: A list containing the same 22-dimensional observation vector for each agent (
self.n_agents), where each vector encodes the current environment state.
Observation Vector Details:¶
obs_[0]: Inverse of the number of objectives.
obs_[1]: Inverse of the number of variables.
obs_[2]: Normalized current generation count.
obs_[3]: Normalized stagnation count.
obs_[4]: Current hypervolume metric.
obs_[5]: Ratio of non-dominated solutions.
obs_[6]: Average distance metric.
obs_[7]: Recent change in hypervolume.
obs_[8]: Recent change in non-dominated solution ratio.
obs_[9]: Recent change in average distance.
obs_[10]: Mean of the last 5 hypervolume values.
obs_[11]: Mean of the last 5 non-dominated solution ratios.
obs_[12]: Mean of the last 5 average distances.
obs_[13]: Standard deviation of the last 5 hypervolume values.
obs_[14]: Standard deviation of the last 5 non-dominated solution ratios.
obs_[15]: Standard deviation of the last 5 average distances.
obs_[16]: Running mean of hypervolume.
obs_[17]: Running mean of non-dominated solution ratio.
obs_[18]: Running mean of average distance.
obs_[19]: Running variance of hypervolume.
obs_[20]: Running variance of non-dominated solution ratio.
obs_[21]: Running variance of average distance.
- get_action(action_idx, action)[source]¶
Introduction¶
Maps an action index and action value to a specific agent parameter value based on predefined lists.
Args:¶
action_idx (int): The index indicating which agent parameter to select.
0: Neighbor size
1: Operator strategy
2: Probability constant
3: Weight
action (int): The index within the selected parameter list to retrieve the value.
Returns:¶
int | float | str: The value from the corresponding agent parameter list based on the provided indices.
Raises:¶
IndexError: If
actionis out of range for the selected parameter list.
- update(action, problem)[source]¶
Introduction¶
Performs a single update step in the MOEA/D (Multi-Objective Evolutionary Algorithm based on Decomposition) optimization process. This includes solution generation, selection, and adaptive weight adjustment.
Args:¶
action (list): A list of parameters controlling the update step, including neighborhood size, operator type, operator parameter, and weight adjustment flag.
problem (object): The optimization problem instance, providing evaluation and problem-specific methods.
Returns:¶
obs (object): The updated state observation.
rewards (list): A list of reward values for each agent.
done (bool): Whether the optimization process has reached its stopping condition.
info (dict): Additional information, including best and last IGD (Inverted Generational Distance) values.
Raises:¶
Exception: If the weight adjustment flag in
action[3]is greater than 1.
- update_information()[source]¶
Introduction¶
Updates the optimizer’s internal information by identifying the non-dominated solutions (Pareto front) in the current population and storing relevant metadata.
Args:¶
None
Built-in Attribute:¶
self.population_obj (list): Objective values of the current population.
self.population (list): Current population of solutions.
self.metadata (dict): Dictionary to store historical populations and their objective values.
self.cost (list): Stores the objective values of the Pareto front.
Returns:¶
None
Raises:¶
None
- update_ep()[source]¶
Introduction¶
Updates the current evolutionary population (EP) by incorporating offspring, filtering non-dominated solutions, and maintaining the population size using a crowding distance-based selection.
Args:¶
None
Returns:¶
None
Details:¶
Extends the current population (
EPandEP_obj) with offspring solutions.Filters the population to retain only non-dominated solutions.
If the population exceeds the maximum allowed size (
EP_MaxSize), removes overcrowded solutions based on a crowding distance metric to maintain diversity.
- update_weight()[source]¶
Introduction¶
Updates the weights and population of subproblems in the optimizer by removing overcrowded subproblems and introducing new ones from the external population (EP). Also updates the neighborhoods for each subproblem based on the new weights.
Args:¶
None
Built-in Attribute:¶
self.population (list): The current population of solutions.
self.population_obj (list): The objective values of the current population.
self.weights (list): The weight vectors associated with each subproblem.
self.EP (list): The external population containing elite solutions.
self.EP_obj (list): The objective values of the external population.
self.ideal_point (list or np.ndarray): The ideal point in the objective space.
self.nus (int): The number of new subproblems to introduce.
self.n_obj (int): The number of objectives.
self.population_size (int): The size of the population.
self.moead_sort_weights (callable): Function to sort weights for neighborhood calculation.
self.moead_neighborhood_maxsize (int): Maximum size of the neighborhood.
self.neighborhoods (list): The neighborhoods for each subproblem.
Returns:¶
None
Raises:¶
None
- update_igd(value)[source]¶
Introduction¶
Updates the Inverted Generational Distance (IGD) value and tracks stagnation in optimization.
Args:¶
value (float): The new IGD value to be evaluated and potentially set as the best IGD.
Built-in Attribute:¶
self.best_igd (float): The current best IGD value.
self.stag_count (int): The number of consecutive iterations without improvement.
self.last_igd (float): The most recent IGD value.
Returns:¶
None
Raises:¶
None
- moead_update_ideal(solution_obj)[source]¶
Introduction¶
Updates the ideal point in the MOEA/D (Multi-Objective Evolutionary Algorithm based on Decomposition) optimizer using the objective values of a given solution.
Args:¶
solution_obj (np.ndarray): A 1D array containing the objective values of the current solution.
Built-in Attribute:¶
self.ideal_point (np.ndarray): The current ideal point vector, which is updated in-place.
Returns:¶
None
Raises:¶
None
- moead_update_solution(solution, solution_obj, mating_indices)[source]¶
Introduction¶
Updates the MOEA/D population by potentially replacing individuals in the mating neighborhood with a given solution if it improves the scalarized fitness value. Ensures that the number of replacements does not exceed a predefined threshold.
Args:¶
solution (Any): The candidate solution to potentially insert into the population.
solution_obj (Any): The objective values associated with the candidate solution.
mating_indices (List[int]): Indices of the population members considered for replacement.
Returns:¶
None
Notes:¶
The method shuffles the mating indices, evaluates each candidate in the neighborhood, and replaces it with the new solution if the new solution has a better scalarized fitness value according to the corresponding weight vector.
The number of replacements is limited by
self.moead_eta.
- static moead_sort_weights(base, weights)[source]¶
Introduction¶
Sorts a list of weight vectors by their Euclidean distance to a given base weight vector, returning the indices of the weights in ascending order of distance.
Args:¶
base (list[float]): The reference weight vector to which distances are computed.
weights (list[list[float]]): A list of weight vectors to be sorted by proximity to the base.
Returns:¶
list[int]: A list of indices representing the order of weights sorted by increasing distance to the base vector.
Notes:¶
The function uses Euclidean distance as the metric for sorting.
- moead_get_subproblems()[source]¶
Introduction¶
Determines the order of subproblems to be searched in the MOEA/D optimization process. If utility-based updating is enabled, the method follows the utility-based MOEA/D search; otherwise, it uses the original MOEA/D specification.
Returns:¶
List[int]: A shuffled list of indices representing the subproblems to be searched in the current iteration.
- moead_get_mating_indices(index)[source]¶
Introduction¶
Determines the mating indices for the MOEA/D algorithm based on a probabilistic selection between the neighborhood and the entire population.
Args:¶
index (int): The index of the current individual in the population for which mating indices are to be determined.
Returns:¶
list[int]: A list of indices representing the selected mating pool, either from the individual’s neighborhood or the entire population.
Notes:¶
With probability
moead_delta, the method returns the indices of the individual’s neighborhood up tomoead_neighborhood_size.Otherwise, it returns the indices of the entire population.
- find_non_dominated_indices(population_list)[source]¶
Introduction¶
Identifies the indices of non-dominated solutions (Pareto optimal solutions) in a given population for multi-objective optimization.
Args:¶
population_list (List[List[float]]): A list where each element is a list representing the objective values of a solution in the population.
Returns:¶
numpy.ndarray: An array of indices corresponding to the non-dominated solutions in the population.
Raises:¶
None
- get_ratio_nondom_sol()[source]¶
Introduction¶
Calculates the ratio of non-dominated solutions in the current population and updates historical tracking metrics.
Returns:¶
float: The ratio of non-dominated solutions to the total number of solutions in the population.
Side Effects:¶
Appends the computed ratio to
self.nds_ratio_his.Adds the ratio to
self.nds_ratio_last5.Updates the running average in
self.nds_ratio_runningwith the new ratio value.
- get_average_dist()[source]¶
Introduction¶
Calculates the normalized average pairwise distance between individuals in the population based on their objective values.
Args:¶
None
Returns:¶
float: The normalized average distance between all pairs of individuals in the population.
Raises:¶
SystemExit: If the computed average distance is NaN, prints diagnostic information and exits the program.
- get_pre_k_change(k, value_his)[source]¶
Introduction¶
Calculates the change in value over the last
kgenerations from a history of values.Args:¶
k (int): The number of generations to look back.
value_his (list[float]): A list containing the historical values.
Returns:¶
float: The difference between the most recent value and the value
kgenerations ago. Returns 0 if there are not enough generations.
Raises:¶
IndexError: If
value_hisdoes not contain enough elements for the calculation whenself.moead_generation >= k.
- normal_boundary_weights(nobjs, divisions_outer, divisions_inner=0)[source]¶
Introduction¶
Generates a set of uniformly distributed weight vectors on the simplex using the normal boundary intersection method. These weights are commonly used in multi-objective optimization to sample the objective space.
Args:¶
nobjs (int): The number of objectives (dimensions) for the weight vectors.
divisions_outer (int): The number of divisions for the outer set of weights, controlling the granularity of the boundary weights.
divisions_inner (int, optional): The number of divisions for the inner set of weights, used to generate additional weights inside the simplex. Defaults to 0 (no inner weights).
Returns:¶
list[list[float]]: A list of weight vectors, where each vector is a list of floats summing to 1, representing points on the simplex.
- random_weights(nobjs, population_size)[source]¶
Introduction¶
Generates a set of randomly-generated but uniformly distributed weight vectors for multi-objective optimization. This method ensures that the generated weights are spread out as uniformly as possible in the objective space, which is important for algorithms that require diverse solutions. For two objectives, the weights are generated analytically; for more objectives, a set of candidate weights is generated and the most distant ones are iteratively selected to maximize diversity.
Args:¶
nobjs (int): The number of objectives (dimensions) for the weights.
population_size (int): The number of weight vectors to generate.
Returns:¶
list[list[float]]: A list of weight vectors, each of length
nobjs, where each vector sums to 1.
Raises:¶
None explicitly, but may raise exceptions if invalid arguments are provided (e.g., population_size < nobjs).
- class src.environment.optimizer.madac_optimizer.Hypervolume(reference_set=None, minimum=None, maximum=None)[source]¶
Bases:
src.environment.optimizer.madac_optimizer.IndicatorInitialization
Initializes the Hypervolume object with either a reference set or explicit minimum and maximum values.
Args:¶
reference_set (optional): A set of reference points used for normalization. If provided,
minimumandmaximummust not be specified.minimum (optional): The minimum values for normalization. Must be specified if
reference_setis not provided.maximum (optional): The maximum values for normalization. Must be specified if
reference_setis not provided.
Raises:¶
ValueError: If both
reference_setand eitherminimumormaximumare specified.ValueError: If neither
reference_setnor bothminimumandmaximumare specified.
- invert(solution_normalized_obj: numpy.ndarray)[source]¶
Introduction¶
Inverts the normalized objective values for each objective in the solution array. Each value is clipped to the [0.0, 1.0] range before inversion.
Args:¶
solution_normalized_obj (np.ndarray): A 2D NumPy array where each row represents a solution and each column represents a normalized objective value (expected in [0.0, 1.0]).
Returns:¶
np.ndarray: The input array with each objective value inverted (i.e.,
1.0 - value), maintaining the original shape.
Raises:¶
None
- dominates(solution1_obj, solution2_obj, nobjs)[source]¶
Introduction¶
Determines whether the first solution dominates the second solution in a multi-objective optimization context.
Args:¶
solution1_obj (list or array-like): Objective values of the first solution.
solution2_obj (list or array-like): Objective values of the second solution.
nobjs (int): Number of objectives to consider in the comparison.
Returns:¶
bool: True if
solution1_objdominatessolution2_obj, False otherwise.
Notes:¶
A solution is said to dominate another if it is strictly better in at least one objective and not worse in any other objective.
- filter_nondominated(solutions_obj, nsols, nobjs)[source]¶
Introduction¶
Filters out dominated solutions from a set of candidate solutions based on their objective values, leaving only the non-dominated solutions (i.e., the Pareto front).
Args:¶
solutions_obj (list or array-like): A collection of solution objective vectors to be filtered.
nsols (int): The number of solutions in
solutions_obj.nobjs (int): The number of objectives for each solution.
Returns:¶
int: The number of non-dominated solutions remaining after filtering.
Raises:¶
None explicitly. Assumes that
solutions_obj,nsols, andnobjsare valid and consistent.
- reduce_set(solutions, nsols, obj, threshold)[source]¶
Introduction¶
Filters a set of solutions by removing those whose objective value does not exceed a given threshold. The removal is performed in-place by swapping filtered-out solutions to the end of the array.
Args:¶
solutions (np.ndarray): A 2D array of candidate solutions, where each row represents a solution and columns represent features or objectives.
nsols (int): The current number of valid solutions in the array.
obj (int): The index of the objective column to be compared against the threshold.
threshold (float): The minimum acceptable value for the specified objective.
Returns:¶
int: The updated number of valid solutions remaining after filtering.
Notes:¶
The function assumes that the
swapmethod is defined and correctly swaps two rows in thesolutionsarray.Solutions with objective values less than or equal to the threshold are removed from the valid set.
- calc_internal(solutions_obj: numpy.ndarray, nsols, nobjs)[source]¶
Introduction¶
Recursively calculates the hypervolume (or a similar metric) for a set of multi-objective solutions using a divide-and-conquer approach.
Args:¶
solutions_obj (np.ndarray): A 2D array of shape (nsols, nobjs) representing the objective values of the solutions.
nsols (int): The number of solutions to consider from
solutions_obj.nobjs (int): The number of objectives (dimensions) for the calculation.
Returns:¶
float: The computed hypervolume (or related metric) for the given set of solutions.
Raises:¶
Any exceptions raised by the called methods (
filter_nondominated,calc_internal,surface_unchanged_to,reduce_set) if the input is invalid or computation fails.
- calculate(solutions_obj: numpy.ndarray)[source]¶
Introduction¶
Calculates a metric (such as hypervolume) for a set of solution objective values, considering only feasible solutions within specified bounds.
Args:¶
solutions_obj (np.ndarray): A 2D numpy array of shape (n_solutions, n_objectives) representing the objective values of candidate solutions.
Returns:¶
float: The calculated metric value for the feasible solutions. Returns 0.0 if no feasible solutions are found.
Raises:¶
None
- class src.environment.optimizer.madac_optimizer.InvertedGenerationalDistance(reference_set)[source]¶
Bases:
src.environment.optimizer.madac_optimizer.IndicatorInitialization
- src.environment.optimizer.madac_optimizer.normalize(solutions_obj: numpy.ndarray, minimum: numpy.ndarray = None, maximum: numpy.ndarray = None) numpy.ndarray[source]¶
Introduction¶
Normalizes the objectives of each solution within specified minimum and maximum bounds. If bounds are not provided, they are computed from the input solutions.
Args:¶
solutions_obj (np.ndarray): A 2D numpy array representing the objectives of the solutions to be normalized.
minimum (np.ndarray, optional): The minimum values for each objective used for normalization. If not provided, computed from
solutions_obj.maximum (np.ndarray, optional): The maximum values for each objective used for normalization. If not provided, computed from
solutions_obj.
Returns:¶
np.ndarray: The normalized solutions as a 2D numpy array.
Raises:¶
ValueError: If any objective has an empty range (i.e.,
maximum - minimum < EPSILON).
- src.environment.optimizer.madac_optimizer.chebyshev(solution_obj, ideal_point, weights, min_weight=0.0001)[source]¶
Introduction¶
Calculates the Chebyshev (Tchebycheff) fitness value for a solution with multiple minimized objectives.
Args:¶
solution_obj (np.ndarray or list of float): The objective values of the solution.
ideal_point (list of float): The ideal (reference) point for the objectives.
weights (list of float): The weights assigned to each objective.
min_weight (float, optional): The minimum weight allowed for any objective (default is 0.0001).
Returns:¶
float: The Chebyshev fitness value, representing the maximum weighted deviation from the ideal point.
Raises:¶
IndexError: If the lengths of
solution_obj,ideal_point, orweightsdo not match.