[Bug]: Sparsity threshold increase doesn't change solution sparsity

[souravdey94]

Question

I am performing SEBO in 30 dimensions. My reference vector is a 30-d zero vector. I have one other objective besides sparsity. I have two variants of the workflow. In one, a sum constraint sum xi = 1 is used and in the other no such constraint is used. Ideally, relaxing the sparsity threshold should increase the number of non-zero parameters. But I don't see that. Any insight as to why it's happening would be helpful?

sparsity-thresh_benchmark.xlsx

Please provide any relevant code snippet if applicable.

class GenerationStrategyBuilder:
    """
    A class to build generation strategies for Ax optimization experiments.
    """
    
    def __init__(self):
        pass
    
    def create_strategy(self, target_point, length, surrogate_class=SaasFullyBayesianSingleTaskGP):
        """
        Create a generation strategy for SEBO optimization.
        
        Args:
            target_point_minus1: Target point tensor for optimization
            length: Length parameter (currently unused but kept for compatibility)
            surrogate_class: The surrogate model class to use (default: SaasFullyBayesianSingleTaskGP)
            
        Returns:
            GenerationStrategy: Configured generation strategy
        """
        gs = GenerationStrategy(
            steps=[
                # GenerationStep(
                #     model=Models.SOBOL,
                #     num_trials=15,  # https://github.com/facebook/Ax/issues/922
                # # #     min_trials_observed=3,
                # # #     # max_parallelism=20,
                # # #     model_kwargs={"seed": 999},
                # # #     model_gen_kwargs={},
                # ),
                GenerationStep(
                    model=Models.BOTORCH_MODULAR,
                    num_trials=-1,  # No limitation on how many trials should be produced from this step
                    #max_parallelism=5,  # Parallelism limit for this step, often lower than for Sobol
                    # More on parallelism vs. required samples in BayesOpt:
                    # https://ax.dev/docs/bayesopt.html#tradeoff-between-parallelism-and-total-number-of-trials
                    should_deduplicate=True,
                    model_kwargs={  # Kwargs to pass to `BoTorchModel.__init__`
                        "surrogate": Surrogate(botorch_model_class=surrogate_class),
                        "acquisition_class": SEBOAcquisition,
                        # "botorch_acqf_class": qNoisyExpectedHypervolumeImprovement,
                        "botorch_acqf_class": qLogNoisyExpectedHypervolumeImprovement,
                        "acquisition_options": {
                            "penalty": "L0_norm", # it can be L0_norm or L1_norm.
                            "target_point": target_point, 
                            "sparsity_threshold": 5, #Also 10, 15, 20 respectively
                        },
                    },     
                )
            ]
        )
        
        return gs

Code of Conduct

• I agree to follow this Ax's Code of Conduct