how to set Empirical Bayes in Bandit Optimization

[cyrilmyself]

i want to implement MAB with AX；
i have read the Factorial design with empirical Bayes and Thompson Sampling;
If my experiment is not factorial，just a list of strategy； how to set the search_space

[Balandat]

You should be able to follow the tutorial https://ax.dev/tutorials/factorial but just use a single ChoiceParameter instead (where the values are the list of the choices from your strategy). You can view this is a "degenerate" factorial setting with a single factor (of course the synthetic data generation in the FactorialMetric wouldn't work the same way, but if you're actually getting your observations from some real problem that's not an issue)..

[cyrilmyself]

@Balandat thank you for your replying;
below code is to get the result of every arm；i do not see how to use empirical Bayes and Thompson Sampling to process the result，is it encapsulated？

class FactorialMetric(Metric):
def fetch_trial_data(self, trial):
records = []
for arm_name, arm in trial.arms_by_name.items():
params = arm.parameters
batch_size = 10000
noise_level = 0.0
weight = trial.normalized_arm_weights().get(arm, 1.0)
coefficients = np.array([0.1, 0.2, 0.3, 0.1, 0.2, 0.1, 0.2, 0.3, 0.4])
features = np.array(list(params.values())).reshape(1, -1)
encoded_features = one_hot_encoder.fit_transform(features)
z = (
coefficients @ encoded_features.T
+ np.sqrt(noise_level) * np.random.randn()
)
p = np.exp(z) / (1 + np.exp(z))
plays = np.random.binomial(batch_size, weight)
successes = np.random.binomial(plays, p)
records.append(
{
"arm_name": arm_name,
"metric_name": self.name,
"trial_index": trial.index,
"mean": float(successes) / plays,
"sem": agresti_coull_sem(successes, plays),
}
)
return Ok(value=Data(df=pd.DataFrame.from_records(records)))

[Balandat]

I don't understand - this is just the synthetic data generating process from the tutorial (which wouldn't apply in your setting with a non-factorial design - you'd either just write a metric to return the results from the actual problem you wan to solve, or you'd have to switch out the synthetic data generating process for something else if you just want to use it for testing). The actual EB/TS on top of that happens in sections 3+ of the tutorial.

[cyrilmyself]

@Balandat thank you for your replying and sorry for that i do not express my meaning；
what i meaning is that i did not find the EB processing in the tutorial;
The metric of every arm is processed by EB，then transferred to next for TS；I can find TS step， but did not find EB process