It's great to be here.
My background is in quant trading and social media, but if you do any kind of engineering, you'll like find that you need to run experiments. Experiments are everywhere: software, hardware, medicine, food. Everything around us that works well does so because of long-term, compounding progress, which is made possible by experimentation.
An A/B
test is the simplest experiment. It compares two ways of doing something -- like an eye doctor, "Like this or like this?" You pick the better one and discard the worse one.
Bayesian optimization is the current state of the art of experimentation. It's efficient and flexible. A Bayesian optimization can not only compare A to B, it can compare "A to B to C to ...", tune numerical parameters, and mix-and-match multiple comparisons. And it'll do all of that while trying to minimize the number of experimental measurements you have to make.
I'm looking forward to reading about your experiences with experimentation and answering your questions.
Dave