Doubtnut

We wanted to drive a lot more control into our A/B testing process.

Basic requirements were:

1. Ability to create multiple experiments and multiple variants for an experiment.
2. Should be able to distribute requests as configured (by percentage) between different variants of an experiments.
3. Should do so in a deterministic way. That is, until the rollout of an experiment is changed, a user should always get the same variant of an experiment.
4. Overrides based on user properties.
5. Should have very low latency.
6. Should have data logging capabilities.

Flagr fits the bill for all the above requirements mentioned and does it with a very low latency (it's written in Golang and caches data in memory to avoid it's own DB calls).

We were looking to create a robust template for our micro services setup that could do the following:

• Integrate well with the underlying messaging layer.
• Good logging
• Default container support
• Ability to expose web services, consumers and repeated tasks
• Caching support

Moleculer fits the bill for all of these requirements and much more.

We are still in the experiment phase, with only a couple of services live on Moleculer and will write more about it, after verifying.

Why

We wanted a way to start tracking our own events to have better control over them and support variety of analytics use cases including product and business decisions. Snowplow has done a lot of heavy lifting, especially robust collectors, enrichers and data dumping to S3, using the real time pipeline.

Setup

The setup itself was a little complicated because of not-so-good documentation. However, the Snowplow community is quite active and the maintainers are quick to respond on the forum.