I'm analyzing companies with stacks similar to my company, as we are in a process of breaking the monolith for microservices. I noticed that your stack is very similar to ours, Python, Django, Celery, and so on. Analyzing the technology you use I could see the use of Go and Kafka which made me think that you also went through a similar process. So here's my question I would like to know what were some of the reasons why you adopted GO in your ecosystem?? scalability, performance?? We are looking into the possibility of starting to use GO but for that, I wanted to know why use GO instead of Python?? or why to use both together?? It may be something trivial but all experience and opinion are important to us?? Thanks.
The main reasons for using Go as a Kafka consumer at Deliveroo are (1) simplicity and reliabiliy of code (2) speed of execution.
For (1), Python works just as well and indeed, we do have some low-scale applications using Python as it tends to be more productive for smaller and/or shorter life projects. But Go remains upwards compatible since its inception while the 2->3 conversion in Python was no small matter, and even minor version conversions tend to require many dependency upgrades, which tend not to be synchronised. For (2) at high scale (think 1M transactions/5 minutes), Go is simpler to scale on less hardware. We also use Kotlin and Scala for this because, beyond pure manually written consumers, Go misses a real equivalent to Kafka Streams, as that only exists on the JVM.
also a big thing in production systems , type-safety.
At scale python and similar systems will cry . so Go is a good candidate if you like to keep things maintainable and simple
