I use Python because it is one of the most versatile and easy to read programming languages. The open source community is vibrant and there are so many tutorials and Medium blogs it can be overwhelming, and that's a good problem to have!

I primarily use it for automating backend infrastructure tasks, data exploration via Jupyter, and data engineering development. It's great to maintain most of my stack in one language for consistency.

HOWEVER, when it comes to scaling data engineering workloads compared to other languages like Java and Scala, performance speed degrades significantly. You'll notice that most of the big tech companies use Scala or Java for Spark because the Python API is still a second-class citizen in new releases.

ANOTHER HOWEVER, I'm excited for the future of parallelism in python and how that may replace complex spark workloads. It's still young, but growing: Ray