Kubernetes logo

Kubernetes

Manage a cluster of Linux containers as a single system to accelerate Dev and simplify Ops
26.2K
20.9K
+ 1
576

What is Kubernetes?

Kubernetes is an open source orchestration system for Docker containers. It handles scheduling onto nodes in a compute cluster and actively manages workloads to ensure that their state matches the users declared intentions.
Kubernetes is a tool in the Container Tools category of a tech stack.
Kubernetes is an open source tool with 71.7K GitHub stars and 26K GitHub forks. Here鈥檚 a link to Kubernetes's open source repository on GitHub

Who uses Kubernetes?

Companies
2206 companies reportedly use Kubernetes in their tech stacks, including Google, Shopify, and Slack.

Developers
23538 developers on StackShare have stated that they use Kubernetes.

Kubernetes Integrations

Docker, Microsoft Azure, Ansible, Vagrant, and Google Compute Engine are some of the popular tools that integrate with Kubernetes. Here's a list of all 213 tools that integrate with Kubernetes.
Public Decisions about Kubernetes

Here are some stack decisions, common use cases and reviews by companies and developers who chose Kubernetes in their tech stack.

Emanuel Evans
Senior Architect at Rainforest QA | 13 upvotes 路 476.5K views

We recently moved our main applications from Heroku to Kubernetes . The 3 main driving factors behind the switch were scalability (database size limits), security (the inability to set up PostgreSQL instances in private networks), and costs (GCP is cheaper for raw computing resources).

We prefer using managed services, so we are using Google Kubernetes Engine with Google Cloud SQL for PostgreSQL for our PostgreSQL databases and Google Cloud Memorystore for Redis . For our CI/CD pipeline, we are using CircleCI and Google Cloud Build to deploy applications managed with Helm . The new infrastructure is managed with Terraform .

Read the blog post to go more in depth.

See more
Russel Werner
Lead Engineer at StackShare | 0 upvotes 路 20K views

We began our hosting journey, as many do, on Heroku because they make it easy to deploy your application and automate some of the routine tasks associated with deployments, etc. However, as our team grew and our product matured, our needs have outgrown Heroku. I will dive into the history and reasons for this in a future blog post.

We decided to migrate our infrastructure to Kubernetes running on Amazon EKS. Although Google Kubernetes Engine has a slightly more mature Kubernetes offering and is more user-friendly; we decided to go with EKS because we already using other AWS services (including a previous migration from Heroku Postgres to AWS RDS). We are still in the process of moving our main website workloads to EKS, however we have successfully migrate all our staging and testing PR apps to run in a staging cluster. We developed a Slack chatops application (also running in the cluster) which automates all the common tasks of spinning up and managing a production-like cluster for a pull request. This allows our engineering team to iterate quickly and safely test code in a full production environment. Helm plays a central role when deploying our staging apps into the cluster. We use CircleCI to build docker containers for each PR push, which are then published to Amazon EC2 Container Service (ECR). An upgrade-operator process watches the ECR repository for new containers and then uses Helm to rollout updates to the staging environments. All this happens automatically and makes it really easy for developers to get code onto servers quickly. The immutable and isolated nature of our staging environments means that we can do anything we want in that environment and quickly re-create or restore the environment to start over.

The next step in our journey is to migrate our production workloads to an EKS cluster and build out the CD workflows to get our containers promoted to that cluster after our QA testing is complete in our staging environments.

See more

We have Prometheus as a monitoring engine as a part of our stack which contains Kubernetes cluster, container images and other open source tools. Also, I am aware that Sysdig can be integrated with Prometheus but I really wanted to know whether Sysdig or sysdig+prometheus will make better monitoring solution.

See more
Joshua Dean K眉pper
CEO at Scrayos UG (haftungsbeschr盲nkt) | 1 upvotes 路 19.8K views

We primarily use Prometheus to gather metrics and statistics to display them in Grafana. Aside from that we poll Prometheus for our orchestration-solution "JCOverseer" to determine, which host is least occupied at the moment.

While there are existing orchestration softwares/suites like Kubernetes, that we also plan to adopt in the future, we're of the opinion that those solutions do not fit our special environment within minecraft and our own solution will outperform them in the limited scope that it needs to cover.

See more
Ashish Singh
Tech Lead, Big Data Platform at Pinterest | 34 upvotes 路 579.2K views

To provide employees with the critical need of interactive querying, we鈥檝e worked with Presto, an open-source distributed SQL query engine, over the years. Operating Presto at Pinterest鈥檚 scale has involved resolving quite a few challenges like, supporting deeply nested and huge thrift schemas, slow/ bad worker detection and remediation, auto-scaling cluster, graceful cluster shutdown and impersonation support for ldap authenticator.

Our infrastructure is built on top of Amazon EC2 and we leverage Amazon S3 for storing our data. This separates compute and storage layers, and allows multiple compute clusters to share the S3 data.

We have hundreds of petabytes of data and tens of thousands of Apache Hive tables. Our Presto clusters are comprised of a fleet of 450 r4.8xl EC2 instances. Presto clusters together have over 100 TBs of memory and 14K vcpu cores. Within Pinterest, we have close to more than 1,000 monthly active users (out of total 1,600+ Pinterest employees) using Presto, who run about 400K queries on these clusters per month.

Each query submitted to Presto cluster is logged to a Kafka topic via Singer. Singer is a logging agent built at Pinterest and we talked about it in a previous post. Each query is logged when it is submitted and when it finishes. When a Presto cluster crashes, we will have query submitted events without corresponding query finished events. These events enable us to capture the effect of cluster crashes over time.

Each Presto cluster at Pinterest has workers on a mix of dedicated AWS EC2 instances and Kubernetes pods. Kubernetes platform provides us with the capability to add and remove workers from a Presto cluster very quickly. The best-case latency on bringing up a new worker on Kubernetes is less than a minute. However, when the Kubernetes cluster itself is out of resources and needs to scale up, it can take up to ten minutes. Some other advantages of deploying on Kubernetes platform is that our Presto deployment becomes agnostic of cloud vendor, instance types, OS, etc.

#BigData #AWS #DataScience #DataEngineering

See more

We are building a product that runs both on-prem and on our Google Kubernetes Engine clusters, and I am working on building a monitoring solution.

Our app is dockerized and usually deployed using Kubernetes.

I am currently looking into tools for centralized logging, but there is a catch. Some of our customers do not allow exporting the logs to our cloud solution; so basically, I am looking for a solution that will work for all 3 use cases:

  1. Cloud clusters

  2. On-prem which can report to our central cloud logging solution

  3. On-prem which can be only accessed locally

We are currently using GCP Logging since it's pretty easy to get started with, but if it does not answer our use case, we are fine with replacing it.

I was considering ELK, but in my experience, it can be pretty complicated to manage.

Are there other recommended solutions?

See more

Blog Posts

3
524
May 21, 2020 at 12:02AM
https://img.stackshare.io/stack/650371/default_35848f736f9651e77452a8a4ce3c2c953360dedf.png logo

Rancher Labs

3
988
Apr 16, 2020 at 5:34AM
https://img.stackshare.io/stack/610930/default_dd31eaa73ce9b7325f68f6928685172a21da2b1f.png logo

Rancher Labs

2
714
Dec 4, 2019 at 8:01PM
https://img.stackshare.io/stack/519120/default_d8fbac59cb4c75cc8da7062d0dc1d02daa76fa62.png logo

Pinterest

5
2679

Kubernetes's Features

  • Lightweight, simple and accessible
  • Built for a multi-cloud world, public, private or hybrid
  • Highly modular, designed so that all of its components are easily swappable

Kubernetes Alternatives & Comparisons

What are some alternatives to Kubernetes?
Docker Swarm
Swarm serves the standard Docker API, so any tool which already communicates with a Docker daemon can use Swarm to transparently scale to multiple hosts: Dokku, Compose, Krane, Deis, DockerUI, Shipyard, Drone, Jenkins... and, of course, the Docker client itself.
Nomad
Nomad is a cluster manager, designed for both long lived services and short lived batch processing workloads. Developers use a declarative job specification to submit work, and Nomad ensures constraints are satisfied and resource utilization is optimized by efficient task packing. Nomad supports all major operating systems and virtualized, containerized, or standalone applications.
OpenStack
OpenStack is a cloud operating system that controls large pools of compute, storage, and networking resources throughout a datacenter, all managed through a dashboard that gives administrators control while empowering their users to provision resources through a web interface.
Rancher
Rancher is an open source container management platform that includes full distributions of Kubernetes, Apache Mesos and Docker Swarm, and makes it simple to operate container clusters on any cloud or infrastructure platform.
Docker Compose
With Compose, you define a multi-container application in a single file, then spin your application up in a single command which does everything that needs to be done to get it running.
See all alternatives

Kubernetes's Followers
20904 developers follow Kubernetes to keep up with related blogs and decisions.