What is Octopus Deploy and what are its top alternatives?
Top Alternatives to Octopus Deploy
- Jenkins
In a nutshell Jenkins CI is the leading open-source continuous integration server. Built with Java, it provides over 300 plugins to support building and testing virtually any project. ...
- Ansible
Ansible is an IT automation tool. It can configure systems, deploy software, and orchestrate more advanced IT tasks such as continuous deployments or zero downtime rolling updates. Ansible’s goals are foremost those of simplicity and maximum ease of use. ...
- TeamCity
TeamCity is a user-friendly continuous integration (CI) server for professional developers, build engineers, and DevOps. It is trivial to setup and absolutely free for small teams and open source projects. ...
- Chef
Chef enables you to manage and scale cloud infrastructure with no downtime or interruptions. Freely move applications and configurations from one cloud to another. Chef is integrated with all major cloud providers including Amazon EC2, VMWare, IBM Smartcloud, Rackspace, OpenStack, Windows Azure, HP Cloud, Google Compute Engine, Joyent Cloud and others. ...
- Bamboo
Focus on coding and count on Bamboo as your CI and build server! Create multi-stage build plans, set up triggers to start builds upon commits, and assign agents to your critical builds and deployments. ...
- Docker
The Docker Platform is the industry-leading container platform for continuous, high-velocity innovation, enabling organizations to seamlessly build and share any application — from legacy to what comes next — and securely run them anywhere ...
- Azure DevOps
Azure DevOps provides unlimited private Git hosting, cloud build for continuous integration, agile planning, and release management for continuous delivery to the cloud and on-premises. Includes broad IDE support. ...
- GitLab
GitLab offers git repository management, code reviews, issue tracking, activity feeds and wikis. Enterprises install GitLab on-premise and connect it with LDAP and Active Directory servers for secure authentication and authorization. A single GitLab server can handle more than 25,000 users but it is also possible to create a high availability setup with multiple active servers. ...
Octopus Deploy alternatives & related posts
- Hosted internally523
- Free open source469
- Great to build, deploy or launch anything async318
- Tons of integrations243
- Rich set of plugins with good documentation211
- Has support for build pipelines111
- Easy setup68
- It is open-source66
- Workflow plugin53
- Configuration as code13
- Very powerful tool12
- Many Plugins11
- Continuous Integration10
- Great flexibility10
- Git and Maven integration is better9
- 100% free and open source8
- Github integration7
- Slack Integration (plugin)7
- Easy customisation6
- Self-hosted GitLab Integration (plugin)6
- Docker support5
- Pipeline API5
- Fast builds4
- Platform idnependency4
- Hosted Externally4
- Excellent docker integration4
- It`w worked3
- Customizable3
- Can be run as a Docker container3
- It's Everywhere3
- JOBDSL3
- AWS Integration3
- Easily extendable with seamless integration2
- PHP Support2
- Build PR Branch Only2
- NodeJS Support2
- Ruby/Rails Support2
- Universal controller2
- Loose Coupling2
- Workarounds needed for basic requirements13
- Groovy with cumbersome syntax10
- Plugins compatibility issues8
- Lack of support7
- Limited abilities with declarative pipelines7
- No YAML syntax5
- Too tied to plugins versions4
related Jenkins posts
Often enough I have to explain my way of going about setting up a CI/CD pipeline with multiple deployment platforms. Since I am a bit tired of yapping the same every single time, I've decided to write it up and share with the world this way, and send people to read it instead ;). I will explain it on "live-example" of how the Rome got built, basing that current methodology exists only of readme.md and wishes of good luck (as it usually is ;)).
It always starts with an app, whatever it may be and reading the readmes available while Vagrant and VirtualBox is installing and updating. Following that is the first hurdle to go over - convert all the instruction/scripts into Ansible playbook(s), and only stopping when doing a clear vagrant up
or vagrant reload
we will have a fully working environment. As our Vagrant environment is now functional, it's time to break it! This is the moment to look for how things can be done better (too rigid/too lose versioning? Sloppy environment setup?) and replace them with the right way to do stuff, one that won't bite us in the backside. This is the point, and the best opportunity, to upcycle the existing way of doing dev environment to produce a proper, production-grade product.
I should probably digress here for a moment and explain why. I firmly believe that the way you deploy production is the same way you should deploy develop, shy of few debugging-friendly setting. This way you avoid the discrepancy between how production work vs how development works, which almost always causes major pains in the back of the neck, and with use of proper tools should mean no more work for the developers. That's why we start with Vagrant as developer boxes should be as easy as vagrant up
, but the meat of our product lies in Ansible which will do meat of the work and can be applied to almost anything: AWS, bare metal, docker, LXC, in open net, behind vpn - you name it.
We must also give proper consideration to monitoring and logging hoovering at this point. My generic answer here is to grab Elasticsearch, Kibana, and Logstash. While for different use cases there may be better solutions, this one is well battle-tested, performs reasonably and is very easy to scale both vertically (within some limits) and horizontally. Logstash rules are easy to write and are well supported in maintenance through Ansible, which as I've mentioned earlier, are at the very core of things, and creating triggers/reports and alerts based on Elastic and Kibana is generally a breeze, including some quite complex aggregations.
If we are happy with the state of the Ansible it's time to move on and put all those roles and playbooks to work. Namely, we need something to manage our CI/CD pipelines. For me, the choice is obvious: TeamCity. It's modern, robust and unlike most of the light-weight alternatives, it's transparent. What I mean by that is that it doesn't tell you how to do things, doesn't limit your ways to deploy, or test, or package for that matter. Instead, it provides a developer-friendly and rich playground for your pipelines. You can do most the same with Jenkins, but it has a quite dated look and feel to it, while also missing some key functionality that must be brought in via plugins (like quality REST API which comes built-in with TeamCity). It also comes with all the common-handy plugins like Slack or Apache Maven integration.
The exact flow between CI and CD varies too greatly from one application to another to describe, so I will outline a few rules that guide me in it: 1. Make build steps as small as possible. This way when something breaks, we know exactly where, without needing to dig and root around. 2. All security credentials besides development environment must be sources from individual Vault instances. Keys to those containers should exist only on the CI/CD box and accessible by a few people (the less the better). This is pretty self-explanatory, as anything besides dev may contain sensitive data and, at times, be public-facing. Because of that appropriate security must be present. TeamCity shines in this department with excellent secrets-management. 3. Every part of the build chain shall consume and produce artifacts. If it creates nothing, it likely shouldn't be its own build. This way if any issue shows up with any environment or version, all developer has to do it is grab appropriate artifacts to reproduce the issue locally. 4. Deployment builds should be directly tied to specific Git branches/tags. This enables much easier tracking of what caused an issue, including automated identifying and tagging the author (nothing like automated regression testing!).
Speaking of deployments, I generally try to keep it simple but also with a close eye on the wallet. Because of that, I am more than happy with AWS or another cloud provider, but also constantly peeking at the loads and do we get the value of what we are paying for. Often enough the pattern of use is not constantly erratic, but rather has a firm baseline which could be migrated away from the cloud and into bare metal boxes. That is another part where this approach strongly triumphs over the common Docker and CircleCI setup, where you are very much tied in to use cloud providers and getting out is expensive. Here to embrace bare-metal hosting all you need is a help of some container-based self-hosting software, my personal preference is with Proxmox and LXC. Following that all you must write are ansible scripts to manage hardware of Proxmox, similar way as you do for Amazon EC2 (ansible supports both greatly) and you are good to go. One does not exclude another, quite the opposite, as they can live in great synergy and cut your costs dramatically (the heavier your base load, the bigger the savings) while providing production-grade resiliency.
Releasing new versions of our services is done by Travis CI. Travis first runs our test suite. Once it passes, it publishes a new release binary to GitHub.
Common tasks such as installing dependencies for the Go project, or building a binary are automated using plain old Makefiles. (We know, crazy old school, right?) Our binaries are compressed using UPX.
Travis has come a long way over the past years. I used to prefer Jenkins in some cases since it was easier to debug broken builds. With the addition of the aptly named “debug build” button, Travis is now the clear winner. It’s easy to use and free for open source, with no need to maintain anything.
#ContinuousIntegration #CodeCollaborationVersionControl
Ansible
- Agentless284
- Great configuration210
- Simple199
- Powerful176
- Easy to learn155
- Flexible69
- Doesn't get in the way of getting s--- done55
- Makes sense35
- Super efficient and flexible30
- Powerful27
- Dynamic Inventory11
- Backed by Red Hat9
- Works with AWS7
- Cloud Oriented6
- Easy to maintain6
- Vagrant provisioner4
- Simple and powerful4
- Multi language4
- Simple4
- Because SSH4
- Procedural or declarative, or both4
- Easy4
- Consistency3
- Well-documented2
- Masterless2
- Debugging is simple2
- Merge hash to get final configuration similar to hiera2
- Fast as hell2
- Manage any OS1
- Work on windows, but difficult to manage1
- Certified Content1
- Dangerous8
- Hard to install5
- Doesn't Run on Windows3
- Bloated3
- Backward compatibility3
- No immutable infrastructure2
related Ansible posts
Often enough I have to explain my way of going about setting up a CI/CD pipeline with multiple deployment platforms. Since I am a bit tired of yapping the same every single time, I've decided to write it up and share with the world this way, and send people to read it instead ;). I will explain it on "live-example" of how the Rome got built, basing that current methodology exists only of readme.md and wishes of good luck (as it usually is ;)).
It always starts with an app, whatever it may be and reading the readmes available while Vagrant and VirtualBox is installing and updating. Following that is the first hurdle to go over - convert all the instruction/scripts into Ansible playbook(s), and only stopping when doing a clear vagrant up
or vagrant reload
we will have a fully working environment. As our Vagrant environment is now functional, it's time to break it! This is the moment to look for how things can be done better (too rigid/too lose versioning? Sloppy environment setup?) and replace them with the right way to do stuff, one that won't bite us in the backside. This is the point, and the best opportunity, to upcycle the existing way of doing dev environment to produce a proper, production-grade product.
I should probably digress here for a moment and explain why. I firmly believe that the way you deploy production is the same way you should deploy develop, shy of few debugging-friendly setting. This way you avoid the discrepancy between how production work vs how development works, which almost always causes major pains in the back of the neck, and with use of proper tools should mean no more work for the developers. That's why we start with Vagrant as developer boxes should be as easy as vagrant up
, but the meat of our product lies in Ansible which will do meat of the work and can be applied to almost anything: AWS, bare metal, docker, LXC, in open net, behind vpn - you name it.
We must also give proper consideration to monitoring and logging hoovering at this point. My generic answer here is to grab Elasticsearch, Kibana, and Logstash. While for different use cases there may be better solutions, this one is well battle-tested, performs reasonably and is very easy to scale both vertically (within some limits) and horizontally. Logstash rules are easy to write and are well supported in maintenance through Ansible, which as I've mentioned earlier, are at the very core of things, and creating triggers/reports and alerts based on Elastic and Kibana is generally a breeze, including some quite complex aggregations.
If we are happy with the state of the Ansible it's time to move on and put all those roles and playbooks to work. Namely, we need something to manage our CI/CD pipelines. For me, the choice is obvious: TeamCity. It's modern, robust and unlike most of the light-weight alternatives, it's transparent. What I mean by that is that it doesn't tell you how to do things, doesn't limit your ways to deploy, or test, or package for that matter. Instead, it provides a developer-friendly and rich playground for your pipelines. You can do most the same with Jenkins, but it has a quite dated look and feel to it, while also missing some key functionality that must be brought in via plugins (like quality REST API which comes built-in with TeamCity). It also comes with all the common-handy plugins like Slack or Apache Maven integration.
The exact flow between CI and CD varies too greatly from one application to another to describe, so I will outline a few rules that guide me in it: 1. Make build steps as small as possible. This way when something breaks, we know exactly where, without needing to dig and root around. 2. All security credentials besides development environment must be sources from individual Vault instances. Keys to those containers should exist only on the CI/CD box and accessible by a few people (the less the better). This is pretty self-explanatory, as anything besides dev may contain sensitive data and, at times, be public-facing. Because of that appropriate security must be present. TeamCity shines in this department with excellent secrets-management. 3. Every part of the build chain shall consume and produce artifacts. If it creates nothing, it likely shouldn't be its own build. This way if any issue shows up with any environment or version, all developer has to do it is grab appropriate artifacts to reproduce the issue locally. 4. Deployment builds should be directly tied to specific Git branches/tags. This enables much easier tracking of what caused an issue, including automated identifying and tagging the author (nothing like automated regression testing!).
Speaking of deployments, I generally try to keep it simple but also with a close eye on the wallet. Because of that, I am more than happy with AWS or another cloud provider, but also constantly peeking at the loads and do we get the value of what we are paying for. Often enough the pattern of use is not constantly erratic, but rather has a firm baseline which could be migrated away from the cloud and into bare metal boxes. That is another part where this approach strongly triumphs over the common Docker and CircleCI setup, where you are very much tied in to use cloud providers and getting out is expensive. Here to embrace bare-metal hosting all you need is a help of some container-based self-hosting software, my personal preference is with Proxmox and LXC. Following that all you must write are ansible scripts to manage hardware of Proxmox, similar way as you do for Amazon EC2 (ansible supports both greatly) and you are good to go. One does not exclude another, quite the opposite, as they can live in great synergy and cut your costs dramatically (the heavier your base load, the bigger the savings) while providing production-grade resiliency.
Heroku was a decent choice to start a business, but at some point our platform was too big, too complex & too heterogenic, so Heroku started to be a constraint, not a benefit. First, we've started containerizing our apps with Docker to eliminate "works in my machine" syndrome & uniformize the environment setup. The first orchestration was composed with Docker Compose , but at some point it made sense to move it to Kubernetes. Fortunately, we've made a very good technical decision when starting our work with containers - all the container configuration & provisions HAD (since the beginning) to be done in code (Infrastructure as Code) - we've used Terraform & Ansible for that (correspondingly). This general trend of containerisation was accompanied by another, parallel & equally big project: migrating environments from Heroku to AWS: using Amazon EC2 , Amazon EKS, Amazon S3 & Amazon RDS.
TeamCity
- Easy to configure61
- Reliable and high-quality37
- User friendly32
- On premise32
- Github integration32
- Great UI18
- Smart16
- Free for open source12
- Can run jobs in parallel12
- Crossplatform8
- Chain dependencies5
- Fully-functional out of the box5
- Great support by jetbrains4
- REST API4
- Projects hierarchy4
- 100+ plugins4
- Personal notifications3
- Free for small teams3
- Build templates3
- Per-project permissions3
- Upload build artifacts2
- Smart build failure analysis and tracking2
- Ide plugins2
- GitLab integration2
- Artifact dependencies2
- Official reliable support2
- Build progress messages promoting from running process2
- Repository-stored, full settings dsl with ide support1
- Built-in artifacts repository1
- Powerful build chains / pipelines1
- TeamCity Professional is FREE1
- High-Availability0
- Hosted internally0
- High costs for more than three build agents3
- Proprietary2
- User-friendly2
- User friendly2
related TeamCity posts
Often enough I have to explain my way of going about setting up a CI/CD pipeline with multiple deployment platforms. Since I am a bit tired of yapping the same every single time, I've decided to write it up and share with the world this way, and send people to read it instead ;). I will explain it on "live-example" of how the Rome got built, basing that current methodology exists only of readme.md and wishes of good luck (as it usually is ;)).
It always starts with an app, whatever it may be and reading the readmes available while Vagrant and VirtualBox is installing and updating. Following that is the first hurdle to go over - convert all the instruction/scripts into Ansible playbook(s), and only stopping when doing a clear vagrant up
or vagrant reload
we will have a fully working environment. As our Vagrant environment is now functional, it's time to break it! This is the moment to look for how things can be done better (too rigid/too lose versioning? Sloppy environment setup?) and replace them with the right way to do stuff, one that won't bite us in the backside. This is the point, and the best opportunity, to upcycle the existing way of doing dev environment to produce a proper, production-grade product.
I should probably digress here for a moment and explain why. I firmly believe that the way you deploy production is the same way you should deploy develop, shy of few debugging-friendly setting. This way you avoid the discrepancy between how production work vs how development works, which almost always causes major pains in the back of the neck, and with use of proper tools should mean no more work for the developers. That's why we start with Vagrant as developer boxes should be as easy as vagrant up
, but the meat of our product lies in Ansible which will do meat of the work and can be applied to almost anything: AWS, bare metal, docker, LXC, in open net, behind vpn - you name it.
We must also give proper consideration to monitoring and logging hoovering at this point. My generic answer here is to grab Elasticsearch, Kibana, and Logstash. While for different use cases there may be better solutions, this one is well battle-tested, performs reasonably and is very easy to scale both vertically (within some limits) and horizontally. Logstash rules are easy to write and are well supported in maintenance through Ansible, which as I've mentioned earlier, are at the very core of things, and creating triggers/reports and alerts based on Elastic and Kibana is generally a breeze, including some quite complex aggregations.
If we are happy with the state of the Ansible it's time to move on and put all those roles and playbooks to work. Namely, we need something to manage our CI/CD pipelines. For me, the choice is obvious: TeamCity. It's modern, robust and unlike most of the light-weight alternatives, it's transparent. What I mean by that is that it doesn't tell you how to do things, doesn't limit your ways to deploy, or test, or package for that matter. Instead, it provides a developer-friendly and rich playground for your pipelines. You can do most the same with Jenkins, but it has a quite dated look and feel to it, while also missing some key functionality that must be brought in via plugins (like quality REST API which comes built-in with TeamCity). It also comes with all the common-handy plugins like Slack or Apache Maven integration.
The exact flow between CI and CD varies too greatly from one application to another to describe, so I will outline a few rules that guide me in it: 1. Make build steps as small as possible. This way when something breaks, we know exactly where, without needing to dig and root around. 2. All security credentials besides development environment must be sources from individual Vault instances. Keys to those containers should exist only on the CI/CD box and accessible by a few people (the less the better). This is pretty self-explanatory, as anything besides dev may contain sensitive data and, at times, be public-facing. Because of that appropriate security must be present. TeamCity shines in this department with excellent secrets-management. 3. Every part of the build chain shall consume and produce artifacts. If it creates nothing, it likely shouldn't be its own build. This way if any issue shows up with any environment or version, all developer has to do it is grab appropriate artifacts to reproduce the issue locally. 4. Deployment builds should be directly tied to specific Git branches/tags. This enables much easier tracking of what caused an issue, including automated identifying and tagging the author (nothing like automated regression testing!).
Speaking of deployments, I generally try to keep it simple but also with a close eye on the wallet. Because of that, I am more than happy with AWS or another cloud provider, but also constantly peeking at the loads and do we get the value of what we are paying for. Often enough the pattern of use is not constantly erratic, but rather has a firm baseline which could be migrated away from the cloud and into bare metal boxes. That is another part where this approach strongly triumphs over the common Docker and CircleCI setup, where you are very much tied in to use cloud providers and getting out is expensive. Here to embrace bare-metal hosting all you need is a help of some container-based self-hosting software, my personal preference is with Proxmox and LXC. Following that all you must write are ansible scripts to manage hardware of Proxmox, similar way as you do for Amazon EC2 (ansible supports both greatly) and you are good to go. One does not exclude another, quite the opposite, as they can live in great synergy and cut your costs dramatically (the heavier your base load, the bigger the savings) while providing production-grade resiliency.
@producthunt LambdaTest Selenium JavaScript Java Python PHP Cucumber TeamCity CircleCI With this new release of LambdaTest automation, you can run tests across an Online Selenium Grid of 2000+ browsers and OS combinations to perform cross browser testing. This saves you from the pain of maintaining the infrastructure and also saves you the licensing costs for browsers and operating systems. #testing #Seleniumgrid #Selenium #testautomation #automation #webdriver #producthunt hunted
- Dynamic and idempotent server configuration110
- Reusable components76
- Integration testing with Vagrant47
- Repeatable43
- Mock testing with Chefspec30
- Ruby14
- Can package cookbooks to guarantee repeatability8
- Works with AWS7
- Has marketplace where you get readymade cookbooks3
- Matured product with good community support3
- Less declarative more procedural2
- Open source configuration mgmt made easy(ish)2
related Chef posts
In late 2013, the Operations Engineering team at PagerDuty was made up of 4 engineers, and was comprised of generalists, each of whom had one or two areas of depth. Although the Operations Team ran its own on-call, each engineering team at PagerDuty also participated on the pager.
The Operations Engineering Team owned 150+ servers spanning multiple cloud providers, and used Chef to automate their infrastructure across the various cloud providers with a mix of completely custom cookbooks and customized community cookbooks.
Custom cookbooks were managed by Berkshelf, andach custom cookbook contained its own tests based on ChefSpec 3, coupled with Rspec.
Jenkins was used to GitHub for new changes and to handle unit testing of those features.
Since #ATComputing is a vendor independent Linux and open source specialist, we do not have a favorite Linux distribution. We mainly use Ubuntu , Centos Debian , Red Hat Enterprise Linux and Fedora during our daily work. These are also the distributions we see most often used in our customers environments.
For our #ci/cd training, we use an open source pipeline that is build around Visual Studio Code , Jenkins , VirtualBox , GitHub , Docker Kubernetes and Google Compute Engine.
For #ServerConfigurationAndAutomation, we have embraced and contributed to Ansible mainly because it is not only flexible and powerful, but also straightforward and easier to learn than some other (open source) solutions. On the other hand: we are not affraid of Puppet Labs and Chef either.
Currently, our most popular #programming #Language course is Python . The reason Python is so popular has to do with it's versatility, but also with its low complexity. This helps sysadmins to write scripts or simple programs to make their job less repetitive and automating things more fun. Python is also widely used to communicate with (REST) API's and for data analysis.
- Integrates with other Atlassian tools10
- Great notification scheme4
- Great UI2
- Has Deployment Projects1
- Expensive6
- Low community support1
- Bad UI1
- Bad integration with docker1
related Bamboo posts
I am choosing a DevOps toolset for my team. GitLab is open source and quite cloud-native. Jenkins has a very popular environment system but old-style technicals. Bamboo is very nice but integrated only with Atlassian products.
Need for CI/CD as part of data testing team.
Please suggest.
Currently, Bamboo is most commonly used in the company while GitLab is rated highly by the internal team.
Docker
- Rapid integration and build up823
- Isolation692
- Open source521
- Testability and reproducibility505
- Lightweight460
- Standardization218
- Scalable185
- Upgrading / downgrading / application versions106
- Security88
- Private paas environments85
- Portability34
- Limit resource usage26
- Game changer17
- I love the way docker has changed virtualization16
- Fast14
- Concurrency12
- Docker's Compose tools8
- Fast and Portable6
- Easy setup6
- Because its fun5
- Makes shipping to production very simple4
- It's dope3
- Highly useful3
- Does a nice job hogging memory2
- Open source and highly configurable2
- Simplicity, isolation, resource effective2
- MacOS support FAKE2
- Its cool2
- Docker hub for the FTW2
- HIgh Throughput2
- Very easy to setup integrate and build2
- Package the environment with the application2
- Super2
- Asdfd0
- New versions == broken features8
- Unreliable networking6
- Documentation not always in sync6
- Moves quickly4
- Not Secure3
related Docker posts
Our whole DevOps stack consists of the following tools:
- GitHub (incl. GitHub Pages/Markdown for Documentation, GettingStarted and HowTo's) for collaborative review and code management tool
- Respectively Git as revision control system
- SourceTree as Git GUI
- Visual Studio Code as IDE
- CircleCI for continuous integration (automatize development process)
- Prettier / TSLint / ESLint as code linter
- SonarQube as quality gate
- Docker as container management (incl. Docker Compose for multi-container application management)
- VirtualBox for operating system simulation tests
- Kubernetes as cluster management for docker containers
- Heroku for deploying in test environments
- nginx as web server (preferably used as facade server in production environment)
- SSLMate (using OpenSSL) for certificate management
- Amazon EC2 (incl. Amazon S3) for deploying in stage (production-like) and production environments
- PostgreSQL as preferred database system
- Redis as preferred in-memory database/store (great for caching)
The main reason we have chosen Kubernetes over Docker Swarm is related to the following artifacts:
- Key features: Easy and flexible installation, Clear dashboard, Great scaling operations, Monitoring is an integral part, Great load balancing concepts, Monitors the condition and ensures compensation in the event of failure.
- Applications: An application can be deployed using a combination of pods, deployments, and services (or micro-services).
- Functionality: Kubernetes as a complex installation and setup process, but it not as limited as Docker Swarm.
- Monitoring: It supports multiple versions of logging and monitoring when the services are deployed within the cluster (Elasticsearch/Kibana (ELK), Heapster/Grafana, Sysdig cloud integration).
- Scalability: All-in-one framework for distributed systems.
- Other Benefits: Kubernetes is backed by the Cloud Native Computing Foundation (CNCF), huge community among container orchestration tools, it is an open source and modular tool that works with any OS.
Often enough I have to explain my way of going about setting up a CI/CD pipeline with multiple deployment platforms. Since I am a bit tired of yapping the same every single time, I've decided to write it up and share with the world this way, and send people to read it instead ;). I will explain it on "live-example" of how the Rome got built, basing that current methodology exists only of readme.md and wishes of good luck (as it usually is ;)).
It always starts with an app, whatever it may be and reading the readmes available while Vagrant and VirtualBox is installing and updating. Following that is the first hurdle to go over - convert all the instruction/scripts into Ansible playbook(s), and only stopping when doing a clear vagrant up
or vagrant reload
we will have a fully working environment. As our Vagrant environment is now functional, it's time to break it! This is the moment to look for how things can be done better (too rigid/too lose versioning? Sloppy environment setup?) and replace them with the right way to do stuff, one that won't bite us in the backside. This is the point, and the best opportunity, to upcycle the existing way of doing dev environment to produce a proper, production-grade product.
I should probably digress here for a moment and explain why. I firmly believe that the way you deploy production is the same way you should deploy develop, shy of few debugging-friendly setting. This way you avoid the discrepancy between how production work vs how development works, which almost always causes major pains in the back of the neck, and with use of proper tools should mean no more work for the developers. That's why we start with Vagrant as developer boxes should be as easy as vagrant up
, but the meat of our product lies in Ansible which will do meat of the work and can be applied to almost anything: AWS, bare metal, docker, LXC, in open net, behind vpn - you name it.
We must also give proper consideration to monitoring and logging hoovering at this point. My generic answer here is to grab Elasticsearch, Kibana, and Logstash. While for different use cases there may be better solutions, this one is well battle-tested, performs reasonably and is very easy to scale both vertically (within some limits) and horizontally. Logstash rules are easy to write and are well supported in maintenance through Ansible, which as I've mentioned earlier, are at the very core of things, and creating triggers/reports and alerts based on Elastic and Kibana is generally a breeze, including some quite complex aggregations.
If we are happy with the state of the Ansible it's time to move on and put all those roles and playbooks to work. Namely, we need something to manage our CI/CD pipelines. For me, the choice is obvious: TeamCity. It's modern, robust and unlike most of the light-weight alternatives, it's transparent. What I mean by that is that it doesn't tell you how to do things, doesn't limit your ways to deploy, or test, or package for that matter. Instead, it provides a developer-friendly and rich playground for your pipelines. You can do most the same with Jenkins, but it has a quite dated look and feel to it, while also missing some key functionality that must be brought in via plugins (like quality REST API which comes built-in with TeamCity). It also comes with all the common-handy plugins like Slack or Apache Maven integration.
The exact flow between CI and CD varies too greatly from one application to another to describe, so I will outline a few rules that guide me in it: 1. Make build steps as small as possible. This way when something breaks, we know exactly where, without needing to dig and root around. 2. All security credentials besides development environment must be sources from individual Vault instances. Keys to those containers should exist only on the CI/CD box and accessible by a few people (the less the better). This is pretty self-explanatory, as anything besides dev may contain sensitive data and, at times, be public-facing. Because of that appropriate security must be present. TeamCity shines in this department with excellent secrets-management. 3. Every part of the build chain shall consume and produce artifacts. If it creates nothing, it likely shouldn't be its own build. This way if any issue shows up with any environment or version, all developer has to do it is grab appropriate artifacts to reproduce the issue locally. 4. Deployment builds should be directly tied to specific Git branches/tags. This enables much easier tracking of what caused an issue, including automated identifying and tagging the author (nothing like automated regression testing!).
Speaking of deployments, I generally try to keep it simple but also with a close eye on the wallet. Because of that, I am more than happy with AWS or another cloud provider, but also constantly peeking at the loads and do we get the value of what we are paying for. Often enough the pattern of use is not constantly erratic, but rather has a firm baseline which could be migrated away from the cloud and into bare metal boxes. That is another part where this approach strongly triumphs over the common Docker and CircleCI setup, where you are very much tied in to use cloud providers and getting out is expensive. Here to embrace bare-metal hosting all you need is a help of some container-based self-hosting software, my personal preference is with Proxmox and LXC. Following that all you must write are ansible scripts to manage hardware of Proxmox, similar way as you do for Amazon EC2 (ansible supports both greatly) and you are good to go. One does not exclude another, quite the opposite, as they can live in great synergy and cut your costs dramatically (the heavier your base load, the bigger the savings) while providing production-grade resiliency.
Azure DevOps
- Complete and powerful56
- Huge extension ecosystem32
- Azure integration27
- Flexible and powerful26
- One Stop Shop For Build server, Project Mgt, CDCI26
- Everything I need. Simple and intuitive UI15
- Support Open Source13
- Integrations8
- GitHub Integration7
- Cost free for Stakeholders6
- One 4 all6
- Crap6
- Project Mgmt Features6
- Runs in the cloud5
- Agent On-Premise(Linux - Windows)3
- Aws integration2
- Link Test Cases to Stories2
- Jenkins Integration2
- GCP Integration1
- Still dependant on C# for agents8
- Half Baked5
- Many in devops disregard MS altogether5
- Not a requirements management tool4
- Jack of all trades, master of none4
- Capacity across cross functional teams not visibile4
- Poor Jenkins integration3
- Tedious for test plan/case creation2
- Switching accounts is impossible1
related Azure DevOps posts
Visual Studio Azure DevOps Azure Functions Azure Websites #Azure #AzureKeyVault #AzureAD #AzureApps
#Azure Cloud Since Amazon is potentially our competitor then we need a different cloud vendor, also our programmers are microsoft oriented so the choose were obviously #Azure for us.
Azure DevOps Because we need to be able to develop a neww pipeline into Azure environment ina few minutes.
Azure Kubernetes Service We already in #Azure , also need to use K8s , so let's use AKS as it's a manged Kubernetes in the #Azure
I use Azure DevOps because for me it gradually walk me from private Git repositories to simplest free option for CI/CD pipelines at the time. I spend 0$ initially to manager CI/CD for my small private projects. No need to go into two different places to setup integration, once I have git repository, I could deploy projects. Right now this is not the case since CI/CD is default for me, so I use it now from memories of old good days. I'm not yet need complexity on the projects, so I don't even consider other options with "more choices". I carefully limit my set of options during development, that's why Azure DevOps (VSTS)
- Self hosted508
- Free431
- Has community edition339
- Easy setup242
- Familiar interface240
- Includes many features, including ci137
- Nice UI113
- Good integration with gitlabci84
- Simple setup57
- Has an official mobile app35
- Free private repository34
- Continuous Integration31
- Open source, great ui (like github)23
- Slack Integration18
- Full CI flow15
- Free and unlimited private git repos11
- All in one (Git, CI, Agile..)10
- User, group, and project access management is simple10
- Intuitive UI8
- Built-in CI8
- Full DevOps suite with Git6
- Both public and private Repositories6
- Integrated Docker Registry5
- So easy to use5
- CI5
- Build/pipeline definition alongside code5
- It's powerful source code management tool5
- Dockerized4
- It's fully integrated4
- On-premises4
- Security and Stable4
- Unlimited free repos & collaborators4
- Not Microsoft Owned4
- Excellent4
- Issue system4
- Mattermost Chat client4
- Great for team collaboration3
- Free private repos3
- Because is the best remote host for git repositories3
- Built-in Docker Registry3
- Opensource3
- Low maintenance cost due omnibus-deployment3
- I like the its runners and executors feature3
- Beautiful2
- Groups of groups2
- Multilingual interface2
- Powerful software planning and maintaining tools2
- Review Apps feature2
- Kubernetes integration with GitLab CI2
- One-click install through DigitalOcean2
- Powerful Continuous Integration System2
- It includes everything I need, all packaged with docker2
- The dashboard with deployed environments2
- HipChat intergration2
- Many private repo2
- Kubernetes Integration2
- Published IP list for whitelisting (gl-infra#434)2
- Wounderful2
- Native CI2
- Supports Radius/Ldap & Browser Code Edits1
- Slow ui performance28
- Introduce breaking bugs every release9
- Insecure (no published IP list for whitelisting)6
- Built-in Docker Registry2
- Review Apps feature1
related GitLab posts
I have mixed feelings on GitHub as a product and our use of it for the Zulip open source project. On the one hand, I do feel that being on GitHub helps people discover Zulip, because we have enough stars (etc.) that we rank highly among projects on the platform. and there is a definite benefit for lowering barriers to contribution (which is important to us) that GitHub has such a dominant position in terms of what everyone has accounts with.
But even ignoring how one might feel about their new corporate owner (MicroSoft), in a lot of ways GitHub is a bad product for open source projects. Years after the "Dear GitHub" letter, there are still basic gaps in its issue tracker:
- You can't give someone permission to label/categorize issues without full write access to a project (including ability to merge things to master, post releases, etc.).
- You can't let anyone with a GitHub account self-assign issues to themselves.
- Many more similar issues.
It's embarrassing, because I've talked to GitHub product managers at various open source events about these things for 3 years, and they always agree the thing is important, but then nothing ever improves in the Issues product. Maybe the new management at MicroSoft will fix their product management situation, but if not, I imagine we'll eventually do the migration to GitLab.
We have a custom bot project, http://github.com/zulip/zulipbot, to deal with some of these issues where possible, and every other large project we talk to does the same thing, more or less.
We use GitLab CI because of the great native integration as a part of the GitLab framework and the linting-capabilities it offers. The visualization of complex pipelines and the embedding within the project overview made Gitlab CI even more convenient. We use it for all projects, all deployments and as a part of GitLab Pages.
While we initially used the Shell-executor, we quickly switched to the Docker-executor and use it exclusively now.
We formerly used Jenkins but preferred to handle everything within GitLab . Aside from the unification of our infrastructure another motivation was the "configuration-in-file"-approach, that Gitlab CI offered, while Jenkins support of this concept was very limited and users had to resort to using the webinterface. Since the file is included within the repository, it is also version controlled, which was a huge plus for us.