Docker

DevOps / Build, Test, Deploy / Virtual Machine Platforms & Containers
CEO at Scrayos UG (haftungsbeschränkt)·

Docker is used as a key part of our new type of infrastructure that relies exclusively on containers. We prefer docker since it offers us all we need in a very stable and battle-proven manner.

READ MORE
1 upvote·11.3K views
Shared insights

I have got a small radio service running on Node.js. Front end is written with React and packed with Webpack . I use Docker for my #DeploymentWorkflow along with Docker Swarm and GitLab CI on a single Google Compute Engine instance, which is also a runner itself. Pretty unscalable decision but it works great for tiny projects. The project is available on https://fridgefm.com

READ MORE
17 upvotes·2 comments·170.5K views
RISHAB SURANA
RISHAB SURANA
·
April 11th 2021 at 4:19PM

nice project

·
Reply
Sergey Sirota
Sergey Sirota
·
April 16th 2021 at 9:52AM

That's cool! Is front end should have white tape on left?)

·
Reply
Engineering Manager at Taylor and Francis·

We are in the process of building a modern content platform to deliver our content through various channels. We decided to go with Microservices architecture as we wanted scale. Microservice architecture style is an approach to developing an application as a suite of small independently deployable services built around specific business capabilities. You can gain modularity, extensive parallelism and cost-effective scaling by deploying services across many distributed servers. Microservices modularity facilitates independent updates/deployments, and helps to avoid single point of failure, which can help prevent large-scale outages. We also decided to use Event Driven Architecture pattern which is a popular distributed asynchronous architecture pattern used to produce highly scalable applications. The event-driven architecture is made up of highly decoupled, single-purpose event processing components that asynchronously receive and process events.

To build our #Backend capabilities we decided to use the following: 1. #Microservices - Java with Spring Boot , Node.js with ExpressJS and Python with Flask 2. #Eventsourcingframework - Amazon Kinesis , Amazon Kinesis Firehose , Amazon SNS , Amazon SQS, AWS Lambda 3. #Data - Amazon RDS , Amazon DynamoDB , Amazon S3 , MongoDB Atlas

To build #Webapps we decided to use Angular 2 with RxJS

#Devops - GitHub , Travis CI , Terraform , Docker , Serverless

READ MORE
14 upvotes·1 comment·1.9M views
Jon Senterfitt
Jon Senterfitt
·
January 29th 2021 at 7:07AM

But why not just use Contentful?

·
Reply

We are planning to choose Docker since it will allow us to build and install libraries and dependencies with ease. Its extensive use in the world will be helpful to provide us with useful discussion boards. This will be the first time any member of the dev team will be using Docker as part of their application. Given the limited readings, we have been able to do about it in the time we had, we a really excited to get to work with it. It seems to have a lot of potential that we would like to explore as a team. Another reason is that our dev team currently only has access to Windows machines and we want our application to be system agnostic. Using Docker will also help us limit the number of CI minutes our application requires.

READ MORE
18 upvotes·49.9K views
CEO at Scrayos UG (haftungsbeschränkt)·

We use GraphQL for the communication between our Minecraft-Proxies/Load-Balancers and our global Minecraft-Orchestration-Service JCOverseer.

This connection proved to be especially challenging, as there were so many available options and very specific requirements and we tried our hardest to put as little complexity into this interface as possible.

Initially we considered designing our very own Netty based Packet-Protocol. While the performance of this approach probably would've been noteworthy, we would have had to write a lot of packets as the individual payloads would differ a lot and for the protocol specification a new project would've been needed, so we scrapped that idea.

Our second idea was to use a combination of Redis Key/Value store (in particular the ability to write whole, complex sets as the values of keys) for existing data, Redis Pub-Sub for the synchronization of new/changed/deleted data and a Vert.x based REST API for the mutation requests of the clients. While this would certainly have been possible, we decided against it, as redis offers no real other data types than strings and typing was important to us.

So we finally settled for GraphQL as it would allow us to define dynamic queries and mutations and additionally has subscriptions in store, so we would only need one component instead of three separate. The proxies register as subscribers to the server changes channel and fetch the current data set in advance. If they need to request changes, this is done through a mutation in GraphQL aswell.

The status of the invidiual servers is fetched through Docker healthchecks and a Docker client in the orchestration service, that subscribes to changed HEALTHINESS values in docker. If a service becomes unhealthy it is unregistered and synchronized through GraphQL. The healthcheck is comparable to a ping packet that expects a response in a given time frame.

READ MORE
5 upvotes·64.6K views
Chief Technology Officer at Axceligent Solutions·
Needs advice
on
Jenkins
and
GoCD

I'm open to anything. just want something that break less and doesn't need me to pay for it, and can be hosted on Docker. our scripting language is powershell core. so it's better to support it. also we are building dotnet core in our pipeline, so if they have anything related that helps with the CI would be nice.

READ MORE
3 upvotes·227.7K views
Replies (1)
Software Developer at CloudCover·

Google cloud build can help you. It is hosted on cloud and also provide reasonable free quota.

READ MORE
1 upvote·211.9K views
Senior Fullstack Developer at QUANTUSflow Software GmbH·

Our whole DevOps stack consists of the following tools:

  • GitHub (incl. GitHub Pages/Markdown for Documentation, GettingStarted and HowTo's) for collaborative review and code management tool
  • Respectively Git as revision control system
  • SourceTree as Git GUI
  • Visual Studio Code as IDE
  • CircleCI for continuous integration (automatize development process)
  • Prettier / TSLint / ESLint as code linter
  • SonarQube as quality gate
  • Docker as container management (incl. Docker Compose for multi-container application management)
  • VirtualBox for operating system simulation tests
  • Kubernetes as cluster management for docker containers
  • Heroku for deploying in test environments
  • nginx as web server (preferably used as facade server in production environment)
  • SSLMate (using OpenSSL) for certificate management
  • Amazon EC2 (incl. Amazon S3) for deploying in stage (production-like) and production environments
  • PostgreSQL as preferred database system
  • Redis as preferred in-memory database/store (great for caching)

The main reason we have chosen Kubernetes over Docker Swarm is related to the following artifacts:

  • Key features: Easy and flexible installation, Clear dashboard, Great scaling operations, Monitoring is an integral part, Great load balancing concepts, Monitors the condition and ensures compensation in the event of failure.
  • Applications: An application can be deployed using a combination of pods, deployments, and services (or micro-services).
  • Functionality: Kubernetes as a complex installation and setup process, but it not as limited as Docker Swarm.
  • Monitoring: It supports multiple versions of logging and monitoring when the services are deployed within the cluster (Elasticsearch/Kibana (ELK), Heapster/Grafana, Sysdig cloud integration).
  • Scalability: All-in-one framework for distributed systems.
  • Other Benefits: Kubernetes is backed by the Cloud Native Computing Foundation (CNCF), huge community among container orchestration tools, it is an open source and modular tool that works with any OS.
READ MORE
28 upvotes·2 comments·3.1M views
Larry Gryziak
Larry Gryziak
·
April 30th 2020 at 6:34PM

So why is your deployment different for your (Heroku) test/dev and your stage/production?

·
Reply
Simon Reymann
Simon Reymann
·
May 1st 2020 at 10:32AM

When it comes to testing our web app we do not demand great computational resources and need a very simple, convenient and fast PaaS solution for deploying the app to our testers. In production though, the demand of great computational resources can rise very fast. With Amazon we are able to control that in better way.

·
Reply
Sr. Business Analyst_PMO at IT·

Hey!

We are planning to migrate web application with the same UI AngularJS to #AWS cloud with AWS Elastic Load Balancing (ELB), Jenkins, Docker stack, to check its performance for 200 users. Any suggestions for alternative technologies?

What is the infra required?

Thanks in advance.

READ MORE
2 upvotes·18.7K views
Software Engineer at Amazon·

I only know Java and so thinking of building a web application in the following order. I need some help on what alternatives I can choose. Open to replace components, services, or infrastructure.

  • Frontend: AngularJS, Bootstrap
  • Web Framework: Spring Boot
  • Database: Amazon DynamoDB
  • Authentication: Auth0
  • Deployment: Amazon EC2 Container Service
  • Local Testing: Docker
  • Marketing: Mailchimp (Separately Export from Auth0)
  • Website Domain: GoDaddy
  • Routing: Amazon Route 53

PS: Open to exploring options of going completely native ( AWS Lambda, AWS Security but have to learn all)

READ MORE
6 upvotes·106.3K views
Replies (2)
Founder and CEO at Facile Technolab Pvt Ltd·

I would recommend to upgrade your stack and consider Angular.

Also, if you are working with docker, instead of manually managing your EC2 and docker inside it, switch to ECS as its free of cost and hassle free way to deploy and keep running your containers efficiently.

Good luck.

READ MORE
4 upvotes·7.1K views
Recommends
Ambassador

Instead of Docker , no doubt its great but it has vulnerabilitis and restricitions with dameon and root thread. I would pickup Podman. Also Ambasador is a culmination of Gateway LB and ServiceMesh on istio and Envoy. Great for both East-west and North south microservices communication, policy managment and security with Istio. Spring Boot is not a WebFW. For platform web fw one can use Reactive like SPring WebFlow rather than Spring MVC. For java experience, Spring provides great assets.

I will switch to using Kubernetes whether managed or custom depends on several factors rather than AWS ecs. For LB Amabassador is a great alternative on AWS. One can simply use this on top of ECS clusters. Instead of running in to different frameworks one can simply use one FW at both client and server side for consuming and SSE. I believe one can look at Lot of it depends what you need a full FW or a light librarry like React to be part of V in your MVC. Whether you need a SPA , on Mobile etc... in that case KOTLIN is also another option on Java. Dont go with Android. Best luck. Swapnil S

READ MORE
3 upvotes·6K views

We would like to connect a number of (about 25) video streams, from an Amazon S3 bucket containing video data to endpoints accessible to a Docker image, which, when run, will process the input video streams and emit some JSON statistics.

The 25 video streams should be synchronized. Could people share their experiences with a similar scenario and perhaps offer advice about which is better (Wowza, Amazon Kinesis Video Streams) for this kind of problem, or why they chose one technology over the other?

The video stream duration will be quite long (about 8 hours each x 25 camera sources). The 25 video streams will have no audio component. If you worked with a similar problem, what was your experience with scaling, latency, resource requirements, config, etc.?

READ MORE
6 upvotes·50.2K views
Replies (1)
Software Architect at Payoneer·
Recommends
AWS Lambda

I have different experience with processing video files that I'll describe below. It might be helpful or at least make you think a bit diffferent about the problem. What I did (part of it is a mistake): To increase the level of parallelism at the time consuming step which was the video upload, using a custom cmd tool written in Python, I splitted the input videos to much smaller chunks (without losing their ordering - just file name labeling with timestamp) . It then uploaded the chunks to S3. That triggered a few Lambdas that each first pulled a chunked video, did the processing with ffmpeg (the Lambdas were the mistake - at that time the local Lambda storage was up to 512MB so lots of chunks and lots of Lambdas had to be in place, also Lambda are hell to debug), later called Rekognition and later using AWS Elemental MediaConvert to rebuild the full length video. I would use some sort of ECS deployment where processing is triggered by S3 event, and scale the number of Fargate nodes dependent on the number of chucks/videos. Then each processor pulls its video (not stream) to its local storage (local EBS drive) and works. I failed to understand why are you trying to stream videos that are basically static, as a file, or that putting the files on S3 is a current limitation (while your input videos are 'live' and streaming) that you're trying to remove ?

READ MORE
6 upvotes·5.5K views