What is ceph and what are its top alternatives?
Ceph is a distributed storage system known for its scalability and reliability. It is designed to provide object, block, and file storage in a unified system. Key features of Ceph include self-healing capabilities, efficient data distribution, and the ability to run on commodity hardware. However, some limitations of Ceph include complexity in setting up and maintaining the system, as well as potential performance issues under certain workloads.
- GlusterFS: GlusterFS is an open-source distributed file system that can scale out to multiple petabytes. It is known for its high availability and flexible deployment options. Pros of GlusterFS include easy installation and integration with various applications, but it may not perform as well as Ceph in certain scenarios.
- Swift: Swift is an open-source object storage system that offers scalability and durability. It is a part of the OpenStack project and is known for its simplicity and robustness. However, Swift may lack some of the advanced features found in Ceph.
- MinIO: MinIO is an open-source object storage server that is compatible with Amazon S3 APIs. It is designed for high performance and scalability, making it a popular choice for cloud-native applications. Pros of MinIO include its ease of deployment and use, but it may not offer the same level of data redundancy as Ceph.
- LizardFS: LizardFS is a distributed file system that provides high scalability and reliability. It is designed to be easy to manage and can handle large amounts of data. However, LizardFS may not have the same level of community support as Ceph.
- HDFS: Hadoop Distributed File System (HDFS) is a distributed file system that is part of the Apache Hadoop project. It is known for its reliability and fault tolerance, making it suitable for big data applications. However, HDFS may not offer the same level of flexibility as Ceph.
- QuantumFS: QuantumFS is an open-source distributed file system that is designed for high performance and scalability. It offers features such as snapshotting and data deduplication. Pros of QuantumFS include its performance and scalability, but it may lack some of the advanced features found in Ceph.
- BeeGFS: BeeGFS is a parallel file system that is designed for high-performance computing and data-intensive workloads. It offers features such as automatic failover and data replication. However, BeeGFS may not be as easy to set up and configure as Ceph.
- MooseFS: MooseFS is a distributed file system that is designed for scalability and reliability. It offers features such as data replication and snapshotting. Pros of MooseFS include its ease of use and management, but it may not offer the same level of performance as Ceph.
- XtreemFS: XtreemFS is an open-source distributed file system that is designed for cloud environments. It offers features such as replication and encryption. However, XtreemFS may not have the same level of adoption and community support as Ceph.
- IPFS: InterPlanetary File System (IPFS) is a peer-to-peer distributed file system that aims to create a more resilient and decentralized web. It offers features such as content addressing and data versioning. Pros of IPFS include its decentralization and data integrity, but it may not be suitable for all use cases compared to Ceph.
Top Alternatives to ceph
- Minio
Minio is an object storage server compatible with Amazon S3 and licensed under Apache 2.0 License ...
- Swift
Writing code is interactive and fun, the syntax is concise yet expressive, and apps run lightning-fast. Swift is ready for your next iOS and OS X project — or for addition into your current app — because Swift code works side-by-side with Objective-C. ...
- FreeNAS
It is the simplest way to create a centralized and easily accessible place for your data. Use it with ZFS to protect, store, backup, all of your data. It is used everywhere, for the home, small business, and the enterprise. ...
- Portworx
It is the cloud native storage company that enterprises depend on to reduce the cost and complexity of rapidly deploying containerized applications across multiple clouds and on-prem environments. ...
- Hadoop
The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage. ...
- JavaScript
JavaScript is most known as the scripting language for Web pages, but used in many non-browser environments as well such as node.js or Apache CouchDB. It is a prototype-based, multi-paradigm scripting language that is dynamic,and supports object-oriented, imperative, and functional programming styles. ...
- Git
Git is a free and open source distributed version control system designed to handle everything from small to very large projects with speed and efficiency. ...
- GitHub
GitHub is the best place to share code with friends, co-workers, classmates, and complete strangers. Over three million people use GitHub to build amazing things together. ...
ceph alternatives & related posts
- Store and Serve Resumes & Job Description PDF, Backups10
- S3 Compatible8
- Simple4
- Open Source4
- Encryption and Tamper-Proof3
- Lambda Compute3
- Private Cloud Storage2
- Pluggable Storage Backend2
- Scalable2
- Data Protection2
- Highly Available2
- Performance1
- Deletion of huge buckets is not possible3
related Minio posts
I need to decide between Minio and MongoDB . The main features that I want to evaluate are :
- speed of save and retrieve documents;
- simply to develope of software for they use;
- costs;
Swift
- Ios259
- Elegant180
- Not Objective-C126
- Backed by apple107
- Type inference93
- Generics61
- Playgrounds54
- Semicolon free49
- OSX38
- Tuples offer compound variables36
- Clean Syntax24
- Easy to learn24
- Open Source22
- Beautiful Code21
- Functional20
- Dynamic12
- Linux12
- Protocol-oriented programming11
- Promotes safe, readable code10
- No S-l-o-w JVM9
- Explicit optionals8
- Storyboard designer7
- Optionals6
- Type safety6
- Super addicting language, great people, open, elegant5
- Best UI concept5
- Its friendly4
- Highly Readable codes4
- Fail-safe4
- Powerful4
- Faster and looks better4
- Swift is faster than Objective-C4
- Feels like a better C++4
- Easy to learn and work3
- Much more fun3
- Protocol extensions3
- Native3
- Its fun and damn fast3
- Strong Type safety3
- Easy to Maintain3
- Protocol as type2
- All Cons C# and Java Swift Already has2
- Esay2
- MacOS2
- Type Safe2
- Protocol oriented programming2
- Can interface with C easily1
- Actually don't have to own a mac1
- Free from Memory Leak1
- Swift is easier to understand for non-iOS developers.1
- Numbers with underbar1
- Optional chain1
- Great for Multi-Threaded Programming1
- Runs Python 8 times faster1
- Objec1
- Must own a mac5
- Memory leaks are not uncommon2
- Very irritatingly picky about things that’s1
- Complicated process for exporting modules1
- Its classes compile to roughly 300 lines of assembly1
- Is a lot more effort than lua to make simple functions1
- Overly complex options makes it easy to create bad code0
related Swift posts
Hi Community! Trust everyone is keeping safe. I am exploring the idea of building a #Neobank (App) with end-to-end banking capabilities. In the process of exploring this space, I have come across multiple Apps (N26, Revolut, Monese, etc) and explored their stacks in detail. The confusion remains to be the Backend Tech to be used?
What would you go with considering all of the languages such as Node.js Java Rails Python are suggested by some person or the other. As a general trend, I have noticed the usage of Node with React on the front or Node with a combination of Kotlin and Swift. Please suggest what would be the right approach!
Excerpts from how we developed (and subsequently open sourced) Uber's cross-platform mobile architecture framework, RIBs , going from Objective-C to Swift in the process for iOS: https://github.com/uber/RIBs
Uber’s new application architecture (RIBs) extensively uses protocols to keep its various components decoupled and testable. We used this architecture for the first time in our new rider application and moved our primary language from Objective-C to Swift. Since Swift is a very static language, unit testing became problematic. Dynamic languages have good frameworks to build test mocks, stubs, or stand-ins by dynamically creating or modifying existing concrete classes.
Needless to say, we were not very excited about the additional complexity of manually writing and maintaining mock implementations for each of our thousands of protocols.
The information required to generate mock classes already exists in the Swift protocol. For Uber’s use case, we set out to create tooling that would let engineers automatically generate test mocks for any protocol they wanted by simply annotating them.
The iOS codebase for our rider application alone incorporates around 1,500 of these generated mocks. Without our code generation tool, all of these would have to be written and maintained by hand, which would have made testing much more time-intensive. Auto-generated mocks have contributed a lot to the unit test coverage that we have today.
We built these code generation tools ourselves for a number of reasons, including that there weren’t many open source tools available at the time we started our effort. Today, there are some great open source tools to generate resource accessors, like SwiftGen. And Sourcery can help you with generic code generation needs:
https://eng.uber.com/code-generation/ https://eng.uber.com/driver-app-ribs-architecture/
(GitHub : https://github.com/uber/RIBs )
- Very Stable2
- Easy to install2
related FreeNAS posts
Portworx
related Portworx posts
- Great ecosystem39
- One stack to rule them all11
- Great load balancer4
- Amazon aws1
- Java syntax1
related Hadoop posts
The early data ingestion pipeline at Pinterest used Kafka as the central message transporter, with the app servers writing messages directly to Kafka, which then uploaded log files to S3.
For databases, a custom Hadoop streamer pulled database data and wrote it to S3.
Challenges cited for this infrastructure included high operational overhead, as well as potential data loss occurring when Kafka broker outages led to an overflow of in-memory message buffering.
Why we built Marmaray, an open source generic data ingestion and dispersal framework and library for Apache Hadoop :
Built and designed by our Hadoop Platform team, Marmaray is a plug-in-based framework built on top of the Hadoop ecosystem. Users can add support to ingest data from any source and disperse to any sink leveraging the use of Apache Spark . The name, Marmaray, comes from a tunnel in Turkey connecting Europe and Asia. Similarly, we envisioned Marmaray within Uber as a pipeline connecting data from any source to any sink depending on customer preference:
https://eng.uber.com/marmaray-hadoop-ingestion-open-source/
(Direct GitHub repo: https://github.com/uber/marmaray Kafka Kafka Manager )
JavaScript
- Can be used on frontend/backend1.7K
- It's everywhere1.5K
- Lots of great frameworks1.2K
- Fast896
- Light weight745
- Flexible425
- You can't get a device today that doesn't run js392
- Non-blocking i/o286
- Ubiquitousness236
- Expressive191
- Extended functionality to web pages55
- Relatively easy language49
- Executed on the client side46
- Relatively fast to the end user30
- Pure Javascript25
- Functional programming21
- Async15
- Full-stack13
- Setup is easy12
- Its everywhere12
- JavaScript is the New PHP11
- Because I love functions11
- Like it or not, JS is part of the web standard10
- Can be used in backend, frontend and DB9
- Expansive community9
- Future Language of The Web9
- Easy9
- No need to use PHP8
- For the good parts8
- Can be used both as frontend and backend as well8
- Everyone use it8
- Most Popular Language in the World8
- Easy to hire developers8
- Love-hate relationship7
- Powerful7
- Photoshop has 3 JS runtimes built in7
- Evolution of C7
- Popularized Class-Less Architecture & Lambdas7
- Agile, packages simple to use7
- Supports lambdas and closures7
- 1.6K Can be used on frontend/backend6
- It's fun6
- Hard not to use6
- Nice6
- Client side JS uses the visitors CPU to save Server Res6
- Versitile6
- It let's me use Babel & Typescript6
- Easy to make something6
- Its fun and fast6
- Can be used on frontend/backend/Mobile/create PRO Ui6
- Function expressions are useful for callbacks5
- What to add5
- Client processing5
- Everywhere5
- Scope manipulation5
- Stockholm Syndrome5
- Promise relationship5
- Clojurescript5
- Because it is so simple and lightweight4
- Only Programming language on browser4
- Hard to learn1
- Test1
- Test21
- Easy to understand1
- Not the best1
- Easy to learn1
- Subskill #41
- Hard 彤0
- A constant moving target, too much churn22
- Horribly inconsistent20
- Javascript is the New PHP15
- No ability to monitor memory utilitization9
- Shows Zero output in case of ANY error8
- Thinks strange results are better than errors7
- Can be ugly6
- No GitHub3
- Slow2
related JavaScript posts
Oof. I have truly hated JavaScript for a long time. Like, for over twenty years now. Like, since the Clinton administration. It's always been a nightmare to deal with all of the aspects of that silly language.
But wowza, things have changed. Tooling is just way, way better. I'm primarily web-oriented, and using React and Apollo together the past few years really opened my eyes to building rich apps. And I deeply apologize for using the phrase rich apps; I don't think I've ever said such Enterprisey words before.
But yeah, things are different now. I still love Rails, and still use it for a lot of apps I build. But it's that silly rich apps phrase that's the problem. Users have way more comprehensive expectations than they did even five years ago, and the JS community does a good job at building tools and tech that tackle the problems of making heavy, complicated UI and frontend work.
Obviously there's a lot of things happening here, so just saying "JavaScript isn't terrible" might encompass a huge amount of libraries and frameworks. But if you're like me, yeah, give things another shot- I'm somehow not hating on JavaScript anymore and... gulp... I kinda love it.
How Uber developed the open source, end-to-end distributed tracing Jaeger , now a CNCF project:
Distributed tracing is quickly becoming a must-have component in the tools that organizations use to monitor their complex, microservice-based architectures. At Uber, our open source distributed tracing system Jaeger saw large-scale internal adoption throughout 2016, integrated into hundreds of microservices and now recording thousands of traces every second.
Here is the story of how we got here, from investigating off-the-shelf solutions like Zipkin, to why we switched from pull to push architecture, and how distributed tracing will continue to evolve:
https://eng.uber.com/distributed-tracing/
(GitHub Pages : https://www.jaegertracing.io/, GitHub: https://github.com/jaegertracing/jaeger)
Bindings/Operator: Python Java Node.js Go C++ Kubernetes JavaScript OpenShift C# Apache Spark
- Distributed version control system1.4K
- Efficient branching and merging1.1K
- Fast959
- Open source845
- Better than svn726
- Great command-line application368
- Simple306
- Free291
- Easy to use232
- Does not require server222
- Distributed27
- Small & Fast22
- Feature based workflow18
- Staging Area15
- Most wide-spread VSC13
- Role-based codelines11
- Disposable Experimentation11
- Frictionless Context Switching7
- Data Assurance6
- Efficient5
- Just awesome4
- Github integration3
- Easy branching and merging3
- Compatible2
- Flexible2
- Possible to lose history and commits2
- Rebase supported natively; reflog; access to plumbing1
- Light1
- Team Integration1
- Fast, scalable, distributed revision control system1
- Easy1
- Flexible, easy, Safe, and fast1
- CLI is great, but the GUI tools are awesome1
- It's what you do1
- Phinx0
- Hard to learn16
- Inconsistent command line interface11
- Easy to lose uncommitted work9
- Worst documentation ever possibly made7
- Awful merge handling5
- Unexistent preventive security flows3
- Rebase hell3
- When --force is disabled, cannot rebase2
- Ironically even die-hard supporters screw up badly2
- Doesn't scale for big data1
related Git posts
Our whole DevOps stack consists of the following tools:
- GitHub (incl. GitHub Pages/Markdown for Documentation, GettingStarted and HowTo's) for collaborative review and code management tool
- Respectively Git as revision control system
- SourceTree as Git GUI
- Visual Studio Code as IDE
- CircleCI for continuous integration (automatize development process)
- Prettier / TSLint / ESLint as code linter
- SonarQube as quality gate
- Docker as container management (incl. Docker Compose for multi-container application management)
- VirtualBox for operating system simulation tests
- Kubernetes as cluster management for docker containers
- Heroku for deploying in test environments
- nginx as web server (preferably used as facade server in production environment)
- SSLMate (using OpenSSL) for certificate management
- Amazon EC2 (incl. Amazon S3) for deploying in stage (production-like) and production environments
- PostgreSQL as preferred database system
- Redis as preferred in-memory database/store (great for caching)
The main reason we have chosen Kubernetes over Docker Swarm is related to the following artifacts:
- Key features: Easy and flexible installation, Clear dashboard, Great scaling operations, Monitoring is an integral part, Great load balancing concepts, Monitors the condition and ensures compensation in the event of failure.
- Applications: An application can be deployed using a combination of pods, deployments, and services (or micro-services).
- Functionality: Kubernetes as a complex installation and setup process, but it not as limited as Docker Swarm.
- Monitoring: It supports multiple versions of logging and monitoring when the services are deployed within the cluster (Elasticsearch/Kibana (ELK), Heapster/Grafana, Sysdig cloud integration).
- Scalability: All-in-one framework for distributed systems.
- Other Benefits: Kubernetes is backed by the Cloud Native Computing Foundation (CNCF), huge community among container orchestration tools, it is an open source and modular tool that works with any OS.
Often enough I have to explain my way of going about setting up a CI/CD pipeline with multiple deployment platforms. Since I am a bit tired of yapping the same every single time, I've decided to write it up and share with the world this way, and send people to read it instead ;). I will explain it on "live-example" of how the Rome got built, basing that current methodology exists only of readme.md and wishes of good luck (as it usually is ;)).
It always starts with an app, whatever it may be and reading the readmes available while Vagrant and VirtualBox is installing and updating. Following that is the first hurdle to go over - convert all the instruction/scripts into Ansible playbook(s), and only stopping when doing a clear vagrant up
or vagrant reload
we will have a fully working environment. As our Vagrant environment is now functional, it's time to break it! This is the moment to look for how things can be done better (too rigid/too lose versioning? Sloppy environment setup?) and replace them with the right way to do stuff, one that won't bite us in the backside. This is the point, and the best opportunity, to upcycle the existing way of doing dev environment to produce a proper, production-grade product.
I should probably digress here for a moment and explain why. I firmly believe that the way you deploy production is the same way you should deploy develop, shy of few debugging-friendly setting. This way you avoid the discrepancy between how production work vs how development works, which almost always causes major pains in the back of the neck, and with use of proper tools should mean no more work for the developers. That's why we start with Vagrant as developer boxes should be as easy as vagrant up
, but the meat of our product lies in Ansible which will do meat of the work and can be applied to almost anything: AWS, bare metal, docker, LXC, in open net, behind vpn - you name it.
We must also give proper consideration to monitoring and logging hoovering at this point. My generic answer here is to grab Elasticsearch, Kibana, and Logstash. While for different use cases there may be better solutions, this one is well battle-tested, performs reasonably and is very easy to scale both vertically (within some limits) and horizontally. Logstash rules are easy to write and are well supported in maintenance through Ansible, which as I've mentioned earlier, are at the very core of things, and creating triggers/reports and alerts based on Elastic and Kibana is generally a breeze, including some quite complex aggregations.
If we are happy with the state of the Ansible it's time to move on and put all those roles and playbooks to work. Namely, we need something to manage our CI/CD pipelines. For me, the choice is obvious: TeamCity. It's modern, robust and unlike most of the light-weight alternatives, it's transparent. What I mean by that is that it doesn't tell you how to do things, doesn't limit your ways to deploy, or test, or package for that matter. Instead, it provides a developer-friendly and rich playground for your pipelines. You can do most the same with Jenkins, but it has a quite dated look and feel to it, while also missing some key functionality that must be brought in via plugins (like quality REST API which comes built-in with TeamCity). It also comes with all the common-handy plugins like Slack or Apache Maven integration.
The exact flow between CI and CD varies too greatly from one application to another to describe, so I will outline a few rules that guide me in it: 1. Make build steps as small as possible. This way when something breaks, we know exactly where, without needing to dig and root around. 2. All security credentials besides development environment must be sources from individual Vault instances. Keys to those containers should exist only on the CI/CD box and accessible by a few people (the less the better). This is pretty self-explanatory, as anything besides dev may contain sensitive data and, at times, be public-facing. Because of that appropriate security must be present. TeamCity shines in this department with excellent secrets-management. 3. Every part of the build chain shall consume and produce artifacts. If it creates nothing, it likely shouldn't be its own build. This way if any issue shows up with any environment or version, all developer has to do it is grab appropriate artifacts to reproduce the issue locally. 4. Deployment builds should be directly tied to specific Git branches/tags. This enables much easier tracking of what caused an issue, including automated identifying and tagging the author (nothing like automated regression testing!).
Speaking of deployments, I generally try to keep it simple but also with a close eye on the wallet. Because of that, I am more than happy with AWS or another cloud provider, but also constantly peeking at the loads and do we get the value of what we are paying for. Often enough the pattern of use is not constantly erratic, but rather has a firm baseline which could be migrated away from the cloud and into bare metal boxes. That is another part where this approach strongly triumphs over the common Docker and CircleCI setup, where you are very much tied in to use cloud providers and getting out is expensive. Here to embrace bare-metal hosting all you need is a help of some container-based self-hosting software, my personal preference is with Proxmox and LXC. Following that all you must write are ansible scripts to manage hardware of Proxmox, similar way as you do for Amazon EC2 (ansible supports both greatly) and you are good to go. One does not exclude another, quite the opposite, as they can live in great synergy and cut your costs dramatically (the heavier your base load, the bigger the savings) while providing production-grade resiliency.
GitHub
- Open source friendly1.8K
- Easy source control1.5K
- Nice UI1.3K
- Great for team collaboration1.1K
- Easy setup867
- Issue tracker504
- Great community486
- Remote team collaboration482
- Great way to share451
- Pull request and features planning442
- Just works147
- Integrated in many tools132
- Free Public Repos121
- Github Gists116
- Github pages112
- Easy to find repos83
- Open source62
- It's free60
- Easy to find projects60
- Network effect56
- Extensive API49
- Organizations43
- Branching42
- Developer Profiles34
- Git Powered Wikis32
- Great for collaboration30
- It's fun24
- Clean interface and good integrations23
- Community SDK involvement22
- Learn from others source code20
- Because: Git16
- It integrates directly with Azure14
- Newsfeed10
- Standard in Open Source collab10
- Fast8
- It integrates directly with Hipchat8
- Beautiful user experience8
- Easy to discover new code libraries7
- Smooth integration6
- Cloud SCM6
- Nice API6
- Graphs6
- Integrations6
- It's awesome6
- Quick Onboarding5
- Remarkable uptime5
- CI Integration5
- Hands down best online Git service available5
- Reliable5
- Free HTML hosting4
- Version Control4
- Simple but powerful4
- Unlimited Public Repos at no cost4
- Security options4
- Loved by developers4
- Uses GIT4
- Easy to use and collaborate with others4
- IAM3
- Nice to use3
- Ci3
- Easy deployment via SSH3
- Good tools support2
- Leads the copycats2
- Free private repos2
- Free HTML hostings2
- Easy and efficient maintainance of the projects2
- Beautiful2
- Never dethroned2
- IAM integration2
- Very Easy to Use2
- Easy to use2
- All in one development service2
- Self Hosted2
- Issues tracker2
- Easy source control and everything is backed up2
- Profound1
- Owned by micrcosoft53
- Expensive for lone developers that want private repos37
- Relatively slow product/feature release cadence15
- API scoping could be better10
- Only 3 collaborators for private repos8
- Limited featureset for issue management3
- GitHub Packages does not support SNAPSHOT versions2
- Does not have a graph for showing history like git lens2
- No multilingual interface1
- Takes a long time to commit1
- Expensive1
related GitHub posts
I was building a personal project that I needed to store items in a real time database. I am more comfortable with my Frontend skills than my backend so I didn't want to spend time building out anything in Ruby or Go.
I stumbled on Firebase by #Google, and it was really all I needed. It had realtime data, an area for storing file uploads and best of all for the amount of data I needed it was free!
I built out my application using tools I was familiar with, React for the framework, Redux.js to manage my state across components, and styled-components for the styling.
Now as this was a project I was just working on in my free time for fun I didn't really want to pay for hosting. I did some research and I found Netlify. I had actually seen them at #ReactRally the year before and deployed a Gatsby site to Netlify already.
Netlify was very easy to setup and link to my GitHub account you select a repo and pretty much with very little configuration you have a live site that will deploy every time you push to master.
With the selection of these tools I was able to build out my application, connect it to a realtime database, and deploy to a live environment all with $0 spent.
If you're looking to build out a small app I suggest giving these tools a go as you can get your idea out into the real world for absolutely no cost.
StackShare Feed is built entirely with React, Glamorous, and Apollo. One of our objectives with the public launch of the Feed was to enable a Server-side rendered (SSR) experience for our organic search traffic. When you visit the StackShare Feed, and you aren't logged in, you are delivered the Trending feed experience. We use an in-house Node.js rendering microservice to generate this HTML. This microservice needs to run and serve requests independent of our Rails web app. Up until recently, we had a mono-repo with our Rails and React code living happily together and all served from the same web process. In order to deploy our SSR app into a Heroku environment, we needed to split out our front-end application into a separate repo in GitHub. The driving factor in this decision was mostly due to limitations imposed by Heroku specifically with how processes can't communicate with each other. A new SSR app was created in Heroku and linked directly to the frontend repo so it stays in-sync with changes.
Related to this, we need a way to "deploy" our frontend changes to various server environments without building & releasing the entire Ruby application. We built a hybrid Amazon S3 Amazon CloudFront solution to host our Webpack bundles. A new CircleCI script builds the bundles and uploads them to S3. The final step in our rollout is to update some keys in Redis so our Rails app knows which bundles to serve. The result of these efforts were significant. Our frontend team now moves independently of our backend team, our build & release process takes only a few minutes, we are now using an edge CDN to serve JS assets, and we have pre-rendered React pages!
#StackDecisionsLaunch #SSR #Microservices #FrontEndRepoSplit