Need advice about which tool to choose?Ask the StackShare community!
Microsoft SQL Server vs RocksDB: What are the differences?
Introduction
In this analysis, we will outline the key differences between Microsoft SQL Server and RocksDB to better understand their unique capabilities and strengths in database management.
- Storage Model: Microsoft SQL Server utilizes a traditional relational database management system (RDBMS) approach, organizing data into tables with defined relationships. On the other hand, RocksDB is a key-value store that offers high performance and scalability by efficiently storing and retrieving data based on keys.
- Use Cases: SQL Server is primarily designed for handling structured data and complex queries in transactional systems or data warehouses. In contrast, RocksDB is better suited for scenarios that require fast and reliable storage and retrieval of key-value pairs, often used in data caching or real-time analytics applications.
- Replication and Sharding: When it comes to data replication and sharding, SQL Server offers built-in features for maintaining data consistency and distributing data across multiple servers. RocksDB, on the other hand, relies on external tools and frameworks to achieve similar scalability and fault tolerance.
- Programming Interfaces: SQL Server supports SQL queries for data manipulation and retrieval, making it easier for developers familiar with relational databases to work with. RocksDB provides a simpler set of key-value APIs, focusing on performance and low latency for storage operations without the overhead of SQL query processing.
- Data Durability: SQL Server ensures data durability through transaction logging and recovery mechanisms, guaranteeing that committed transactions are saved even in the event of a system failure. RocksDB, being an embedded database, offers similar durability features but may require additional configurations for achieving strong consistency in distributed environments.
- Community Support and Licensing: Microsoft SQL Server has a large community of users and enterprise support options but may require licensing fees for certain editions. RocksDB, being an open-source project by Facebook, benefits from community contributions and continuous improvements without direct licensing costs, making it a cost-effective option for many developers.
In Summary, Microsoft SQL Server and RocksDB differ in their storage models, use cases, replication strategies, programming interfaces, data durability, and community support, catering to distinct database management needs based on performance, scalability, and licensing considerations.
I am researching different querying solutions to handle ~1 trillion records of data (in the realm of a petabyte). The data is mostly textual. I have identified a few options: Milvus, HBase, RocksDB, and Elasticsearch. I was wondering if there is a good way to compare the performance of these options (or if anyone has already done something like this). I want to be able to compare the speed of ingesting and querying textual data from these tools. Does anyone have information on this or know where I can find some? Thanks in advance!
You've probably come to a decision already but for those reading...here are some resources we put together to help people learn more about Milvus and other databases https://zilliz.com/comparison and https://github.com/zilliztech/VectorDBBench. I don't think they include RocksDB or HBase yet (you could could recommend on GitHub) but hopefully they help answer your Elastic Search questions.
I have a project (in production) that a part of it is generating HTML from JSON object normally we use Microsoft SQL Server only as our main database. but when it comes to this part some team members suggest working with a NoSQL database as we are going to handle JSON data for both retrieval and querying. others replied that will add complexity and we will lose SQL Servers' Unit Of Work which will break the Atomic behavior, and they suggest to continue working with SQL Server since it supports working with JSON. If you have practical experience using JSON with SQL Server, kindly share your feedback.
I agree with the advice you have been given to stick with SQL Server. If you are on the latest SQL Server version you can query inside the JSON field. You should set up a test database with a JSON field and try some queries. Once you understand it and can demonstrate it, show it to the other developers that are suggesting MongoDB. Once they see it working with their own eyes they may drop their position of Mongo over SQL. I would only seriously consider MongoDB if there was no other SQL requirements. I wouldn't do both. I'd be all SQL or all Mongo.
I think the key thing to look for is what kind of queries you're expecting to do on that JSON and how stable that data is going to be. (And if you actually need to store the data as JSON; it's generally pretty inexpensive to generate a JSON object)
MongoDB gets rid of the relational aspect of data in favor of data being very fluid in structure.
So if your JSON is going to vary a lot/is unpredictable/will change over time and you need to run queries efficiently like 'records where the field x exists and its value is higher than 3', that's a great use case for MongoDB.
It's hard to solve this in a standard relational model: Indexing on a single column that has wildly different values is pretty much impossible to do efficiently; and pulling out the data in its own columns is hard because it's hard to predict how many columns you'd have or what their datatypes would be. If this sounds like your predicament, 100% go for MongoDB.
If this is always going to be more or less the same JSON and the fields are going to be predictably the same, then the fact that it's JSON doesn't particularly matter much. Your indexes are going to approach it similar to a long string.
If the queried fields are very predictable, you should probably consider storing the fields as separate columns to have better querying capabilities. Ie if you have {"x":1, "y":2}, {"x":5, "y":6}, {"x":9, "y":0} - just make a table with an x and y column and generate the JSON. The CPU hit is worth it compared to the querying capabilities.
I am a Microsoft SQL Server programmer who is a bit out of practice. I have been asked to assist on a new project. The overall purpose is to organize a large number of recordings so that they can be searched. I have an enormous music library but my songs are several hours long. I need to include things like time, date and location of the recording. I don't have a problem with the general database design. I have two primary questions:
- I need to use either MySQL or PostgreSQL on a Linux based OS. Which would be better for this application?
- I have not dealt with a sound based data type before. How do I store that and put it in a table? Thank you.
Hi Erin,
Honestly both databases will do the job just fine. I personally prefer Postgres.
Much more important is how you store the audio. While you could technically use a blob type column, it's really not ideal to be storing audio files which are "several hours long" in a database row. Instead consider storing the audio files in an object store (hosted options include backblaze b2 or aws s3) and persisting the key (which references that object) in your database column.
Hi Erin, Chances are you would want to store the files in a blob type. Both MySQL and Postgres support this. Can you explain a little more about your need to store the files in the database? I may be more effective to store the files on a file system or something like S3. To answer your qustion based on what you are descibing I would slighly lean towards PostgreSQL since it tends to be a little better on the data warehousing side.
Hey Erin! I would recommend checking out Directus before you start work on building your own app for them. I just stumbled upon it, and so far extremely happy with the functionalities. If your client is just looking for a simple web app for their own data, then Directus may be a great option. It offers "database mirroring", so that you can connect it to any database and set up functionality around it!
Hi Erin! First of all, you'd probably want to go with a managed service. Don't spin up your own MySQL installation on your own Linux box. If you are on AWS, thet have different offerings for database services. Standard RDS vs. Aurora. Aurora would be my preferred choice given the benefits it offers, storage optimizations it comes with... etc. Such managed services easily allow you to apply new security patches and upgrades, set up backups, replication... etc. Doing this on your own would either be risky, inefficient, or you might just give up. As far as which database to chose, you'll have the choice between Postgresql, MySQL, Maria DB, SQL Server... etc. I personally would recommend MySQL (latest version available), as the official tooling for it (MySQL Workbench) is great, stable, and moreover free. Other database services exist, I'd recommend you also explore Dynamo DB.
Regardless, you'd certainly only keep high-level records, meta data in Database, and the actual files, most-likely in S3, so that you can keep all options open in terms of what you'll do with them.
Hi Erin,
- Coming from "Big" DB engines, such as Oracle or MSSQL, go for PostgreSQL. You'll get all the features you need with PostgreSQL.
- Your case seems to point to a "NoSQL" or Document Database use case. Since you get covered on this with PostgreSQL which achieves excellent performances on JSON based objects, this is a second reason to choose PostgreSQL. MongoDB might be an excellent option as well if you need "sharding" and excellent map-reduce mechanisms for very massive data sets. You really should investigate the NoSQL option for your use case.
- Starting with AWS Aurora is an excellent advise. since "vendor lock-in" is limited, but I did not check for JSON based object / NoSQL features.
- If you stick to Linux server, the PostgreSQL or MySQL provided with your distribution are straightforward to install (i.e. apt install postgresql). For PostgreSQL, make sure you're comfortable with the pg_hba.conf, especially for IP restrictions & accesses.
Regards,
I recommend Postgres as well. Superior performance overall and a more robust architecture.
Easy to start, lightweight and open source.
When I started with PHP, MySQL was everywhere so this is how I started with it. I am no expert in databases but I started learning joins, stored procedures, triggers, etc. with MySQL.
Recently used it in one of my projects - Picfam.com with Node.js + Express backend
Needed to transform intranet desktop application to the web-based one, as mid-term project. My choice was to use Django/Angular stack - Django since it, in conjunction with Python, enabled rapid development, an Angular since it was stable and enterprise-level framework. Deadlines were somewhat tight since the project to migrate was being developed for several years and had a lot of domain knowledge integrated into it. Definitely was good decision, since deadlines was manageable, juniors were able to enter the project very quickly and we were able to continuously deploy very well.
Pros of Microsoft SQL Server
- Reliable and easy to use139
- High performance101
- Great with .net95
- Works well with .net65
- Easy to maintain56
- Azure support21
- Full Index Support17
- Always on17
- Enterprise manager is fantastic10
- In-Memory OLTP Engine9
- Security is forefront2
- Easy to setup and configure2
- Docker Delivery1
- Columnstore indexes1
- Great documentation1
- Faster Than Oracle1
- Decent management tools1
Pros of RocksDB
- Very fast5
- Made by Facebook3
- Consistent performance2
- Ability to add logic to the database layer where needed1
Sign up to add or upvote prosMake informed product decisions
Cons of Microsoft SQL Server
- Expensive Licensing4
- Microsoft2