Need advice about which tool to choose?Ask the StackShare community!
Cassandra vs Google BigQuery: What are the differences?
Cassandra: A partitioned row store. Rows are organized into tables with a required primary key. Partitioning means that Cassandra can distribute your data across multiple machines in an application-transparent matter. Cassandra will automatically repartition as machines are added and removed from the cluster. Row store means that like relational databases, Cassandra organizes data by rows and columns. The Cassandra Query Language (CQL) is a close relative of SQL; Google BigQuery: Analyze terabytes of data in seconds. Run super-fast, SQL-like queries against terabytes of data in seconds, using the processing power of Google's infrastructure Load data with ease. Bulk load your data using Google Cloud Storage or stream it in. Easy access. Access BigQuery by using a browser tool, a command-line tool, or by making calls to the BigQuery REST API with client libraries such as Java, PHP or Python..
Cassandra and Google BigQuery are primarily classified as "Databases" and "Big Data as a Service" tools respectively.
"Distributed" is the primary reason why developers consider Cassandra over the competitors, whereas "High Performance" was stated as the key factor in picking Google BigQuery.
Cassandra is an open source tool with 5.27K GitHub stars and 2.35K GitHub forks. Here's a link to Cassandra's open source repository on GitHub.
According to the StackShare community, Cassandra has a broader approval, being mentioned in 342 company stacks & 240 developers stacks; compared to Google BigQuery, which is listed in 160 company stacks and 41 developer stacks.
The problem I have is - we need to process & change(update/insert) 55M Data every 2 min and this updated data to be available for Rest API for Filtering / Selection. Response time for Rest API should be less than 1 sec.
The most important factors for me are processing and storing time of 2 min. There need to be 2 views of Data One is for Selection & 2. Changed data.
Cassandra is quite capable of the task, in a highly available way, given appropriate scaling of the system. Remember that updates are only inserts, and that efficient retrieval is only by key (which can be a complex key). Talking of keys, make sure that the keys are well distributed.
Scylla can handle 1M/s events with a simple data model quite easily. The api to query is CQL, we have REST api but that's for control/monitoring
i love syclla for pet projects however it's license which is based on server model is an issue. thus i recommend cassandra
By 55M do you mean 55 million entity changes per 2 minutes? It is relatively high, means almost 460k per second. If I had to choose between Scylla or Cassandra, I would opt for Scylla as it is promising better performance for simple operations. However, maybe it would be worth to consider yet another alternative technology. Take into consideration required consistency, reliability and high availability and you may realize that there are more suitable once. Rest API should not be the main driver, because you can always develop the API yourself, if not supported by given technology.
Cloud Data-warehouse is the centerpiece of modern Data platform. The choice of the most suitable solution is therefore fundamental.
Our benchmark was conducted over BigQuery and Snowflake. These solutions seem to match our goals but they have very different approaches.
BigQuery is notably the only 100% serverless cloud data-warehouse, which requires absolutely NO maintenance: no re-clustering, no compression, no index optimization, no storage management, no performance management. Snowflake requires to set up (paid) reclustering processes, to manage the performance allocated to each profile, etc. We can also mention Redshift, which we have eliminated because this technology requires even more ops operation.
BigQuery can therefore be set up with almost zero cost of human resources. Its on-demand pricing is particularly adapted to small workloads. 0 cost when the solution is not used, only pay for the query you're running. But quickly the use of slots (with monthly or per-minute commitment) will drastically reduce the cost of use. We've reduced by 10 the cost of our nightly batches by using flex slots.
Finally, a major advantage of BigQuery is its almost perfect integration with Google Cloud Platform services: Cloud functions, Dataflow, Data Studio, etc.
BigQuery is still evolving very quickly. The next milestone, BigQuery Omni, will allow to run queries over data stored in an external Cloud platform (Amazon S3 for example). It will be a major breakthrough in the history of cloud data-warehouses. Omni will compensate a weakness of BigQuery: transferring data in near real time from S3 to BQ is not easy today. It was even simpler to implement via Snowflake's Snowpipe solution.
We also plan to use the Machine Learning features built into BigQuery to accelerate our deployment of Data-Science-based projects. An opportunity only offered by the BigQuery solution
Pros of Cassandra
- Distributed118
- High performance97
- High availability81
- Easy scalability74
- Replication52
- Reliable26
- Multi datacenter deployments26
- Schema optional10
- OLTP9
- Open source8
- Workload separation (via MDC)2
- Fast1
Pros of Google BigQuery
- High Performance28
- Easy to use25
- Fully managed service21
- Cheap Pricing19
- Process hundreds of GB in seconds16
- Full table scans in seconds, no indexes needed11
- Big Data11
- Always on, no per-hour costs8
- Good combination with fluentd6
- Machine learning4
Sign up to add or upvote prosMake informed product decisions
Cons of Cassandra
- Reliability of replication3
- Size1
- Updates1
Cons of Google BigQuery
- You can't unit test changes in BQ data1
Sign up to add or upvote consMake informed product decisions
What is Cassandra?
What is Google BigQuery?
Need advice about which tool to choose?Ask the StackShare community!
What companies use Google BigQuery?
Sign up to get full access to all the companiesMake informed product decisions
Sign up to get full access to all the tool integrationsMake informed product decisions
Blog Posts
+5
+17
+16
+25+42
+47
+31+39