Google BigQuery vs InfluxDB

Overview

Google BigQuery

Stacks1.8K

Followers1.5K

Votes152

InfluxDB

Stacks1.0K

Followers1.2K

Votes175

Google BigQuery vs InfluxDB: What are the differences?

Key Differences between Google BigQuery and InfluxDB

Google BigQuery and InfluxDB are two popular database management systems that offer different features and functionalities. Here are the key differences between them:

1. Query Language: Google BigQuery uses SQL-like queries to interact with the data, making it easy for users familiar with SQL to work with the database. InfluxDB, on the other hand, uses its own query language called InfluxQL, which is specifically designed for time-series data and provides additional features like downsampling and aggregation.

2. Data Model: BigQuery stores data in a tabular format, similar to traditional relational databases, which makes it suitable for structured and semi-structured data. InfluxDB, being a time-series database, organizes data in measurements, tags, and fields, enabling efficient storage and retrieval of time-stamped data points.

3. Scalability and Performance: BigQuery is built on Google's infrastructure and is designed to handle large-scale datasets and perform complex analytics queries. It offers high scalability and parallel processing capabilities, making it suitable for big data applications. InfluxDB, on the other hand, is optimized for time-series data and provides high write and query performance for time-based data streams, but may not scale as well for other types of data.

4. Replication and High Availability: BigQuery offers built-in replication and high availability across multiple data centers, ensuring data redundancy and reducing the risk of data loss. InfluxDB provides replication capabilities through its clustering feature, allowing users to distribute the workload and ensure data durability.

5. Integrations and Ecosystem: BigQuery integrates well with other products and services in the Google Cloud ecosystem, such as Data Studio, Cloud Storage, and AI Platform, enabling seamless data analysis and machine learning workflows. InfluxDB also offers various integrations and has a growing ecosystem of plugins and libraries, but its integration options may not be as extensive as BigQuery.

6. Cost Structure: BigQuery follows a pay-as-you-go pricing model based on data storage and processing usage, allowing users to scale their resources based on their needs. InfluxDB is available in both open-source and enterprise editions, with different pricing models depending on the deployment type and features required.

In summary, Google BigQuery and InfluxDB differ in their query language, data model, scalability, replication, integrations, and cost structure, making them suitable for different use cases based on specific requirements.

Share your Stack

Help developers discover the tools you use. Get visibility for your team's tech choices and contribute to the community's knowledge.

View Docs

CLI (Node.js)

Manual

Advice on Google BigQuery, InfluxDB

Anonymous

Apr 21, 2020

Needs advice

We are building an IOT service with heavy write throughput and fewer reads (we need downsampling records). We prefer to have good reliability when comes to data and prefer to have data retention based on policies.

So, we are looking for what is the best underlying DB for ingesting a lot of data and do queries easily

381k views381k

Comments

Julien

CTO at Hawk

Sep 19, 2020

Decided

Cloud Data-warehouse is the centerpiece of modern Data platform. The choice of the most suitable solution is therefore fundamental.

Our benchmark was conducted over BigQuery and Snowflake. These solutions seem to match our goals but they have very different approaches.

BigQuery is notably the only 100% serverless cloud data-warehouse, which requires absolutely NO maintenance: no re-clustering, no compression, no index optimization, no storage management, no performance management. Snowflake requires to set up (paid) reclustering processes, to manage the performance allocated to each profile, etc. We can also mention Redshift, which we have eliminated because this technology requires even more ops operation.

BigQuery can therefore be set up with almost zero cost of human resources. Its on-demand pricing is particularly adapted to small workloads. 0 cost when the solution is not used, only pay for the query you're running. But quickly the use of slots (with monthly or per-minute commitment) will drastically reduce the cost of use. We've reduced by 10 the cost of our nightly batches by using flex slots.

Finally, a major advantage of BigQuery is its almost perfect integration with Google Cloud Platform services: Cloud functions, Dataflow, Data Studio, etc.

BigQuery is still evolving very quickly. The next milestone, BigQuery Omni, will allow to run queries over data stored in an external Cloud platform (Amazon S3 for example). It will be a major breakthrough in the history of cloud data-warehouses. Omni will compensate a weakness of BigQuery: transferring data in near real time from S3 to BQ is not easy today. It was even simpler to implement via Snowflake's Snowpipe solution.

We also plan to use the Machine Learning features built into BigQuery to accelerate our deployment of Data-Science-based projects. An opportunity only offered by the BigQuery solution

193k views193k

Comments

Benoit

Principal Engineer at Sqreen

Sep 21, 2019

Decided

I chose TimescaleDB because to be the backend system of our production monitoring system. We needed to be able to keep track of multiple high cardinality dimensions.

The drawbacks of this decision are our monitoring system is a bit more ad hoc than it used to (New Relic Insights)

We are combining this with Grafana for display and Telegraf for data collection

155k views155k

Comments

Detailed Comparison

Google BigQuery	InfluxDB
Run super-fast, SQL-like queries against terabytes of data in seconds, using the processing power of Google's infrastructure. Load data with ease. Bulk load your data using Google Cloud Storage or stream it in. Easy access. Access BigQuery by using a browser tool, a command-line tool, or by making calls to the BigQuery REST API with client libraries such as Java, PHP or Python.	InfluxDB is a scalable datastore for metrics, events, and real-time analytics. It has a built-in HTTP API so you don't have to write any server side code to get up and running. InfluxDB is designed to be scalable, simple to install and manage, and fast to get data in and out.
All behind the scenes- Your queries can execute asynchronously in the background, and can be polled for status.;Import data with ease- Bulk load your data using Google Cloud Storage or stream it in bursts of up to 1,000 rows per second.;Affordable big data- The first Terabyte of data processed each month is free.;The right interface- Separate interfaces for administration and developers will make sure that you have access to the tools you need.	Time-Centric Functions;Scalable Metrics; Events;Native HTTP API;Powerful Query Language;Built-in Explorer
Statistics
Stacks 1.8K	Stacks 1.0K
Followers 1.5K	Followers 1.2K
Votes 152	Votes 175
Pros & Cons
Pros 28 High Performance 25 Easy to use 22 Fully managed service 19 Cheap Pricing 16 Process hundreds of GB in seconds Cons 1 You can't unit test changes in BQ data 0 Sdas	Pros 59 Time-series data analysis 30 Easy setup, no dependencies 24 Fast, scalable & open source 21 Open source 20 Real-time analytics Cons 4 Instability 1 HA or Clustering is only in paid version 1 Proprietary query language
Integrations
Xplenty Fluentd Looker Chartio Treasure Data	No integrations available

What are some alternatives to Google BigQuery, InfluxDB?

MongoDB

MongoDB stores data in JSON-like documents that can vary in structure, offering a dynamic, flexible schema. MongoDB was also designed for high availability and scalability, with built-in replication and auto-sharding.

MySQL

The MySQL software delivers a very fast, multi-threaded, multi-user, and robust SQL (Structured Query Language) database server. MySQL Server is intended for mission-critical, heavy-load production systems as well as for embedding into mass-deployed software.

PostgreSQL

PostgreSQL is an advanced object-relational database management system that supports an extended subset of the SQL standard, including transactions, foreign keys, subqueries, triggers, user-defined types and functions.

Microsoft SQL Server

Microsoft® SQL Server is a database management and analysis system for e-commerce, line-of-business, and data warehousing solutions.

SQLite

SQLite is an embedded SQL database engine. Unlike most other SQL databases, SQLite does not have a separate server process. SQLite reads and writes directly to ordinary disk files. A complete SQL database with multiple tables, indices, triggers, and views, is contained in a single disk file.

Cassandra

Partitioning means that Cassandra can distribute your data across multiple machines in an application-transparent matter. Cassandra will automatically repartition as machines are added and removed from the cluster. Row store means that like relational databases, Cassandra organizes data by rows and columns. The Cassandra Query Language (CQL) is a close relative of SQL.

Memcached

Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering.

MariaDB

Started by core members of the original MySQL team, MariaDB actively works with outside developers to deliver the most featureful, stable, and sanely licensed open SQL server in the industry. MariaDB is designed as a drop-in replacement of MySQL(R) with more features, new storage engines, fewer bugs, and better performance.

RethinkDB

RethinkDB is built to store JSON documents, and scale to multiple machines with very little effort. It has a pleasant query language that supports really useful queries like table joins and group by, and is easy to setup and learn.

ArangoDB

A distributed free and open-source database with a flexible data model for documents, graphs, and key-values. Build high performance applications using a convenient SQL-like query language or JavaScript extensions.

Related Comparisons

Google BigQuery vs InfluxDB: What are the differences?

Key Differences between Google BigQuery and InfluxDB

Google BigQuery and InfluxDB are two popular database management systems that offer different features and functionalities. Here are the key differences between them:

Google BigQuery vs InfluxDB

Overview

Google BigQuery vs InfluxDB: What are the differences?