TimescaleDB

TimescaleDB

Application and Data / Data Stores / Databases
Technical Architect at ERP Studio·
Needs advice
on
CassandraCassandraDruidDruid
and
TimescaleDBTimescaleDB

Developing a solution that collects Telemetry Data from different devices, nearly 1000 devices minimum and maximum 12000. Each device is sending 2 packets in 1 second. This is time-series data, and this data definition and different reports are saved on PostgreSQL. Like Building information, maintenance records, etc. I want to know about the best solution. This data is required for Math and ML to run different algorithms. Also, data is raw without definitions and information stored in PostgreSQL. Initially, I went with TimescaleDB due to PostgreSQL support, but to increase in sites, I started facing many issues with timescale DB in terms of flexibility of storing data.

My major requirement is also the replication of the database for reporting and different purposes. You may also suggest other options other than Druid and Cassandra. But an open source solution is appreciated.

READ MORE
3 upvotes·432.7K views
Replies (1)
Recommends
on
MongoDB

Hi Umair, Did you try MongoDB. We are using MongoDB on a production environment and collecting data from devices like your scenario. We have a MongoDB cluster with three replicas. Data from devices are being written to the master node and real-time dashboard UI is using the secondary nodes for read operations. With this setup write operations are not affected by read operations too.

READ MORE
6 upvotes·1 comment·62.3K views
Don Bizzell
Don Bizzell
·
February 9th 2022 at 5:05PM

You might want to look at Yugabyte DB it is open source, scalable, and can do geographicly distributed clusters. Best of all it fully supports Postgress, so you may not have to change anything but a driver.

·
Reply
Needs advice
on
ElasticsearchElasticsearch
and
PostgreSQLPostgreSQL

Hi, I need advice on which Database tool to use in the following scenario:

I work with Cesium, and I need to save and load CZML snapshot and update objects for a recording program that saves files containing several entities (along with the time of the snapshot or update). I need to be able to easily load the files according to the corresponding timeline point (for example, if the update was recorded at 13:15, I should be able to easily load the update file when I click on the 13:15 point on the timeline). I should also be able to make geo-queries relatively easily.

I am currently thinking about Elasticsearch or PostgreSQL, but I am open to suggestions. I tried looking into Time Series Databases like TimescaleDB but found that it is unnecessarily powerful than my needs since the update time is a simple variable.

Thanks for your advice in advance!

READ MORE
4 upvotes·128.4K views
Replies (1)
CEO, lead developer at Localazy·
Recommends
on
PostgreSQL

In your situation, PostgreSQL seems to be better option. Why? 1) Saving structured data is possible in both PostgreSQL and Elasticsearch. In PostgreSQL, there is JSONB column available and you can build indexes on top of it. 2) If you are able to specify the time as a primary key, both Elasticsearch and PostgreSQL are great options. 3) PostgreSQL allows you to do a lot more with your data and handle them in a relation way. You are not clear whether it's a benefit or not but let's consider extensibility to be an advantage. 4) PostgreSQL comes with PostGIS extension to work with geo data. May be useful for your situation. 5) PostgreSQL may serve for other needs of your app. Managing one database is always better than having two of them.

Thanks to JSONB column type, PostgreSQL is a sweet combination of relational and noSQL database, but there are also drawbacks coming from ACID compliancy and WAL overhead for rapid changes.

READ MORE
4 upvotes·4.1K views