Rupen Makhecha

If your primary focus is just storage of data and read's and not too many frequent writes, in this case you can build your API on serverless platforms ( AWS Lambda ) and store the Data in Dynamo DB's which comes with decent free tier usage. Firebase is preferably a good choice when you need real-time updates to be synced to clients ( Mobile devices or Web) .

Nilesh , The best thing I would suggest considering a premises solution is to go with Multiple Master or Master-Slave replciation of database . This will make hazzle free synching. Make On-premises database as Master and cloud DB as slave. your GO Layer will talk to Mysql or Postgre layer which will be on premises and when there is a network avaibality it will auto sync. Similar you can also try MongoDB ( latest 4.X ) comes with Group replication , we have used in one of our project but i have no tried the same with this model. you can try this apprach if this works .

You can try dedicated server options as well , there are few companies like tier.net, contabo.com who offer great VPS and dedicated servers at great price point.

Let me know if you need any thing else.

Thanks and Regards Rupen www.voilacabs.com

Hello Suhas , We build our product www.voilacabs.com which is in the same lines as yours but we have used a combination of Mysql and MongoDB. When using MySQL, i would recommend doing the following: 1. Use Mysql only for storage only and for realtime updates we recommend MongoDB. 2. Don't try to Join more than 3 tables. ( the moment you reach 3 join stop there and try to un-normalized database. 3. Never or very rarely use Auto-increments. ( we recommend using UUIDS ) . Use UUIDS always for Auto increments for MYSQL. If you using Postgre SQL then i would suggest you to please check this https://instagram-engineering.com/sharding-ids-at-instagram-1cf5a71e5a5c There is a stored procedure that generated unique keys instead of auto-increment keys and that will help you sharding or clustering database without sync errors. 4. Also For MongoDB if you can put a layer of REDIS Cache then that will boost your api performance under large loads. 5. Use Node.js programing language as that function asynchronously .

Let me know if you still need any suggestion's . Thanks & Regards Rupen Makhecha CTO @ Voila Cab's www.voilacabs.com

Hello , Our first version of www.voilacabs.com was programed with Code Ignitor PHP API's, and at times during peak hours when there were huge request's made the response time use to vary from 500 MS to 1 Minute, also at time's we might have to end up restarting PHP-FPM which use to bring things to normal.

Also PHP lacks features like promises queue, Asyc task, etc. which is supported in Node.js. In our V1 release we shifted to Node.js and since last 7 month's we see no lag in any API response, also all our API's are responding in 200-300 Milli Second's with about 8000+ Fleet's in peak hours. So my suggestion is to go with Node.js also you can either go with Mysql or MongoDB ( depending on the usage ) for DB choice's.

When it comes to hosting you can go with group replication which can be replicated across 3 different datacenters for your data safety. This approach will make your system largely scalable at 80% less price then amazon webservcies.

I hope that help's .

We have used Node.js ( MVC pattern ) as core and also used Sailjs.org ( Built-in MVC for Node.js ) for our product voilacabs.com. We are getting about 50+ concurrent API request every second. Also, the app works rock solid on concurrent 10,000 user's using the system every day. Its been 6 months now and absolutely happy with response time and performance of Node.js. We have also used Angular 7.0 (Typescript ) to build GUI's which can be hosted on S3's bucket or even firebase and will get you the best in class speed at pocket friendly price point.

www.voilacabs.com

Voila Cabs is a dispatch engine and it more or less works like eco-system for all taxi aggregator’s / companies. With Min volumes of 100 to 1000+ taxi for each companies who’s driver’s frequently updates their location’s in realtime for right job getting dispatch to right driver all this is amassing several terabytes of traffic and is managed by small team without Amazon stack. We thought it would be great to give a sense of our system’s, there is more in-depth descriptions of some of these component’s used and is a glimpse of how a startup with a small engineering team can scale app’s at budget friendly price point’s.

Our Old Tech Stack

We Started with on Only PHP - using Code Ignitor Framework for first 15 Months,PHP resources comparatively far more expensive both in terms of CPU & memory utilisation than Node.js . For PHP we have to rely on 3rd party for implementation of queue’s and async. task etc. moreover the API response speed we get in Node.js is tremendous. Since last 2 years we have experimented lots of things with our primary focus being speed and scalability and finally settled up with Node.js as core technology for our application stack. We have used a combination of My-Sql and MongoDB database for our App’s.

High availability Dedicated Server Set Up.

When calculating amazon EC2 instances price’s or other cloud cloud server prices, it was working out very expensive and won’t fit for cost effective model. An alternative to this is multiple dedicated machines with large dedicated RAM’s & very high core processor’s located in different zone’s. We have installed Database and Application installed on same Machine, We have High Availability Mysql server’s cluster in Multi Master Replication across multiple region’s along with database balancing done using HAProxy installed on both of them.

OS / Hosting

We run Ubuntu Linux 18.04 on Dedicated Instance’s with 200+ GB RAM’s & 32 Core Processor’s with Node Instances running as Cluster’s. We prefer Dedicated instances over cloud instances due to cost effective benefits. We are with tier.net for our dedicated server’s and so far our experience is great with them.

Load Balancing

Every request to our servers goes through load balancing machines; we used Nginx as load balancer and also Reverse Proxy to route secure request to Application API’s . We use Cloud Flare for DNS, which does instant DNS redirection for new client’s. We terminate our SSL check on loadbalancers for speed and further routing is done using Proxy Pass client’s.

Data storage

Most of our Booking Data, Driver Data etc. lives in MY-SQL; Since our application and database server runs on the same Extra-Large memory instances (with 2 replicas in a different zone.). Our Db instances run on master-replica setup using Streaming Replication, We use RAID arrays for snapshotting, also asset’s like photo’s and static data themselves go straight to Amazon S3, which currently stores several terabytes of data for us.

For our geo-search API we used MongoDB as 2D Index pattern which search nearest driver’s and also we realtime update the same. There is internal database sync between My-Sql and MongoDB for status updates as that’s achieved using Node.js.

Realtime Events & Push Notifications

When a booking request is send to nearest driver’s we dispatch the same using Realtime notification’s services. We have evaluated Rabbit MQ, Amazon SES & SNQ and finally settle ourselves with Firebase Realtime Notification, firebase has handled millions of notifications for us, and has been rock-solid.

Dispatcher & Web Booking Interfaces

When a booking is made it either dispatch manually or automatically depending on company preference’s or his choice of dispatch . Each booking arrived in a dispatch queue. A Realtime dispatcher panel is designed using Angular (Type Script ) ,along with updated realtime notification’s for him to dispatch job to right driver and is update asynchronously for better user experience. Dispatcher’s are designed to white labelled on the fly using fancy coding by our creative programmer’s and it gives super cool experience to dispatcher.

Our cool booking interface is designed using Angular (Type Script ) , Our Angular scripts are hosted on firebase static hosting panel’s and since many months it perform’s flawless.

Mobile App’s

Our Mobile App’s are coded with pure Native and its optimized for great speed.

Your thoughts

If this description of our systems interests you, or if you’re hopping up and down ready to tell us all the things you’d change in the system, we’d love to hear from you.  Feel free to drop me an email @ rupen@techintegrity.in for your queries/suggestion.