We're a non-profit looking to rebuild our system.
Hey everyone - this seems like a little bigger project then I've seen on here! I tried to give a concise description of the project. We haven't started, but this is the general idea for our tech stack and what we're planning!
Description:
- We handle all sorts of features within the medical supply chain, from country-wide warehouses to front-line health facilities.
- We are in around 30 developing countries, and it would be essential to migrate our clients, who currently have their 4D relational databases, to our new system.
Our current system has:
- A single central server, with many clients/servers scattered throughout a country. Client servers are in hospitals, pharmacies, health centers, small regional warehouses. The main server is in a country's main warehouse.
- Countries are independent and run their own system.
- Countries operate on what can only be described as .. extremely poor internet (assume 56k modem in the middle of the Amazon)
- Our system is a transactional system run all through the same 4D database, which has a lot of extra features:
- Allows clients to create orders to other members of the system - i.e., hospital->warehouse.
- Allows dispensing medicines to patients.
- Allows managing where stock is in a facility.
- Manage expiry dates, stocktakes, inventory adjustments
- Reporting on stock movements through the single 4D database
Main requirements are:
- Extensibility. Due to being in so many countries and expanding, we have a lot of customizations. We'd most likely be wanting to look into some sort of plugin system.
- Eventual consistency & availability. Due to operating in developing countries, we use offline-first filtered replication throughout the system and require each of the client servers to have (close to!) 100% availability of data, regardless of their network connection. We need eventual consistency in the main server so that we can report country-wide.
- Scalability. We don't need to scale to massive amounts. Our maximum capacity looks to be a country around 200 million. That's about.. 1000 million records every year syncing into the main server.
Current thoughts: We're currently looking to run a PostgreSQL instance on each of the client servers. Each client server running an ExpressJS server. React as a front end. We are looking to roll our own sync system through most likely -a RESTful API.
This is a pretty vague and small description. I was wondering if anyone had any thoughts on a tech stack?
I think your choice of technologies are fine, but it you are starting new development I think your choice of architecture is most important. You can implement the desired architecture with many different technologies. Separating the layers of the application is important to allow each technology to be replaced or evolve independently. For example, I would write the code in a way that I could replace PostgreSQL with a different DBMS fairly easily in case a better alternative came along. The same goes for the React front end. If you have a good REST API you could replace the React front any with any framework, you could easily allow different locations to use Angular, Vue, or other front end technologies as long as the REST API is not designed to specifically target the React front end. I think the architecture of the code behind the REST layer, that would run in Express (the tool/language you are considering) is the decision that you will be stuck with for the longest time and that all other aspects of the application need to work around; so you want to get that architecture right. I think you should consider CQRS and Event Sourcing. https://docs.microsoft.com/en-us/azure/architecture/patterns/event-sourcing https://dzone.com/articles/cqrs-and-event-sourcing-intro-for-developers
I would really avoid building all of the scaffolding you need for the global server-server replication, eventual consistency and offline tools.
As long as your application architecture can fits a Key/Value pair type store, and you want master-master replication, then I'd start with AWS DynamoDB configured in Global Tables mode: https://aws.amazon.com/dynamodb/global-tables/
Then for offline-first access from client devices, use AWS Amplify, where (when used right) it's datastore feature will do most of the online/offline sync for you and let the developers focus on the mobile or web app. https://aws.amazon.com/amplify/features/#DataStore
I've had great success in using Amplify over very spotty connections (moving vehicles), using React and React Native for Web and Mobile applications respectively.
If your back-end data architecture really must be more RDBMS/SQL like, then consider whether you can use a central database in one geography, with all updates finding their way back there, then with read-replicas in other geographies. A good article on this, and suitable cloud-first databases to use, is here: https://read.acloud.guru/why-and-how-do-we-build-a-multi-region-active-active-architecture-6d81acb7d208
