Python is actually the first decision we made in our stack selection process. The entire dev team is familiar with the language and more importantly, it is the language of choice for most of the leading machine learning research and applications. Another thing that we considered is that using python allows us to more easily hire developers in the future. Python is generally the kind of language in which it is really easy to get something started with minimal effort, which is ideal for us given our short timeline
We are planning to choose Docker since it will allow us to build and install libraries and dependencies with ease. Its extensive use in the world will be helpful to provide us with useful discussion boards. This will be the first time any member of the dev team will be using Docker as part of their application. Given the limited readings, we have been able to do about it in the time we had, we a really excited to get to work with it. It seems to have a lot of potential that we would like to explore as a team. Another reason is that our dev team currently only has access to Windows machines and we want our application to be system agnostic. Using Docker will also help us limit the number of CI minutes our application requires.
Overview: To put it simply, we plan to use the MERN stack to build our web application. MongoDB will be used as our primary database. We will use ExpressJS alongside Node.js to set up our API endpoints. Additionally, we plan to use React to build our SPA on the client side and use Redis on the server side as our primary caching solution. Initially, while working on the project, we plan to deploy our server and client both on Heroku . However, Heroku is very limited and we will need the benefits of an Infrastructure as a Service so we will use Amazon EC2 to later deploy our final version of the application.
Serverside: nodemon will allow us to automatically restart a running instance of our node app when files changes take place. We decided to use MongoDB because it is a non relational database which uses the Document Object Model. This allows a lot of flexibility as compared to a RDMS like SQL which requires a very structural model of data that does not change too much. Another strength of MongoDB is its ease in scalability. We will use Mongoose along side MongoDB to model our application data. Additionally, we will host our MongoDB cluster remotely on MongoDB Atlas. Bcrypt will be used to encrypt user passwords that will be stored in the DB. This is to avoid the risks of storing plain text passwords. Moreover, we will use Cloudinary to store images uploaded by the user. We will also use the Twilio SendGrid API to enable automated emails sent by our application. To protect private API endpoints, we will use JSON Web Token and Passport. Also, PayPal will be used as a payment gateway to accept payments from users.
Client Side: As mentioned earlier, we will use React to build our SPA. React uses a virtual DOM which is very efficient in rendering a page. Also React will allow us to reuse components. Furthermore, it is very popular and there is a large community that uses React so it can be helpful if we run into issues. We also plan to make a cross platform mobile application later and using React will allow us to reuse a lot of our code with React Native. Redux will be used to manage state. Redux works great with React and will help us manage a global state in the app and avoid the complications of each component having its own state. Additionally, we will use Bootstrap components and custom CSS to style our app.
Other: Git will be used for version control. During the later stages of our project, we will use Google Analytics to collect useful data regarding user interactions. Moreover, Slack will be our primary communication tool. Also, we will use Visual Studio Code as our primary code editor because it is very light weight and has a wide variety of extensions that will boost productivity. Postman will be used to interact with and debug our API endpoints.
RDBMS like Postgres can also store, index and query schemaless data as JSON fields, while also supporting relations where it makes sense. A document model is actually a downside, since usually data will still have relations, and it makes modeling them inconvenient.
I like the tech stack you guys have selected. You guys seem to have it all figured out, and well planned. Good luck!
We choose React for our client-side implementation because of React's virtual DOM implementation and component rendering optimization. It can help our app to be more stable and easier to debug. Also, react has strong support from the dev community. There is an enormous amount of reacting libraries we could use, which will speed up our development process.
We chose to include Git and GitHub for version control and Stack Overflow for support in our stack. These are extremely popular tools among the software development community. Git allows the team to split and merge our work efficiently. GitHub provides a range of functions including code management, tracking the progress of the product and reviewing the contributions of other team members. Stack Overflow contains solutions of many development problems across many different languages. It also provides the ability to create new inquiries if a solution is not found. Therefore it is a necessity in the development process.
We initially though we would use Django because it seemed to have a lot of the things we needed out of the box. After a bit of research we realized that using Flask would be a better option since it is more flexible and would be lighter for our purposes. Having set up our REST api using Flask we believe that we did make the right decision. We found that the flexibility of Flask along with the many extensions available for it to be very appealing. We were able to add the functionality we needed without much difficulty thanks to the quality of the extensions and their documentation.
Dane, if flask use extension, that means developer has to structure their own project. Do you know any good example or article in how to structure flask REST project
That's a good point. There is more you have to do but it also gives you more flexibility!
The Flask documentation is actually quite good and has some useful examples. Take a look at https://flask-restful.readthedocs.io/en/latest/ for the documention about Flask-RESTful
We decided to use Python for our backend because it is one of the industry standard languages for data analysis and machine learning. It also has a lot of support due to its large user base.
Web Server: We chose Flask because we want to keep our machine learning / data analysis and the web server in the same language. Flask is easy to use and we all have experience with it. Postman will be used for creating and testing APIs due to its convenience.
Machine Learning: We decided to go with PyTorch for machine learning since it is one of the most popular libraries. It is also known to have an easier learning curve than other popular libraries such as Tensorflow. This is important because our team lacks ML experience and learning the tool as fast as possible would increase productivity.
Data Analysis: Some common Python libraries will be used to analyze our data. These include NumPy, Pandas , and matplotlib. These tools combined will help us learn the properties and characteristics of our data. Jupyter notebook will be used to help organize the data analysis process, and improve the code readability.
UI: We decided to use React for the UI because it helps organize the data and variables of the application into components, making it very convenient to maintain our dashboard. Since React is one of the most popular front end frameworks right now, there will be a lot of support for it as well as a lot of potential new hires that are familiar with the framework. CSS 3 and HTML5 will be used for the basic styling and structure of the web app, as they are the most widely used front end languages.
State Management: We decided to use Redux to manage the state of the application since it works naturally to React. Our team also already has experience working with Redux which gave it a slight edge over the other state management libraries.
Data Visualization: We decided to use the React-based library Victory to visualize the data. They have very user friendly documentation on their official website which we find easy to learn from.
- Caching: We decided between Redis and memcached because they are two of the most popular open-source cache engines. We ultimately decided to use Redis to improve our web app performance mainly due to the extra functionalities it provides such as fine-tuning cache contents and durability.
- Database: We decided to use a NoSQL database over a relational database because of its flexibility from not having a predefined schema. The user behavior analytics has to be flexible since the data we plan to store may change frequently. We decided on MongoDB because it is lightweight and we can easily host the database with MongoDB Atlas . Everyone on our team also has experience working with MongoDB.
- Deployment: We decided to use Heroku over AWS, Azure, Google Cloud because it is free. Although there are advantages to the other cloud services, Heroku makes the most sense to our team because our primary goal is to build an MVP.
Communication Slack will be used as the primary source of communication. It provides all the features needed for basic discussions. In terms of more interactive meetings, Zoom will be used for its video calls and screen sharing capabilities.
Source Control The project will be stored on GitHub and all code changes will be done though pull requests. This will help us keep the codebase clean and make it easy to revert changes when we need to.
We choose Next.js for our React framework because it's very minimal and has a very organized file structure. Also, it offers key features like zero setups, automatic server rendering and code splitting, typescript support. Our app requires some loading time to process the video, server-side rendering will allow our website to display faster than client-side rending.
We decided to use React for our front end. We initially thought of using vanilla JS but during our planning future-proofing our application was a concern. Though we want to keep our app lightweight, prioritizing that now but ensuring a rewrite as our app expands, is not something we wanted to do.
Out of the current top web frameworks we went with React because of the team's familiarity with it. Additionally, the facts that React has been around for so long and has a large community made us more confident that we'd be able to get the support we need.
We changed to Python instead of Java to have the back-end processing in the same language as our data analysis module. In addition, Python has a lot of libraries for data-processing. We intend to use Flask for our back-end web development. Flask is a simple, straight-forward framework for our purposes. Flask also has a large community which is beneficial to the development process.