Ajay M

Also, for database I'm using mongo for my this huge personal project but I checked Aerospike (more like redis + mongo) you may use this instead of mongo for better performance. Or if you plan on going on with mongo try Tokumx by http://percona.com/. It literally says MongoDB on steriods (I tried but didn't go with my setup). Percona is really known for pumping databases like MySQL for better performance.

I'd recommend using CloudTrail, it helped me a lot. But depending on your situation I'd recommed building a custom solution(like aws amazon-ssm-agent) which on configuration change makes an API call and logs them in grafana or kibana.

Go with RabbitMQ over redis. Not that I had any problem with redis but I used (without any problem) RabbitMQ with over 10 million records on a 1GB system and 3 cpu with 25+ (50 workers each node) nodes to handle scraping for our project. I'm sure redis can also handle that but it may need more ram.

As far I can understand, you want to generate jpg from a pdf file on the fly, right? I'd recommend looking into client side js libraries with which you can render and capture the page as jpg.

Please go with node js. Cause even if you go with php, you'd have to write another API in node to make it interact with php code base. There has been php socket libraries but I think the community is not large enough.

I don't think you can achieve parallelism with rails as I know ruby also got GIL similar to python. I'd recommend going with system languages, I'd some luck with Go to auto deploy scrapy scrapers. Or try java even though it's slow compared to other system languages but it's robust.