Rails

When creating the web infrastructure for our start-up, I wanted to host our app on a PaaS to get started quickly.

A very popular one for Rails is Heroku, which I love for free hobby side projects, but never used professionally. On the other hand, I was very familiar with the AWS ecosystem, and since I was going to use some of its services anyways, I thought: why not go all in on it?

It turns out that Amazon offers a PaaS called AWS Elastic Beanstalk, which is basically like an “AWS Heroku”. It even comes with a similar command-line utility, called "eb”. While edge-case Rails problems are not as well documented as with Heroku, it was very satisfying to manage all our cloud services under the same AWS account. There are auto-scaling options for web and worker instances, which is a nice touch. Overall, it was reliable, and I would recommend it to anyone planning on heavily using AWS.

Our front-end team decided to use React Storybook for our primary React development environment. It allows us to write components in isolation without the need to fire up our Rails stack. When writing components in isolation; you can focus on styling, behaviour and prop design. It forces you to think about how your component is going to be used by others. React Storybook uses webpack and hot module reloading under the hood. This allows us to write components very quickly since it hot reloads in the browser as you code!

The knobs add-on is great for testing different edge cases for the component props. There is even an add-on that will auto-render and snapshot your components with every prop permutation allows by your defined knobs. These snapshots can then be part of your CI testing.

We have a step in our build process that publishes a static React Storybook site on our production server. This allows our entire team to interactively test components before they are integrated into larger features. Once we are happy with our components in isolation, we integrate them into connected feature components which are wired up to Apollo and GraphQL to provide the data and state.

There are heaps of React Storybook add-ons to checkout. If you aren't using it, you should be.

At StackShare we were discussing how to increase the retention of our newly signed up users. We hypothesized that if we made certain changes to the emails in our on-boarding process we could increase our retention and activation of users.

We decided to use sendwithus because it offered us the ability to A/B test our transactional emails. We also utilized the sendwithus analytics dashboard to gain real time insight into the performance of our email campaigns. Furthermore sendwithus has a Rails gem that allowed us to easily integrate the product into our application. We were also able to integrate sendwithus with our SendGrid account. #ABTestingAnalytics #TransactionalEmail

Shortly after I joined Algolia as a developer advocate, I knew I wanted to establish a place for the community to congregate and share their projects, questions and advice. There are a ton of platforms out there that can be used to host communities, and they tend to fall into two categories - real-time sync (like chat) and async (like forums). Because the community was already large, I felt that a chat platform like Discord or Gitter might be overwhelming and opted for a forum-like solution instead (which would also create content that's searchable from Google).

I looked at paid, closed-source options like AnswerHub and ForumBee and old-school solutions like phpBB and vBulletin, but none seemed to offer the power, flexibility and developer-friendliness of Discourse. Discourse is open source, written in Rails with Ember.js on the front-end. That made me confident I could modify it to meet our exact needs. Discourse's own forum is very active which made me confident I could get help if I needed it.

It took about a month to get Discourse up-and-running and make authentication tied to algolia.com via the SSO plugin. Adding additional plugins for moderation or look-and-feel customization was fairly straightforward, and I even created a plugin to make the forum content searchable with Algolia. To stay on top of answering questions and moderation, we used the Discourse API to publish new messages into our Slack. All-in-all I would say we were happy with Discourse - the only caveat would be that it's very helpful to have technical knowledge as well as Rails knowledge in order to get the most out of it.

As is common in the Rails stack, since the very beginning, we've stayed with MySQL as a relational database, Memcached for key/value storage and Redis for queues and background jobs.

In 2014, we could no longer store all our data in a single MySQL instance - even by buying better hardware. We decided to use sharding and split all of Shopify into dozens of database partitions.

Sharding played nicely for us because Shopify merchants are isolated from each other and we were able to put a subset of merchants on a single shard. It would have been harder if our business assumed shared data between customers.

The sharding project bought us some time regarding database capacity, but as we soon found out, there was a huge single point of failure in our infrastructure. All those shards were still using a single Redis. At one point, the outage of that Redis took down all of Shopify, causing a major disruption we later called “Redismageddon”. This taught us an important lesson to avoid any resources that are shared across all of Shopify.

Over the years, we moved from shards to the concept of "pods". A pod is a fully isolated instance of Shopify with its own datastores like MySQL, Redis, memcached. A pod can be spawned in any region. This approach has helped us eliminate global outages. As of today, we have more than a hundred pods, and since moving to this architecture we haven't had any major outages that affected all of Shopify. An outage today only affects a single pod or region.

The core Shopify app has remained a Rails monolith, but we also have hundreds of other Rails apps across the organization. These are not microservices, but domain-specific apps: Shipping (talks with various shipping providers), Identity (single sign on across all Shopify stores), and App Store to name a few. Managing a hundred apps and keeping them up to date with security updates can be tough, so we've developed ServicesDB, an internal app that keeps track of all production services and helps developers to make sure that they don't miss anything important.

ServicesDB keeps a checklist for each app: ownership, uptime, logs, on-call rotation, exception reporting, and gem security updates. If there are problems with any of those, ServicesDB opens a GitHub issue and pings owners of the app to ask them to address it. ServicesDB also makes it easy to query the infrastructure and answer questions like, “How many apps are on Rails 4.2? How many apps are using an outdated version of gem X? Which apps are calling this service?”.

In 2004, Shopify’s CEO and founder, Tobi Lütke, was building out an e-commerce store for snowboarding products. Unsatisfied with the existing e-commerce products on the market, Tobi decided to build his own SaaS platform using Ruby on Rails.

At that time, Rails wasn't even 1.0 yet, and the only version of the framework was exchanged as a .zip archive by email. Tobi joined Rails creator David Heinemeier Hansson (DHH) and started contributing to Ruby on Rails while building Shopify.

Shopify is now one of the world's largest and oldest Rails apps. It’s never been rewritten and still uses the original codebase, though it has matured considerably over the past decade. All of Tobi’s original commits are still in the version control history.

The bet on Rails greatly shaped how we think at Shopify and empowered us to deliver product as fast as possible. While there are parts of the framework that sometimes make it harder to scale (e.g. ActiveRecord callbacks and code organization), many of us tend to agree with Tobi that Rails is what allowed Shopify to move from a garage startup to a public company.

One of the challenges we've had to deal with as our product surface area has grown, is identifying and reproducing bugs. We use Sentry for exception monitoring, however, it's usually difficult to try to reproduce bugs. I first heard about FullStory from our friends over at Flexport (check out the Stack Story and you'll hear them mention it: https://stackshare.io/posts/how-flexport-builds-software-to-move-over-1-billion-dollars-in-merchandise). FullStory let's you record user sessions, and play them back to help you identify bugs and UX issues. You're even able to view the console errors live as they happen during the sessions!

We were pretty blown away at how comprehensive the product was at first, and it seems to be getting better every time I use it. Only complaint is that it's super expensive once you're in the hundreds of thousands of sessions so we had to stop trying to record logged out sessions, we only use it for auth'd sessions. We also started out using it via Segment but once we needed to watch out for the number of sessions we were recording we realized that it was impossible to restrict FullStory recordings on a per-page basis without ripping it out of Segment, so we ended up just using their JS snippet and putting that in the Rails views that we wanted to monitor closely.

The ability to share specific portions of sessions, speed them up, skip inactivity, and all sorts of other little features all add up to a really solid product that helps both our PMs and engineers improve our own product much quicker. I officially requested a Sentry + FullStory integration a while back https://twitter.com/yonasbe/status/871987738777616384, still waiting on this! #UserFeedbackAsAService #reproducing-bugs #sessionrecording #bug-squashing