Editor's note: Evan Stites-Clayton is Co-Founder at Teespring. Eric Koslow is Lead Developer at Teespring.
Teespring lets anyone easily create and sell t-shirts. Design a shirt using their designer tool, set a price, add a goal and start selling. Sounds pretty simple on the surface. But what most people don't know is that Teespring has revolutionized the way in which t-shirts are actually made. They've come up with a way to automate what has traditionally been a labor-intensive fulfillment and shipping process, using custom-built software and a handful of developer services. We sat down with Evan and Eric to find out exactly how they did it.
"I built out an entire admin app for non-technical people to manage the platform...We wanted people to be able to design their t-shirt, launch a campaign, gather orders and tip the campaign, and then have those t-shirts printed and delivered with the fewest possible number of human touch points we could get."
"We built an app that is dedicated to interfacing with the printers. Each day the printers have a number of new jobs automatically assigned to them based on their capacity, which appear in a queue in what we call the 'printer portal.'"
"Building out our shipping systems, particularly generating shipping labels, has been one of our biggest pain points because of the addresses, domestic and international, and the different way people input that information...For a long time it was a developer’s full time job to correct the address inputs."
Teespring's Tech Stack
PHP CodeIgniter Ruby Rails Backbone.js MySQL PostreSQL
Heroku GitHub CircleCI Amazon RDS Amazon S3 Amazon CloudFront Honeybadger SendGrid Google Analytics EasyPost SmartyStreets Stripe Desk.com Grasshopper Nicereply Olark Flowdock Trello
Jump to the cloud services
LS: So let's talk about the beginning.
Evan: Walker and I met while we were both students at Brown. I was a computer science major, and he was a freelance designer and front-end developer. We clicked and the combination worked well, and we ended up working on a number of hackathons together before jumping into a full blown startup our Sophomore year. Unfortunately, that one never really got off the ground, though the technology still powers the Rhode Island state job board today. Our usual stack was CodeIgniter, a PHP MVC framework, with MySQL on the backend, all on Slicehost, which later became Rackspace Cloud.
Teespring came about our senior year when an infamous student dive bar, Fish Co., was shut down after they raided the place for underage drinking. It was the only thing people at Brown were talking about. We immediately decided to do some sort of project around that energy. We decided on t-shirts because we felt they were a pretty universal way to communicate a message, and to remember the bar by. With screen printed t-shirts, we knew we would get a high-quality product that we could price reasonably and still get a healthy margin from.
We knew we had to get our web app up in a matter of hours before the energy started dying down, so we just hacked together the simplest thing we could: A one pager with some basic Paypal Checkout integration. You can actually still see the original at freefishco.com. We were in such a time crunch that we didn’t even automate the progress bar -- we just figured we’d manually update it every hour or so.
We pushed the app live at about 11pm and the response surprised us. Orders started rolling in immediately. I think in the first 24 hours we sold over 300 shirts on about 6,000 visits. Because we’d done so many projects before, and had a startup we’d been working on for years, we were well aware that it wasn’t usually that easy to sell something. Between that and the emails we got from other people wanting us to build them a one pager for their idea, it made us wonder if there was something bigger underneath what we had done - a platform for other people who had great ideas.
In the summer of 2011, we started to build what would eventually become Teespring. The first thing I coded was a flash application to allow users to design their own t-shirts. It was completely pre-alpha -- not even close to what we're using now, but that was the first thing we wanted to attack because we knew it was going to one of the harder technical challenges we’d face.
LS: So how long were you just using Flex?
Evan: Well, the Flex design tool was never live. I just mentioned it as part of the story here because it was the first thing I ever worked on for Teespring.
The only purpose of the Flex design tool was just to show the world, and more specifically investors, that I was capable of actually creating something that was akin to what we were saying we were going to do. It didn't have anything to do with users using it.
LS: Okay cool. Can you talk a little bit about the behind the scenes, fulfillment tech?
Evan: A big part of the dev work I did in those early days, some of which we're still using today, was admin and process tools. I built out an entire admin app for non-technical people to manage the platform, an app for our printing and fulfillment teams to interface with incoming orders, and so on. We focused, and I think this was key, on efficiency from the very beginning. We wanted people to be able to design their t-shirt, launch a campaign, gather orders and tip the campaign, and then have those t-shirts printed and delivered with the fewest possible number of human touch points we could get.
In terms of the whole process and flow, the user designs the shirt by uploading images or using our font and art libraries. Once they’re done, we generate a .jpg for the mockup they’ll use on their sales page, as well as the pure design as an .EPS vector for the printers to print from. The printer receives everything in an extremely functional format, the vector art is sized and ready to go, fonts are translated into outlines, colors are separated, and shipping labels are ready as a pdf -- It’s all incredibly easy. The ease of the system allowed us to convince some of the best printers in the country to work with us and to adopt our system, which is why we think we’re able to keep our print quality so high.
LS: So that's where a lot of the optimization is.
Evan: Yeah. We're basically taking a complicated process and automating as much of it as we possibly can. We’d seen some of the big t-shirt companies fail to get that automation in place early, and they end up needing hundreds and hundreds of employees as they grow. That was something we wanted to avoid.
LS: So how do you interact with the printing companies from a technical perspective?
Eric: We built an app that is dedicated to interfacing with the printers. Each day the printers have a number of new jobs automatically assigned to them based on their capacity, which appear in a queue in what we call the “printer portal.” From this queue they can get overall metrics of their performance and see newly assigned jobs with all the necessary assets: vector art, placement details, and PMS colors. They also have access to invoices, jobs in progress prioritized by the day they need to ship, as well as some other management functionality such as managing the team that can login into the portal, what access they have, etc.
Another step in this process is the actual ordering of the blank products for the printers to print on. As you can imagine, now that these partners are printing thousands of products a day, we need to make sure they are always stocked with “blanks.”
LS: How do you order the blanks?
Eric: So there is a standard called EDI, electronic data interchange. It's a standardized way of placing an order in bulk in this kind of system. It's honestly very archaic and outdated, but it’s what’s available. Basically these large blank product suppliers have these EDI systems that we can hook into and see pricing, inventory levels etc. The EDI isn’t something we’ve always used, however. It is what the larger t-shirts companies use though, so we knew we’d need it eventually.
LS: So why didn't you guys use EDI in the beginning since everyone is already using it?
Eric: Because we just weren't working at that kind of scale.
Evan: Yeah. We found that it was actually not super time consuming, at the scale that we were at, to call in each order, and then we transferred that to the web app more recently.
Evan: Yeah, we're very satisfied with that solution. We don't think that it needs to change really. It's a good application.
Eric: It's still in house. It's custom to us.
Eric: Sure, so recently we started using more Backbone, not for the designer tool but just for random js display across the site. The designer is kind of its own separate silo of application and then the rest of the site where you actually make the orders and check out and stuff, we consider that a separate app. And the fulfillment system is a separate app. We have many apps that we kind of silo across: We have the designer, which is its own separate app, we have the campaign pages and checkout, which is its own kind of separate app, and label generation. Shipping label generation is actually a very tricky problem because many addresses aren't perfect and having to correct those addresses and fix those problems before we ship is really important, so that is kind of its own separate application as well. Then there are the printer portal and the admin portal for our own internal administrators, who kind of work with sellers to manage their campaigns.
LS: Okay so you basically have five different apps.
Eric: And we also have an analytics app as well that we use for internal analytics. And these are all completely separate code repositories that are deployed separately.
LS: Okay yeah so let's talk about your actual process here: build, test, and deploy process.
Eric: We use GitHub for posting all of our code and a centralized place for all the developers to work together. We try to use a pull request model for code reviews. We use different feature branches that each developer works on and then once they are done, they create a pull request for that feature.
We have a few staging servers that we use. Everything is deployed to Heroku. Except for our old PHP codebase which are on Rackspace servers. We're slowly phasing that out as we move over to a Ruby-based system.
So we still have a few little segments of PHP in production, though they won’t be around for long. We just recently moved both the admin and printing portals to Ruby, for instance. There is some work still to be done to move it out. The designer is, again, like half and half. It's kind of a transitionary phase right now.
I really like Heroku, especially with a small team like ours. We don't have any DevOps people, we're all just developers. And it's really nice to just be able to push code to another repository and in a couple of minutes it's deployed to the site.
For testing, we use RSPEC. We try to do behavior-driven design, so it's kind of like top level features down to the unit test. So a top level request spec and then as you need to, drive down to unit testing. We use CircleCI for our continuous integration server. And we're trying to make sure that master's always green.
All the apps have their own separate staging environments. Every time we push to master it automatically gets deployed to staging, so what we see on staging is always what's going to be in the next push to production. And every app is deployed separately. So what's nice about this is that different developers can kind of own a project because we are so small still, about seven engineers. Basically, every developer in the company has their own application that they kind of manage. And they are the ones that deploy it and really know all the intricacies of it. So no one's ever blocked by anyone else, and they can really focus on their piece. So Ricky, who is doing the analytics program, can deploy that analytics program ten times a day. He doesn't have to worry about ever slowing down the campaign pages from being served up or anything. It actually makes it easier to manage.
Now, scaling multiple developers who are doing the same thing, I think it will work for us as well, because eventually developers will have teams and they work on different things and that team can deploy without having to affect any other team.
LS: For Heroku, are you using anything for autoscaling?
Eric: No. We haven't really felt a need for it. Currently, we just kind of run our workers much higher than we ever feel like we'll ever hit traffic and it's still very profitable for us.
Evan: We're not too worried about like the server cost becoming inordinate compared to how much money we are bringing in. We make more, on average, than $1 per visit. It’s more important for us to always be up than to over-optimize for server costs.
Eric: Our servers are actually don't consume that much. We don't have too many demos running for each application. Our traffic is relatively low and it’s very simple to serve up. Most of our campaigns are completely cached. No db queries are happening when someone loads a campaign, so we try to cache as much as possible so that we're serving as quickly as possible and not using as much resources.
LS: Yeah. Speaking of database, what do you guys use?
Eric: We did have MySQL hosted on Rackspace for a pretty long time, and recently about two months ago or so we moved over to Amazon RDS so that we can better scale. Because we're in this transition period between PHP and Ruby, we couldn't use PostgreSQL on Heroku since Heroku Postgres does not support PHP. And we needed to support legacy until it's gone. Some of our other applications are using PostgreSQL, but the main campaign page, orders and transactions, that's all MySQL.
Evan: We're going to do some refactoring in the database.
LS: What do you guys use for error monitoring?
Evan: We use Honeybadger right now. I really like Honeybadger. I've used Airbrake and others in the past and Honeybadger seems to be the best. It's simple but it works very well. Everyone on the team gets the emails. And everyone is in Flowdock -- It's kind of like HipChat, but we prefer it. So all the error messages also go there for the developers to look at and fix. So Flowdock is part chat, part email, part notifications, basically. We have different rooms for different teams. We have the developer room, everyone in the company room, sales room, SF office, Providence office. It works well for our use case. We started using it about 6 months ago.
We get notifications from CircleCI when builds fail, Heroku whenever someone deploys, Honeybadger whenever there's an error, and GitHub whenever someone commits code to a repository.
We use Trello for project management. For features and bugs, it's all Trello. The other non-engineering teams also use Trello but their boards just look very different. Sales people have race tracks, you know how developers have "in-progress, ready for QA, deployed, etc,” they have "cold lead, hot lead, etc."
LS: Going back to some of the frameworks and tools you use, is there anything that's been particularly useful?
Evan: Building out our shipping systems, particularly generating shipping labels, has been one of our biggest pain points because of the addresses, domestic and international, and the different way people input that information. There’s just so much variance in how people give you their address, and if USPS doesn’t like it you’re not going to be able to generate a label later when you’re ready to ship.
For a long time we were using Endicia, which is one way to buy shipping labels for USPS. When the label purchase failed, you’d get a very cryptic error message of why it doesn't like that address. And we banged our heads against that for a very long time before creating our own systems, or our own heuristics of what type of addresses will be accepted and what type aren't.
For a long time it was almost a full time job to correct shipping labels, to correct the address inputs.
Eric: It was a developer’s full time job.
Evan: Eventually we started to improve the system, reduce the frequency of errors, through address verification tools like SmartyStreets for domestic address verification. For international orders, we run a test label purchase shortly after they checkout. If it fails, we send an email to the customer letting them know and giving them a time window to correct the error before we ship. It’s kind of a hack, but the only way to really know if the label purchase is going to work is to run a test purchase.
After they place their order, we try to make a test shipment with USPS for their address and if the test shipment fails then we know that USPS won't accept it and we tell the customer that sorry, USPS says that they can't ship there, please fix your address here. For that, we use a service called EasyPost instead of Endicia. We actually buy postage through them as well. SmartyStreets just verifies addresses, but we literally try to test buy postage from EasyPost and if they give us an error message back then we know we can't ship there. And that's why it's sort of a hack.
LS: What is the difference between a test order and a real one, is it just fake credit card?
Evan: Test orders go through staging servers on USPS, which are designed to be used when you’re first integrating, but unfortunately it’s the only way for us to be sure the label will eventually generate. It's just one of the headaches that come along with shipping at scale.
It used to be that we would try to correct addresses from labels that failed to generate, but we had to do it extremely quickly because we don’t generate labels until close to when the orders ship, as we want to give customers as much time as possible to change their address, cancel, etc. By moving that test earlier in the process, and creating a tool that helps customers correct the errors themselves, it’s a lot less painful than it was.
Eric: Yeah, and having the customer correct the address happens pretty quick. We send them an email via SendGrid that they have to follow up with. That's actually a very small percentage of users but the percentage is high enough that we know it was taking up someone's full time job. I mean, we just do so many shipments now that even if zero to 1% of them have bad addresses, it's going to take a very long time to fix them all.
LS: Very cool. Any other services or tools that have been really helpful for you guys?
Evan: We use Desk.com for ticketing, Olark for live chat, phone lines for calls, and Grasshopper for voicemails. Grasshopper uses speech to text and then sends that to Desk.com, so even when we’re not around we can field calls. In terms of monitoring performance, the support team also uses Nicereply, which allows us to track our Net Promoter Score and keep an eye on our customer satisfaction.
Google Analytics and Stripe. Google Analytics is obviously just a jack-of-all-trades for conversion funnels, traffic and location data, etc. Stripe makes credit card processing easy to set up and once you’re set up you never have to think about it. There's a lot of moving parts at Teespring, not having to think about something like credit card processing is great, especially for a small team like ours. It makes us more efficient.
For customer analytics, we have our own user-facing analytics program that we've been slowly rolling out to some of our top sellers. On the backend, we do collect a lot of our own analytics now that Google Analytics doesn't necessarily give us. We can see, on a per campaign basis, conversion rates by source, social sharing rates, and all that kind of deeper analytics information. We built the internal analytics app because we wanted to be able to control and visualize the data honestly. This way we get the perfect system and we own the data.
We use CloudFront for our CDN. It serves a lot of our assets so our application doesn’t deal with the full load. There is a ton of image data that we're serving up. We use Amazon S3 for storage.
LS: You guys have really taken the hacker approach to putting in orders for t-shirts and then fulfilling them, in a way that has never been done before.
Eric: Yeah. We're so small that we can innovate quickly, we can do whatever. We can try new things and switch directions very, very quickly in a way that other competitors of ours cannot do. We've moved into not just t-shirts but basically all apparel and even different products. We've sold everything from posters to phone cases, custom jewelry to Christmas ornaments.
Evan: It's not just about t-shirts, it's a better way of selling online. Anything can be custom. Most manufacturing processes can and should be standardized and made accessible in the way we’ve done it for t-shirts.
Evan: It's about taking great ideas and bringing them into reality. We take away all the risk, all the costs, all the compromises, and just make it purely about having a great idea and being able to find people to support that idea. That's the nice thing about Teespring. Normally when you want to sell something, you have to buy it all upfront and take on a ton of risk before selling. How could an average person have any idea how many products they need to print, much less in what sizes? With Teespring, there is no costs or risk, no wondering how many items you need to pre-produce -- We make it all about the idea.
LS: Very cool. So what's next from a technology perspective?
Evan: First step is getting rid of PHP and migrating everything to Rails. We’re also hoping to offer people more product categories to sell in the future, which is going to require a lot of change. Right now we're very apparel-based, so a move to a more product agnostic platform is going to require a lot of changes, from the db schema to the logistics systems.
It’s all towards this vision of shipping millions, tens of millions hopefully, of items a month and changing the way people think about selling.
Learn more about how Teespring works.