A few years ago I sold all my stuff to explore the world, creating 12 startups in 12 months and building $1M+/y companies as an indie maker such as Nomad List and Remote OK. I'm also a big pusher of remote work and async and analyze the effects it has on society. Follow me on Twitter or see my list of posts. My first book MAKE is out now. Contact me
Subscribing you...
Subscribed! Check your inbox to confirm your email.
levels.io

How I sped up Nomad List by 31% with SPDY, CloudFront and PageSpeed

Nomad List, Tech
May 8, 2015

These days building an MVP is easy, launching it is a challenge but if you succeed, usually your site will stay up (since it’s such a basic version). The problem really starts when you start growing your site, and your site’s traffic starts growing with it.

That’s what happened with Nomad List. In half a year it grew to over 500,000 pageviews per month, with over 100 million assets served per month. That’s about 200 assets per page load, mostly pictures of the cities). My tiny Linode NGINX server has been pretty good at handling it, but it hasn’t made the site particularly fast.

The thing is, when your site is going well, you’re now competing with all the other big sites. And they have people dedicated to making everything fast. And the more mainstream you go with your site, the share of “mega fans” of your site that are willing to wait for the site to load will decrease. You’ll be seeing “normal” people visit your site, you know the ones coming from Facebook etc., and if your site isn’t loaded in a few seconds, they’ll call it “slow”.

I mean, honestly, I do the same. So a few weeks ago, I was done with it and stuff had to speed up.

Testing

I used Pingdom to test load times. And always picked San Jose, CA as a base server. My Linode is located in London, so I felt that’d be a good test. I must say Pingdom is not incredibly reliable, so I’d take all of this data with a big grain of salt. Load times do vary somewhat even without changing anything.

We continue.

Basic caching

I already implemented basic caching months ago. Why? Well, the city database had become so big with 550 cities and 65 attributes per city (like apartment rent cost, hotel cost etc.), that it was now over 35,000 data points. That’s fine, but there was calculations done on those and they were done on every page load. I know, stupid. It worked in the beginning with 50 cities and 500 data points, not with 35,000. Doh.

I made this:

php -f /srv/http/nomadlist.com/app/index.php — ‘currency=usd’ — ‘units=metric’ > /srv/http/nomadlist.com/cache/index-usd-metric.html

And built a script to generate static pages for each combination of currency (EUR, USD, GBP etc.) and unit system (metric or imperial). A cron job runs it every few hours or so. I know, tedious, but it works for now until I deploy a new version of the site that does this in a better way.

Caching brought down the load time from a crazy 29.89s (which users never saw, as I implemented caching way earlier) to 3.90s.

Conclusion: Caching processing in a page speeds up everything drastically (well doh)

SPDY

With SPDY enabled, it actually pushed the load time from 3.90s (on SSL) to 3.97s. Not really sure why, maybe Pingdom doesn’t support/measure SPDY? For browsers supporting SPDY, it definitely sped it up noticeable making it much more snappy. Especially on subsequent loads. That means once you’re already on a site, the page loads from then on are much faster. SPDY keeps a live connection between the user and server, so that means it doesn’t have to re-connect on every page load (like HTTP/HTTPS).

Conclusion: SPDY doesn’t change speed significantly according to Pingdom, but does make it snappier on browsers that support it

CloudFront by AWS

@levelsio Why don't you load the static files from a CDN (Cloudfront with a custom origin). The CDN is location aware.

— Pieter Bogaerts (@PizzaPete) May 8, 2015

@PizzaPete on Twitter recommended me to set up CloudFront. It’s Amazon AWS’s CDN service. A CDN is pretty much a lot of servers around the world that mirror your content, so that when a user requests the file (like a photo of a city on Nomad List), it gets sent from the closest location to the user (e.g. a user in Amsterdam gets the image from London, a user in Shanghai gets the image from Tokyo).

Setting this up was quite easy.

I simply made a script that after scaling my images, would put them up on S3 with S3cmd:

s3cmd put /srv/http/nomadlist.com/public/assets/cities/500px/amsterdam-netherlands.jpg s3://nomadlistcdnbucket/assets/cities/500px/amsterdam-netherlands.jpg

Then I went into AWS panel and set up a CloudFront distribution that connected to my S3 bucket.

This means that instead of loading the images from https://nomadlist.com/assets/cities/500px/amsterdam-netherlands.jpg, it would now load them from https://d39d3kdh3l.cloudfront.net/assets/cities/500px/amsterdam-netherlands.jpg.

So did it speed things up? Let’s see.

CloudFront doesn’t support SPDY, only SSL. But with my own server’s SPDY enabled and CloudFront over SSL, load time went from 3.97s to 3.86s. Again, a very tiny difference. Without SPDY and just SSL it decreased to 3.82s. Over HTTP, CloudFront’s speed up was the biggest, from 5.40s to 3.64s. That’s almost 2 seconds!

Conclusion: CloudFront seem to improve load times over HTTP, SSL and SPDY with a very strong effect ovr HTTP

PageSpeed

@levelsio pagespeed module is pretty awesome too

— KJ Prince (@KJ_Prince) May 1, 2015

Another tip was PageSpeed, recommended to me by KJ Prince. I think I’m pretty late to the party as Google seems to be deprecating it. But I wanted to try it any way.

It decreased load times on SSL from 3.86s to 3.82s. Over SPDY it actually increased load times from 3.97s to 4.06s! Again, over HTTP the effect was strongest: from 5.40s to 4.13s load times. Almost 1 second won.

Conclusion: PageSpeed seem to improve load times over HTTP and SSL, not over SPDY, with a very strong effect over HTTP

PageSpeed + CloudFront

Let’s combine the two and see what happens.

With both enabled, on SPDY we get 3.95s. Without both enabled, SPDY was actually faster! Over SSL we get our fastest time of 3.79s. And over HTTP, we go from 5.40s to our fastest time of 3.56s. Again almost 2 seconds won.

Conclusion: PageSpeed and CloudFront combined seem to improve load times over HTTP and SSL, not over SPDY.

SSL vs. SPDY on Chrome

Since Pingdom doesn’t give us realistic results for SPDY, let’s test it with Chrome on OSX. I tested it from Bangkok, Thailand on a solid connection. Since my server is in London, that’s 9,526 km far. Pingdom’s connection from San Jose to London was about equal at 8,627 km distance.

Conclusion: On Chrome, SPDY seems to improve load times vs SSL, from 5.24s to 3.58s (-1.66s) or a 31% speed increase.

Results

     Load timeChange%
PingdomCacheHTTPCloudFrontPageSpeed3.56s-1.84s+34%
PingdomCacheHTTPCloudFront 3.64s-1.76s+32%
PingdomCacheHTTP PageSpeed4.13s-1.27s+23%
PingdomCacheHTTP  5.40s
Pingdom 
PingdomCacheSSLCloudFront SSLPageSpeed3.79s-0.10s+2%
PingdomCacheSSLCloudFront SSL 3.82s-0.08s | +2%
PingdomCacheSSL  3.90s 
Pingdom 
PingdomCacheSPDYCloudFront SSL 3.86s-0.11s+3%
PingdomCacheSPDYCloudFront SSLPageSpeed3.95s-0.02s+0.5%
PingdomCacheSPDY  3.97s 
PingdomCacheSPDY PageSpeed4.06s+0.09s
Pingdom 
PingdomNo cacheSPDY  28.45s
PingdomNo cacheSSL  29.72s
PingdomNo cacheHTTP  29.89s
Chrome 
ChromeCacheSPDYCloudFront SSLPageSpeed3.58s-1.66s+31%
ChromeCacheSSLCloudFront SSLPageSpeed5.24s

Conclusion

The best result we got is CloudFront + PageSpeed over HTTP with a speed increase of 34% (-1.84s), the second best was from SSL to SPDY with CloudFront + PageSpeed enabled on both with 31% (-1.66s).

The odd result here that there’s no real difference between using only SPDY (3.97s) or only SSL (3.90s) or using SPDY + CloudFront SSL + PageSpeed (3.95s) or using SSL + CloudFront SSL + PageSpeed (3.97s). That might be explained by that SSL slows down all connections, and SPDY only speeds up connections if everything is on one server. If you use a CDN like CloudFront you add more servers to the load times, which the client has to connect with. On SPDY the first connection actually takes a lot of time. From then on it’s fast. So for some sites, it might be faster to NOT use a CDN if they use SPDY.

PageSpeed’s results were negligible.

I managed to speed up the site on HTTP with 34% and on SPDY with 31%. I will keep CloudFront’s CDN enabled for the mere fact that the CDN still might help speed up connections in remote parts of the world. That’s a guess though.

P.S. I'm on Twitter too if you'd like to follow more of my stories. And I wrote a book called MAKE about building startups without funding. See a list of my stories or contact me. To get an alert when I write a new blog post, you can subscribe below:

Subscribing you...
Subscribed! Check your inbox to confirm your email.

2022
18 Sep
This House Does Not Exist
2022
14 Jul
Sam Parr + Shaan Puri asked me about bootstrapping, open startups and lifestyle inflation (My First Million Podcast)
2022
16 May
Thinking and doing for yourself (Life Done Differently Podcast)
2022
10 May
Relocation of remote workers (Building Remotely Podcast)
2022
26 Jan
Money, happiness and productivity as a solo founder (Indiehackers Podcast)
2022
20 Jan
Bootstrapping, moving to Portugal and setting up Rebase (Wannabe Entrepreneur Podcast)
2021
25 Mar
Why I'm unreachable and maybe you should be too
2021
25 Mar
The next frontier after remote work is async
2021
19 Mar
List of all my projects ever
2021
08 Mar
Why coliving economics still don't make sense
2021
14 Feb
Inflation Chart: the stock market adjusted for the US-dollar money supply
2021
10 Jan
I did a live 4+ hour AMA on Twitch w/ @roxkstar74
2020
20 Dec
No one should ever work
2020
10 Dec
Normalization of non-deviance
2020
05 Dec
Copywriting for entrepreneurs: explain your product how you'd explain it to a friend
2020
30 Nov
Entrepreneurs are the heroes, not the villains
2020
12 Nov
The future of remote work: how the greatest human migration in history will happen in the next ten years
2020
05 Nov
Will millions of remote workers become location independent in 2021?
2020
11 Apr
5 years in startups with Abadesi
2020
11 Jan
Twitter giveaways can be hacked to win every time
2019
16 Oct
Lorn - The Slow Blade ✕ Hong Kong
2019
28 Sep
Most decaf coffee is made from paint stripper
2019
12 Sep
The odds of getting a remote job are less than 1% (because everyone wants one)
2019
08 Sep
In the future writing actual code will be like using a pro DSLR camera, and no code will be like using a smartphone camera
2019
29 Aug
Instead of hiring people, do things yourself to stay relevant
2019
28 Aug
Nobody cares about you after you're dead and the universe destroys itself
2019
28 Aug
The only real validation is people paying for your product
2019
05 Aug
Monitoring Bali's undersea internet cable
2019
29 Jul
Nomad List turns 5
2018
29 Jan
I'm Product Hunt's Maker of the Year again!
2018
28 Jan
Why Korean Jimjilbangs and Japanese Onsens are great
2018
24 Jan
Turning side projects into profitable startups
2018
03 Jan
What I learnt from 100 days of shipping
2017
28 Dec
As decentralized as cryptocurrency is: so will be the people working on it
2017
22 Oct
How to 3d scan any object with just your phone's camera
2017
09 Aug
In a world of outrage, mute words
2017
03 Aug
How to pack for world travel with just a carry-on bag
2017
26 Jul
Building a startup in public: from first line of code to frontpage of Reddit
2017
24 Jul
Facebook and Google are building their own cities: the inevitable future of private tech worker towns
2017
21 Jul
The TL;DR MBA
2017
12 Jul
We did it! Namecheap has introduced 2FA
2017
08 Jun
It's about time for a digital work permit for remote workers
2017
23 May
Using Uptime Robot to build unit tests for the web
2017
08 May
Namecheap still doesn't support 2FA in 2017 (update: they do now!)
2017
03 May
Taipei is boring, and maybe that's not such a bad thing
2017
16 Apr
What we can learn from Stormzy about transparency
2017
17 Feb
The ICANN mafia has taken my site hostage for 2 days now
2017
10 Feb
Most coworking spaces don't make money; here's how they can adapt to survive the future
2017
11 Jan
A society of total automation in which the need to work is replaced with a nomadic life of creative play
2017
07 Jan
Nomad List Founder
2016
12 Dec
Make your own Olark feedback form without Olark
2016
29 Oct
How to fix flying
2016
19 Oct
Robots make mistakes too: How to log your server with push notifications straight to your phone
2016
17 Oct
Hong Kong Express - 上海 (Shanghai)
2016
17 Oct
Choosing entrepreneurship over a corporate career
2016
13 Oct
"I can't buy happiness anymore. I've bought everything that I ever wanted. There's not really anything I want anymore."
2016
11 Oct
From web dev to VR: How to get started with VR development
2016
05 Oct
What I would do if I was 18 now
2016
22 Sep
Bootstrapping Side Projects into Profitable Startups
2016
27 Aug
Kids
2016
13 Aug
How I cured my anxiety (mostly)
2016
26 Jul
We have an epidemic of bad posture
2016
17 Jul
Fixing "Inf and NaN cannot be JSON encoded" in PHP the easy way
2016
26 Jun
My third time in a float tank and practicing visualizing the future
2016
15 Jun
How to add shareable pictures to your website with some PhantomJS magic
2016
29 May
My chatbot gets catcalled
2016
19 May
From web dev to 3d: Learning 3d modeling in a month
2016
09 Mar
My second time in a sensory deprivation chamber
2016
04 Mar
Day 30 of Learning 3d 🎮 Cloning objects 👾👾👾
2016
02 Mar
Day 29 of Learning 3d 🎮 Glass, reflectives, HD, coloring and more details
2016
29 Feb
Day 27 of Learning 3d 🎮 Details, details, DETAILS!
2016
25 Feb
Day 23 of Learning 3d 🎮 Filling up the street and adding shadows
2016
24 Feb
Day 22 of Learning 3d 🎮 Added rain, blinking lights, sound, textured menu sign and a VR web app
2016
23 Feb
Day 21 of Learning 3d 🎮 High res textures, physical rendering and ambient occlusion
2016
22 Feb
Day 20 of Learning 3d 🎮 Objects and camera perspectives 🙆
2016
19 Feb
My first time floating in a sensory deprivation tank ☺️
2016
12 Feb
Day 10 of Learning 3d 🎮 Making complex objects by combining shapes 🙆
2016
06 Feb
Day 4 of Learning 3d: @shoinwolfe visits the actual street I'm modeling 🏮😎🏮
2016
03 Feb
Day 1 of Learning 3d 🎮 I learnt how to make shapes, move, rotate and scale them + how to texturize, and add colored lights 💆
2016
02 Feb
I'm Learning 3d 🎮
2016
27 Jan
The things I have to do to read an email sent to me by my government
2016
12 Jan
How to use your iPhone as a better Apple TV alternative (with VPN)
2015
23 Dec
Here's a crazy idea: automatically pause recurring subscription of users when you detect they aren't actually using your app
2015
17 Dec
Stop calling night owls lazy, we're not
2015
16 Dec
We are the heroes of our own stories
2015
25 Oct
There will be 1 billion digital nomads by 2035
2015
21 Oct
Tobias van Schneider interviewed me about everything
2015
18 Oct
Why doesn't Twitter just asks its users to pay?
2015
17 Oct
Punk died the moment we learnt that the world WAS in fact getting better, not worse
2015
15 Oct
Stop being everyone's friend
2015
14 Oct
Vaporwave is the only music that fits the feeling futuristic Asian mega cities give me
2015
09 Sep
We live in a world built by dead people
2015
01 Sep
Why global roaming data solutions don't make any sense
2015
26 Aug
How to export your Slack's entire archive as HTML message logs
2015
24 Aug
How to play GTA V on your MacBook (and any other PC game)
2015
14 May
I uploaded 4 terabyte over Korea's 4G, and paid $48
2015
08 May
How I sped up Nomad List by 31% with SPDY, CloudFront and PageSpeed
2015
04 May
My weird code commenting style based on HTML tags
2015
01 May
Now is probably the time to make HTTPS the default on all your sites and apps
2015
17 Apr
Do the economics of remote work retreats make any sense?