Creating your own CDN with Nginx

If you've used WordPress, then you know that it has a nice little folder called upload where all the pictures and videos you upload get stored. This is all fine and dandy, until people start accessing those assets.

I learned it the hard way when one of my blog posts became popular. I had a total of 17 files being downloaded after each requests. This is no big deal with the everyday traffic I was getting. (my blog was only a few months old). Suddenly, I started receiving traffic and my server went Kaput.

One thing I learned is that downloading assets shouldn't hinder your server. In fact, your server shouldn't even feel it. That's why most big websites use CDNs (content delivery networks). The advantage is that those networks serve your assets without using any processing power from your main server.

But, not every website is big and using an amazon instance is most of the time overkill for a blog or small website. That's why I decided to roll my own.

The reason for a CDN is to make sure your content is delivered as fast as possible. Some have locations all around the world and serve content from the machine closest to your user. Which is good if you are a Google or facebook, but for your blog a simple server will do the work.

I spun a new server on Digitalocean and set it up as my CDN. Most of my traffic is from the US according to Google Analytics, so I chose a server located in New York. My visitors from overseas are not having a hard time accessing me either, it is just as fast.

users location

Daily traffic location

To tell you the truth, a CDN is just a web server and there is no magic to it. All I did was install Nginx on it.

Nginx is a webserver, just like Apache. The one important difference is that Nginx is very good with serving static content. I don't need to have this machine do any processing, all I care is that it serves content. After receiving thousands of requests, my Apache web server simply died.

The motto is Nginx can serve 10,000 requests per second. I never had to serve that much content but it worked very well for me during spikes. So I set up nginx, moved all my assets to the server, and updated the DNS record. Now all my static files are served from cdn.idiallo.com. Very easy.

This works for me because I use a very primitive process for uploading my files, at least I used to (scp). If you are using a WordPress installation it will be complicated to customize the code to upload the files to a different server instead. But there is a simple solution.

Shared folders. If you are hosting with shared hosting providers you are out of luck. But for others, you can make your upload folder and shared folder, accessible from your CDN machine. This way you upload in one machine and it automatically reflects on the other.

Once your CDN is setup properly, you never have to touch it. All it does is serve files and update when you make new files available.

Installing Nginx

If you are using a debian machine all you need is this command:

sudo apt-get install nginx

Choose a folder where you want your asset folder to be: (/var/www/assets/). Then you can point Nginx to it. Create an Nginx config file:

vim /etc/nginx/sites-available/yourwebsite.com

And add these settings:

server {
  listen 80;
  server_name cdn.yourwebsite.com;

  location / {
    expires 90d;
    root /var/www/assets/;
  }
}

Also make sure you create a symlink in the sites-enabled folder.

That's all there is to it. You can restart your server and start serving files from your own custom CDN now. You can update your DNS to use your new CDN. subdomain, which should be available in your hosting provider GUI.

Enjoy serving fast content without breaking your server.

Update:

If you want to upload files on your webserver but want to make them available on your CDN server, here is how you can share files between your webserver and CDN


Comments

PulloDimo :

Very Nice article I love it!

ibrahim :

I'm glad it was of use to you @PulloDimo :)

Brian T. :

A single instance setup to serve files (in addition to your OTHER single instance serving your main site) isn't really a CDN. There's no true redundancy or "cloud" aspect to it (other than being a VPS).

Also a huge benefit to a CDN is serving data based on location to the end client (for the lowest latency), while also providing fail safe routes in the event of a failure on one (of the many) CDN servers. Because of this a simple DNS round-robin would also not be considered true CDN (even if the DNS server served records based on the requesting IP's location).

I'd be interested in seeing a post about setting up a true CDN. Or at least in this post explaining anything with a single point failure isn't a CDN (and wouldn't work like say AWS CloudFront or CloudFlare). Since someone new to cloud services could get confused by possible misinformation.

Ibrahim :

@BrianT No true Scotsman would call it a CDN. However I do mention in the post that having AWS or Location based could be overkill for a small website, the alternative would be to use cloudflare since they have a free tier.

But the article serves its purpose of introducing the barebone of CDN. Once the static files are independent of the webserver, it allows for the creation of extra features like redundancy and location based.

Hopefully in a future post I will be able to explain these features.

Kris :

You could easily setup up a couple of these VPS's, and use amazons route 53 DNS services to load balance the traffic, based on server health and the users location. Amazons route 53 is cheap .50 cents per domain.

Ibrahim :

That's a very good idea @kris, it would certainly take care of the geolocation problem and speed up the requests.

Nope :

Hi,

I found this article very interesting. I did a similar experiment with Apache to serve my blog images. As a page could be composed by 10-15 pictures and most browsers limit the number of connection to a domain, I created another domain and it website on a vps to speed images download.

Ibrahim :

@Nope thanks for your comment :)

One of the main advantages of a cdn, whether it runs on Apache nginx or any, is that it takes the load of your main server.

This way the server that handles the logic of your application doesn't also have to serve files.

Amnuay Jitklang :

I try this but my nginx path is usr/local/nginx/ where should i create sites-enabled folder? Thanks

Ibrahim :

Hi Amnuay

You probably don't need the folder. Try to set it up without it and see if you can access the web server through the web browser.

Bellash :

Great post thanks.

Ibrahim :

Glad it was helpful to you @Bellash :)

trent :

How to do support REDIS caching etc?

Ibrahim :

@trent for this, you don't need redis. Nginx already has a caching mechanism and it is pretty good at serving files.

Check out proxy_cache it allows you to cache files beyond the user browser.

Tailor :

Nice post Ibrahim... I was searching for an answer and I read you content. I have a question. I have site for which I want to create CDN, as the server is US and target customer are in India the response time is poor. My question is if I create a CDN server locally which is hosted at home like environment and suppose if the hosted CDN network is down. Is there a mechanism to fall back to the original server ? so there is no down time is zero. If there is a alternative to this down time I can try Nginx or something similar. I appreciate all your thoughts on this. Thanks a lot.

Ibrahim :

Hi Tailor

I use digital ocean (my referral link), and I believe they have a data center in India.

What you can do is use nginx as a load balancer. You can use a couple more servers that use a shared drive to serve your files and use a round robin with fail safe directive:

Round robin

# main nginx server
upstream myapp1 {
    max_fails = 5; # number of times the server fails before we take it out
    # multiple servers for redundancy.
    server srv1.example.com;
    server srv2.example.com;
    server srv3.example.com;
}

You can control the number of times the smaller servers have to fail before you take it out of the equation, and use the rest.

I hope this answers your question.

Tailor :

Wow!!! thanks Ibrahim... this is what I was looking for. Thanks a ton :) I will have to work on this before implementing on the live server. Yes it answers my question. I appreciate your time.

olivedev :

I never thought that creating your own CDN is this easy. Still, why someone would want to create their own CDN instead of using popular CDNs? I am using Cloudflare because it can be easily enabled from the Cloudways platform I am using. I am satisfied with the result it has given me so far.

Ibrahim :

Hi @Olivedev

Using your own CDN gives you a control that is not available with any third-party platform. Sometimes is a simple as wanting privacy for your users.

If you are using a jquery library from the cdn, it makes it faster for your users to load, at the expense of routing their request to a third-party that can map their path around the web. Maybe you trust your CDN, that's fine. But having your own gives you an option if you don't trust them.

There was a very interesting article in the subject a few days ago, and this one is better at explaining as I am.

Another obvious reason to create your own cdn is, well because you can :)

Let's hear your thoughts

For my eyes only