Sharing files and directories in Ubuntu Server

In case you want to set up a CDN

If you read my post about creating your own CDN you must have wondered how to share files across servers. I use one server to host my code and another to host my image files or assets. This could be problematic for most simple set ups because you have to upload your files into a different server.

The solution is to use shared folders. You create a folder on your CDN server, and make it accessible through the application server. It's actually very easy.

Software requirement NFS.

You need NFS (network file system). The CDN will have the server application and our code server will have the client application.

Server (CDN)

Install the nfs server program:

sudo apt-get install nfs-kernel-server portmap

Half the work is already complete. All you have to do now is setup the directories you want to share.

Let's say the folder we want to share is called /var/assets. In my case I didn't have to change the ownership but if it fails to share, do change the owner to nobody and the group to nogroup.

sudo chown nobody:nogroup /var/assets

Now you folder is ready to be shared. We can add an entry inside /etc/exports to let the machine know that this folder is to be exported.

sudo vim /etc/exports

You can find your machine IP address using the command ifconfig right after inet addr:<ipaddress>. Let's assume 10.0.0.2 for our CDN server and 10.0.0.1 for our Application server.

Inside the export file add this entry:

/var/assets       10.0.0.1(rw,sync,no_root_squash,no_subtree_check)

Here we are telling our server that a client with ip 10.0.0.1 will have access to this folder.

Save and confirm these entries using the exportfs command.

exportfs -a

Now our server application is ready to broadcast the shared folders to the client. Now let's setup the client machine.

Client (Application)

Install the nfs client program:

sudo apt-get install nfs-common portmap

All we need to do now is create the directories where we want the shared folders to appear. The standard is to have the directory in the /mnt directory since what we are doing is called mounting technically.

Let's create the folders:

mkdir -p /mnt/nfs/var/assets

Now we can mount our shared folder at this location:

mount 10.0.0.2:/var/assets /mnt/nfs/var/assets

That's it! Note that this is the IP of our CDN server. You can run the df command to ensure that it works correctly.

df -h

Filesystem             Size  Used Avail Use% Mounted on
/dev/sda                20G  120M   29G   5% /
udev                   119M  4.0K  102M   1% /dev
tmpfs                   49M  405K   49M   1% /run
none                   6.0M     0  6.0M   0% /run/lock
10.0.0.2:/var/assets    20G  948M   19G   5% /mnt/nfs/var/assets

You can also create a file from the client and check if it is available in the server.

echo "hello world" > /mnt/nfs/var/assets/test.txt

One thing I like to do is create a symlink of the shared folder this way I don't have to type all this path in my application when uploading a file.

ln -s /mnt/nfs/var/assets /var/www/assets

This makes it part of my web application without really being there.

Also if you want to unmount a folder all you need is use the umount command.

sudo umount /mnt/nfs/var/assets

Last but not least, if your client server were to be rebooted for some reason, you would lose the connection. You would have to re-mount the folder. You can solve this by making sure that your shared folder is mounted at start time.

Add this entry inside /etc/fstab

10.0.0.2:/var/assets /mnt/nfs/var/assets   nfs     auto,noatime,nolock,bg,nfsvers=3,intr,tcp,actimeo=1800 0 0

That's it now you know you can enjoy uploading files from your we application and serving those files through your CDN.

Ressources:


Comments(7)

Rusel :

Hi when I try to mount the shared folder I get a connection time out, do you know what could be causing this?

Ibrahim :

Hi Russel,

without knowing all the details, I can only assume two things:

  1. check that you path and IP are correct. Unfortunately when you try to connect to the wrong ip, sometimes i doesn't just fail, instead it just hangs until the connection is terminated. So double check that.

  2. Is there a firewall blocking the connection?

    sudo iptables -L

Other than that I can't really tell you what is wrong. You can write more details of the issue on ServerFault and share the link with me here. This way you can get more people to help you :)

Rusel :

Hi man, thx a lot i was mounting the folder without the right permissions so i just set 777 to the folder i want to share and it works.

Now i have another problem, im using django and i want to use my new cnd.server to serve all the images but on the setting where you tell django to serve the static files is not doing it.

This are the lines on my setting file:

MEDIA_ROOT = 'http://162.243.30.116/'
MEDIA_URL = 'http://162.243.30.116/'

im trying first with the ip address then if it works ill buy a domain for it.

Thx again!

Ibrahim :

Hi Russel

Note that MEDIA_ROOT is not supposed to be a URL. Instead, you should use the path to your new shared folder:

MEDIA_ROOT = '/mnt/shared/folder'
MEDIA_URL = 'http://162.243.30.116/'

To test, try putting a file in that folder and access it on the browser like so:

http://162.243.30.116/image.jpg

This should be served with NGINX. It needs to work on its own, even if there is no django.

Rusel :

That was perfect man, thx a lot, it works amazing now! excellent post.

Ibrahim :

@Rusel You are very welcome :)

Ahmed Mohamedin :

hello

I am using wordpress, I tried to setup as u did and followed everything step by step, but I get permission denied while trying to amount.

I am using amazon light sail Ubuntu i wish there is anyway I can contact u to guide me please.

Let's hear your thoughts

For my eyes only