Making PHP as fast as a Web server to send files

Web servers are very good at streaming files and currently Nginx holds the throne when it comes to serving static files. My blog used to go down every time I get a spike in traffic but now I serve all my static files with Nginx. However, there are times where you don't want to serve files directly. You want to add a layer of security before serving it. How do you serve the files efficiently with PHP?

There are many scenarios where you want to use PHP for the job. For example, on a website that requires a log in, you want to serve a report that requires proper authorization before downloading. The file in question is not publicly available, let's say you save it in a non web facing folder.

This is common and very easy to do. But the simplest way to do it in PHP is not always the best. Let's explore an example:

$file_path = "/non/web/facing/file.csv"
if ($user->isAuthorized()){
    if (file_exists($file_path)){
        $file = file_get_contents($file_path);
        header("Content-Type: text/plain");
        echo $file;
        exit;

    }else {
        throw Error("File does not exist");
        exit;
    }
}

This is self explanatory. First we make sure the file exists, we fetch it, then we print it on the page. This should work fine if you are serving small files. But, what happens when you serve a file that is bigger then the Allowed Maximum Memory?

Our code takes the file and loads it entirely in memory first then prints it to the user. The file is not manipulated, so it makes no sense to store it in memory.

I've seen it many times in forums where the solution is to allow PHP to use all the memory available. Obviously this does not scale. If you use a small digitalocean instance of 512M of RAM, it will fall short quickly.

Fatal error: Allowed memory size of 16777216 bytes exhausted (tried to allocate 122880 bytes) in /var/www/website.com on line 42

How can Nginx and Apache serve files that are bigger then the maximum memory available? They have the luxury of accessing system calls like sendfile(2)

sendfile() copies data between one file descriptor and another. Because this copying is done within the kernel, sendfile() is more efficient than the combination of read(2) and write(2), which would require transferring data to and from user space.

With sendfile, the web server can pass the file directly to the network card, leaving its hands clean.

Well, we can't use sendfile in PHP (yes we can, but more on that later). But, we have something better then our file_get_contents function: Readfile.

All readfile() does is read a file and write it directly to the output buffer. But here is the biggest advantage:

readfile() will not present any memory issues, even when sending large files, on its own.

The solution is straight forward. We can update our code to make use of readfile.

$file_path = "/non/web/facing/file.csv"
if ($user->isAuthorized()){
    if (file_exists($file_path)){
        header("Content-Type: text/plain");
        readfile($file); // Reading the file into the output buffer
        exit;

    }else {
        throw Error("File does not exist");
        exit;
    }
}

A single change and a huge improvement. You can transfer large files now. We still don't have the speed of web servers but we can live with it.

But, like the title says, we want to match the speed of web servers. So keep on reading.

Making PHP file serving as fast as Apache

Meet mod_xsendfile. It is a small Apache module that checks for the presence of the header X-Sendfile to get started. Once it sees the header, it discards all the headers set in the code and let Apache take over to transmit files. That's because Apache is better optimized to send files. (using sendfile(2) of course).

$file_path = "/non/web/facing/file.csv"
if ($user->isAuthorized()){
    if (file_exists($file_path)){
        header ('X-Sendfile: ' . $file_path);
        header("Content-Type: text/plain");
        header('Content-Disposition: attachment; filename="file.csv"');
        exit;

    }else {
        throw Error("File does not exist");
        exit;
    }
}

This gives you the speed of Apache without the burden of PHP process and high memory usage. The only drawback is that this is not your traditional Apache module. You have to compile it with Apache from source (a turn off for most of us). There are good instructions here on how to set it up.

You get the advantage of using PHP dynamics to only allow authorized users and the web server speed to serve static files.

Also supported with Nginx

The only difference with Nginx is the name of the header you send: X-Accel-Redirect.

$file_path = "/non/web/facing/file.csv"
if ($user->isAuthorized()){
    if (file_exists($file_path)){
        header ('X-Accel-Redirect: ' . $file_path);
        header("Content-Type: text/plain");
        header('Content-Disposition: attachment; filename="file.csv"');
        exit;

    }else {
        throw Error("File does not exist");
        exit;
    }
}

If you need a specific implementation, consult the resources links below.

Resources:

• Apache mod_xsendfile
• Xsendfile with Ngnix
• Why is Nodejs slow for static files?
• PHP readfile
• Sendfile manual

---