Large file uploads using PHP and the PUT HTTP verb

A week ago I made some modifications to the generic file uploading system I use in web development.

It is basically a stand-alone PHP web application that has the sole purpose of storing files. Right now it is dependent on Zend Framework, but in the final version, it should be easy to install anywhere and it should require minimal configuration. It should also be secure. I will call it the Media Repository

One possible use case is when building a classic HTML file upload form with PHP: after the temporary file is stored on the disk, it should get relayed to the Media Repository and stored there.

The Problem

In short, the problem is you can't upload large files. Initially, I was using the Zend Framework HTTP Client to relay the file to the repository. Problem is – when constructing the POST request to the Media Repository, the large file is stored as a string inside the Zend_Http_Client class. This is okay for small files, but it can become a problem for large files. The upload time and size is limited by the following settings:

The total amount of memory that PHP can allocate (php.ini memory_limit)
The maximum execution time and maximum input time (php.ini max_execution_time and max_input_time)
The maximum post size (php.ini post_max_size)
The maximum file upload size (php.ini upload_max_filesize)

The Solution

Increase max_execution_time and max_input_time. If your configuration allows it, it can be done with the ini_set() function, without having access to php.ini
Use the cURL extension. to check if your PHP installation has cURL, see How to check if an extension is installed. cURL allows you to avoid reaching the memory_limit when preparing the request. Using the CURLOPT_INFILE setting, you can instruct curl to transfer specific stream, instead of a string. It avoids storing the entire file content to memory.
It's also a better practice to use the PUT http verb instead of POST when posting files as raw data:

$ch = curl_init();

$source_file = 'example.txt'
$upload_url = 'http://example.com/folder/example.txt'

// Open the sourcefile
$readfile = fopen($source_file, 'rb');

curl_setopt($ch, CURLOPT_URL, $upload_url);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_PUT, true);

curl_setopt($ch, CURLOPT_INFILE, $readfile);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

// Relay the file to http://example.com/
$response = curl_exec($ch);

// Close the source file
fclose($readfile);

// close cURL resource, and free up system resources
curl_close($ch);

On the other side of the problem, to recieve the transmission on the Media Repository, again you have to avoid storing the whole content in a variable. This time, the POST input can be retrieved by opening the php://input stream:

$read_handle = fopen('php://input', 'rb');

// The filename where the input must be stored.
$target_filename = 'example.txt';

// Open for reading and writing; place the file pointer at the beginning of the file and truncate the file to zero length.
// If the file does not exist, attempt to create it.
$write_handle = fopen($target_filename, 'wb');

if (($read_handle == false) || ($write_handle == false))
    $success = false;

while ($success && !feof($read_handle))
{
    $chunk = fread($read_handle, 4096);
    $success = $success && (fwrite($write_handle, $chunk) !== false);
}

$success = $success && fclose($write_handle);
$success = $success && fclose($read_handle);

So this I think should be helpful when dealing with large file uploads in PHP.

POST#0057 2009-FEB-7

Help improve the Fusion Blog - express your opinion about the content on this page: I like it Can be improved

Commentary (2):

COMM#13073 2009-AUG-31

Vladimir Usenco wrote:

Is there any solution for large file uploads? I will need to upload several files which are about 100 Mb. Any idea to create a chunked upload?

COMM#13237 2009-NOV-8

Janybravo wrote:

Article describes only server side script. You'll need firefox 3.6 or Google gears or SWF upload for that