Rio das Ostras, Brazil, 06/2010

Build A Fast Image Processor Service With Lumen

06 July 2015

Yeah, there are <a href="https://imageresizer.io/">some</a> of <a href="http://www.picresize.com/">those</a> out there, but sometimes we just need to create and host our own stuff, right? <h2>TL;DR</h2> <h4>-- How fast?</h4> It's doing, here, around 155 requests per second on a 150px transformed image: <img src="http://i.imgur.com/VueqVBr.png" alt="Image" /> <h4>-- Is it available online?</h4> Yes: <a href="https://github.com/antonioribeiro">Github</a> & <a href="http://api.antoniocarlosribeiro.com/api/v1/image?url=http://laravel.com/assets/img/laravel-logo.png&width=150&brightness=10&contrast=50&flip">Demo</a> <h3>/TD;LR</h3> It is based on <a href="http://image.intervention.io/">Intervention Image</a> and <a href="http://lumen.laravel.com/">Lumen</a>, the stunningly fast micro-framework by Laravel. Before starting, you'll need to create a Lumen app, then install <a href="http://image.intervention.io/getting_started/installation">Intervention and Imagick</a>, <a href="http://carbon.nesbot.com/#gettingstarted">Carbon</a> and <a href="https://github.com/thephpleague/flysystem">Flysystem</a>, and if you want to use Laravel Cache, you probably will have to install Dotenv and enable it on your App Bootstrap file: <pre class="prettyprint"><code >Dotenv::load(__DIR__.'/../'); </code></pre> Our image processor will be URL based, so you and your users will be able to tell it to resize, blur, brighten, invert, flip etc. an image by sending commands via URL parameters. Let's say your base URL is http://api.imgproc.com, to resize an image you'll do: http://api.imgproc.com/api/v1/image?url=http://laravel.com/assets/img/laravel-logo.png&width=50 To also blur the image, after resizing it, you just have to add &blur=2 To the end of the URL and it's done. You can try the <a href="http://api.antoniocarlosribeiro.com/api/v1/image?url=http://laravel.com/assets/img/laravel-logo.png&width=150&brightness=10&contrast=50&flip">demo</a>. Some of the operations curently available are: <ul> <li>width</li> <li>blur</li> <li>brightness</li> <li>colorize</li> <li>contrast</li> <li>crop</li> <li>fill</li> <li>flip</li> <li>fit</li> <li>gamma</li> <li>greyscale</li> <li>heighten</li> <li>invert</li> <li>limitColors</li> <li>line</li> <li>mask</li> <li>opacity</li> <li>orientate</li> <li>(and many others)</li> </ul> This is what happens in the whole processing process: <ol> <li>Check if the file isn't present and cached</li> <li>Download the new file</li> <li>Process the file and save the processed version</li> <li>Create a response</li> <li>Cache the whole response</li> <li>Send the processed image to the browser</li> <li>In a further request to the same file, just send the cached request</li> </ol> Let´s take a look at the code. First create a route for your processor: <pre class="prettyprint"><code >$app->group(['namespace' => 'App\Http\Controllers'], function($app) { $app->get('api/v1/image', [ 'as' => 'process', 'uses' => 'ImagesController@process' ]); }); </code></pre> As we will need the request URL to process our image, our controller will basically instantiate the image processor and pass the current request on: <pre class="prettyprint"><code ><?php namespace App\Http\Controllers; use App\Services\Processor; use Illuminate\Http\Request; class ImagesController extends Controller { private $request; private $processor; public function __construct(Request $request, Processor $processor) { $this->request = $request; $this->processor = $processor; $this->processor->setResponse(response()); } public function process() { return $this->processor->process($this->request); } } </code></pre> Our Image Processor relies on some other classes, so it is also very simple: <pre class="prettyprint"><code ><?php namespace App\Services; class Processor { private $file; private $response; private $cache; public function __construct(File $file, Cache $cache) { $this->file = $file; $this->cache = $cache; } public function process($request) { if ($image = $this->cache->get($request)) { return $image; } $this->file->processRequest($request); if ( ! $this->file->isValid()) { return $this->makeResponseForInvalidFile(); } $response = $this->file->getResponse(); $this->cache->put($request, $response); return $response; } private function makeResponseForInvalidFile() { return $this->response->make( [ 'success' => false, 'error' => $this->file->getError(), ] ); } public function setResponse($response) { $this->response = $response; $this->file->setResponse($response); return $this; } } </code></pre> It basically checks if the request is cached and send it back if it is, otherwise it process the file request, check if the processed file is valid and if it's not it sends back an error, or it gets a new response, caches and returns it. Our File Processor has some more code: <pre class="prettyprint"><code ><?php namespace App\Services; use Carbon\Carbon; use Illuminate\Contracts\Filesystem\Factory as Filesystem; class File { private $request; private $valid = true; private $error; private $fileFinder; private $fileName; private $filesystem; private $url; private $image; private $urlHash; private $wasTransformed = false; private $transformedFileName; private $response; const URL_PARAMETER = 'url'; const FILES_FOLDER = 'files'; const TRANSFORMATION_SEPARATOR = '_'; const SLUG_SEPARATOR = '.'; const ERROR_SLUG_NOT_PROVIDED = 'URL not provided.'; const REQUEST_TTL = 600; const REQUEST_EXPIRING_DAYS = 30; const PATH_DEPTH = 8; public function __construct(FileFinder $fileFinder, Filesystem $filesystem, Image $image) { $this->fileFinder = $fileFinder; $this->filesystem = $filesystem; $this->image = $image; } public function processRequest($request) { $this->request = $request; $this->parseRequest(); } private function parseRequest() { if ( ! $this->url = $this->request->query->get(self::URL_PARAMETER)) { $this->valid = false; $this->error = self::ERROR_SLUG_NOT_PROVIDED; } $this->findFile(); $this->processTransformations(); } public function isValid() { return $this->valid; } public function getError() { return $this->error; } public function getResponse() { $filetype = $this->filesystem->mimeType($this->getFinalFileName()); $response = $this->response->make(file_get_contents($this->getRealFilename($this->getFinalFileName())), 200); $response->header('Content-Type', $filetype); $response->header("Content-Disposition", "filename=" . $this->getOriginalFileName()); $response->setTtl(self::REQUEST_TTL); $response->expire(self::REQUEST_TTL); $response->setExpires(Carbon::now()->addDay(self::REQUEST_EXPIRING_DAYS)); $response->setSharedMaxAge(self::REQUEST_TTL); return $response; } private function findFile() { $this->parseFileName($this->url); if ($this->fileFinder->find($this->transformedFileName = $this->getTransformedFileName())) { $this->wasTransformed = true; } elseif ( ! $this->fileFinder->find($this->fileName)) { $this->fetchOriginal(); } } private function parseFileName($url) { $this->urlHash = SHA1($url); $extension = $this->getExtension($url); $path = $this->getBaseDir() . DIRECTORY_SEPARATOR . $this->makeDeepPath($this->urlHash); $this->fileName = $path . DIRECTORY_SEPARATOR . $this->urlHash . self::SLUG_SEPARATOR . $extension; $this->makeTransformedFileName($path); $this->image->setFilename($this->getRealFilename()); } private function makeDeepPath($string) { $path = ''; for ($x = 0; $x <= min(self::PATH_DEPTH, strlen($string)); $x++) { $path .= ($path ? DIRECTORY_SEPARATOR : '') . $string[$x]; } return $path; } private function getBaseDir() { return env('STORAGE_FILES_DIR', self::FILES_FOLDER); } private function fetchOriginal() { $contents = file_get_contents($this->url); $this->filesystem->put($this->fileName, $contents); } private function processTransformations() { if ( ! $this->wasTransformed) { foreach ($this->request->except(self::URL_PARAMETER) as $command => $value) { $this->image->transform($command, $value); $this->wasTransformed = true; } if ($this->wasTransformed) { $this->image->save($this->getRealFilename($this->getTransformedFileName())); } } } private function getTransformedFileName() { return $this->transformedFileName; } private function getExtension($fileName) { return pathinfo($fileName, PATHINFO_EXTENSION); } private function makeFileName($fileName) { return pathinfo($fileName, PATHINFO_FILENAME); } private function getRealFilename($fileName = null) { return $this ->filesystem ->getDriver() ->getAdapter() ->applyPathPrefix($fileName ?: $this->fileName); } private function getOriginalFileName() { return basename($this->url); } private function makeTransformedFileName($path) { $path = $path ? $path . DIRECTORY_SEPARATOR : $path; if ($this->transformedFileName) { return $this->transformedFileName; } $extension = $this->getExtension($this->fileName); $filename = $this->makeFileName($this->fileName); foreach ($this->request->except(self::URL_PARAMETER) as $key => $transformation) { $filename .= self::TRANSFORMATION_SEPARATOR . $key . self::TRANSFORMATION_SEPARATOR . $transformation; } return $this->transformedFileName = $path . str_slug($filename) . self::SLUG_SEPARATOR . $extension; } private function getFinalFileName() { if ($this->wasTransformed) { return $this->transformedFileName; } return $this->fileName; } public function setImage($image) { $this->image->setImage($image); } public function getImage() { return $this->image->getImage(); } public function getFileName() { return $this->fileName; } public function setResponse($response) { $this->response = $response; } } </code></pre> Nothing really complicated here, it tries, locally, to find the original file, if it's not present it will download the file, and this step may take a long time, then it will check if it's a valid image and process all transformations on it. The original file URL will be used to create a hashed name (SHA1) for our file, stored in a deep path in your Storage folder. Here's an example of a processed file: <pre class="prettyprint"><code >storage/app/files/6/4/d/9/a/1/6/4/1/64d9a16411fc278a75915a895025994277dd1472-width-60-blur-3.png </code></pre> And the original version: <pre class="prettyprint"><code >storage/app/files/6/4/d/9/a/1/6/4/1/64d9a16411fc278a75915a895025994277dd1472.png </code></pre> We'll keep everything stored because, even if the file is not in the cache anymore, it will still be really fast to download it again. We also need a File Finder, which role is basically look for a file and recursivelly create directories: <pre class="prettyprint"><code ><?php namespace App\Services; use Illuminate\Contracts\Filesystem\Factory as Filesystem; class FileFinder { private $filesystem; public function __construct(Filesystem $filesystem) { $this->filesystem = $filesystem; } public function find($file, $recursivelyCreateDirectories = true) { $dirname = dirname($file); if ( ! $this->filesystem->exists($dirname)) { if ($recursivelyCreateDirectories) { $this->filesystem->makeDirectory($dirname, 0775, true); } return false; } if ( ! $this->filesystem->exists($file)) { return false; } return true; } } </code></pre> Our Cache class basically uses Lumen (Laravel) cache, and it exists only because we need to create a cache key based on the URL query: <pre class="prettyprint"><code ><?php namespace App\Services; use Carbon\Carbon; class Cache { /** * @var \Illuminate\Cache\CacheManager */ private $cache; public function __construct() { $this->cache = app('cache'); } public function get($request) { return $this->cache->get($this->makeCacheKey($request)); } public function put($request, $file) { if ($file) { $this->cache->put( $this->makeCacheKey($request), $file, Carbon::now()->addMinutes(env('CACHE_EXPIRE_MINUTES')) ); } return $this; } private function makeCacheKey($query) { $key = ''; foreach($query->query() as $name => $value) { $key .= "$name=$value&"; } return $key; } } </code></pre> And, finally our Image Processor, which uses Intervention: <pre class="prettyprint"><code ><?php namespace App\Services; use Intervention\Image\ImageManager; class Image { private $manager; private $image; private $values; private $fileName; public function __construct(ImageManager $imageManager = null) { $this->manager = $imageManager ?: new ImageManager(['driver' => 'imagick']); } public function setFilename($fileName) { $this->fileName = $fileName; $this->makeManager(); return $this; } public function transform($command, $value = null) { $this->makeManager(); $this->values = explode(' ', $value); $zero = $this->getValue(0); $one = $this->getValue(1); $two = $this->getValue(2); $three = $this->getValue(3); $four = $this->getValue(4); if ($command == 'width') { $this->image->resize($zero, null, $this->checkConstraint($one ? $one : 'aspect')); } elseif ($command == 'blur') { $this->image->blur($this->getValue(0, 15)); } elseif ($command == 'brightness') { $this->image->brightness($this->getValue(0, 25)); } elseif ($command == 'canvas') { } elseif ($command == 'circle') { } elseif ($command == 'colorize') { $this->image->colorize($zero, $one, $two); } elseif ($command == 'contrast') { $this->image->contrast($zero); } elseif ($command == 'crop') { $this->image->crop($zero, $one, $two, $three); } elseif ($command == 'destroy') { } elseif ($command == 'ellipse') { } elseif ($command == 'encode') { } elseif ($command == 'exif') { } elseif ($command == 'filesize') { } elseif ($command == 'fill') { $this->image->fill($zero, $one, $two); } elseif ($command == 'flip') { $this->image->flip($zero); } elseif ($command == 'fit') { $this->image->fit($zero, $one, $this->checkConstraint($two), $two); } elseif ($command == 'gamma') { $this->image->gamma($zero); } elseif ($command == 'getCore') { } elseif ($command == 'greyscale') { $this->image->greyscale(); } elseif ($command == 'height') { } elseif ($command == 'heighten') { $this->image->heighten($zero, $this->checkConstraint($two)); } elseif ($command == 'insert') { $this->image->insert($zero, $one, $two, $three); } elseif ($command == 'interlace') { $this->image->interlace($zero == 'yes'); } elseif ($command == 'invert') { $this->image->invert(); } elseif ($command == 'iptc') { } elseif ($command == 'limitColors') { $this->image->limitColors($zero, $one); } elseif ($command == 'line') { $this->image->line($zero, $one, $two, $three); } elseif ($command == 'make') { } elseif ($command == 'mask') { $this->image->mask($zero, $one == 'alpha'); } elseif ($command == 'mime') { } elseif ($command == 'opacity') { $this->image->opacity($zero); } elseif ($command == 'orientate') { $this->image->orientate(); } elseif ($command == 'pickColor') { } elseif ($command == 'pixel') { $this->image->pixel($zero, $one, $two); } elseif ($command == 'pixelate') { $this->image->pixelate($zero); } elseif ($command == 'polygon') { } elseif ($command == 'reset') { } elseif ($command == 'resize') { $this->image->resize($zero, $one, $this->checkConstraint($two, $three)); } elseif ($command == 'resizeCanvas') { $this->image->resizeCanvas($zero, $one, $two, $three, $four); } elseif ($command == 'response') { $this->image->response($zero, $one); } elseif ($command == 'rotate') { $this->image->rotate($zero, $one); } elseif ($command == 'save') { } elseif ($command == 'sharpen') { $this->image->sharpen($zero); } elseif ($command == 'text') { $this->image->text($zero, $one, $two); } elseif ($command == 'trim') { } elseif ($command == 'widen') { $this->image->widen($zero, $this->checkConstraint($one)); } elseif ($command == 'width') { } return $this; } public function setImage($image) { $this->image = $image; return $this; } public function getImage() { return $this->image; } function __call($name, $arguments) { $this->image = call_user_func_array([$this->image, $name], $arguments); return $this; } private function getValue($pos, $default = null) { if (isset($this->values[$pos])) { return $this->values[$pos]; } return $default; } private function checkConstraint($zero, $one = null) { $constraint = function ($constraint) use ($zero, $one) { if ($zero == 'upzise' || $one == 'upzise') { $constraint->upsize(); } if ($zero == 'aspect' || $one == 'aspect') { $constraint->aspectRatio(); } }; return $constraint; } private function makeManager() { if (file_exists($this->fileName) && ! $this->image) { $this->image = $this->manager->make($this->fileName); } } } </code></pre> As you can see, easy stuff, I just mapped some of the Intervention's methods. Why not create a package/module/bundle for Lumen? Also easy, and you can do it!, but in this particular case I prefer not to waste some precious miliseconds. <h3>That's it. Enjoy!</h3>