File Conveyor is a daemon written in Python to detect, process and
Designed to sync files to CDNs. Amazon S3 & CloudFront and Rackspace Cloud Files, as well as any Origin Pull CDN or (S)FTP Push CDN are supported.
This daemon is designed to discover new, changed and deleted files via the operating system's built-in file system monitor. After discovering the files, they can be optionally be processed by a chain of processors – you can easily write new ones yourself. After files have been processed, they can also be synced (“transported”) to a server.
Discovery of changes of the filesystem happens instantaneously through inotify on Linux, through FSEvents on Mac OS X and through polling on other operating systems.↓
Processors are simple Python scripts that can change the filename and apply any sort of processing to the file's contents. E.g.:
filename, which can convert spaces to underscores or dashes
google_closure_compiler, to compress JS files using Google Closure Compiler
image_optimizer, which is capable of optimizing images without losing quality, i.e. it stores images more efficiently (similar to the service that smush.it provides).
link_updater, which should be used to update the URLs in CSS files so that the CSS file refers to the images, fonts … on the CDN
unique_filename, to give files a unique name based on its last modification time (
mtime) or MD5 hash
yui_compressor, to compress CSS and JS files using the YUI Compressor
Transporters are simple threaded abstractions around Django Custom Storage Systems. Currently, the following transporters (and their corresponding protocols) are available:
Configuring File Conveyor happens through a simple XML format. After looking at it for a couple of minutes, it should already make sense. Here's a sample configuration file.
To further simplify usage of File Conveyor, the following CMSes explicitly support File Conveyor:
You can also clone the project with Git by running:
$ git clone git://github.com/wimleers/fileconveyor.git
Or, if you have pip installed, you can do this:
pip install -e git+https://github.com/wimleers/fileconveyor@master#egg=fileconveyor
GPL v2 or UNLICENSE
Wim Leers — http://wimleers.com/contact