Cloudydict is a simple python module that makes accessing cloudfiles as easy as using a dictionary. With cloudydict, gone are worries about proprietary APIS and vendor lockin. No matter whom your vendor, everything looks like a dictionary.
Its on Pypi. Just run this:
$ pip install --upgrade cloudydict
Assuming you've setup your .boto configuration file, working with S3 and cloudydict is as simple as:
>>> from cloudydict import cloudydict >>> d = cloudydict('s3:cloudydict-demo') # Choose your own bucket name
The returned cloudydict object acts like a dictionary. Items can be added.
>>> d['a'] = 'Hello world' >>> d['b'] = 'How are you?' >>> d.keys() [u'a', u'b'] >>> 'c' in d False >>> 'a' in d True >>> 'd' in d False >>> d.update(dict(d='你好世界')) >>> 'd' in d True
Retrieval from cloudydict is lazy. Values are returned as cloudydict objects that can be treated either as strings or files.
>>> d['a'] <cloudydict.s3.RemoteObject instance at 0x2466b00> >>> len(d['d'].read()) 12 >>> import sys, codec >>> sys.stdout = codecs.getwriter('utf8')(sys.stdout) >>> print d['d'].decode('UTF-8') 你好世界 >>> d.values() [<cloudydict.s3.RemoteObject instance at 0x2362ea8>, <cloudydict.s3.RemoteObject instance at 0x235cf38>] >>> d.items() [(u'a', <cloudydict.s3.RemoteObject instance at 0x23c2a70>), (u'b', <cloudydict.s3.RemoteObject instance at 0x23c2560>)]
__setitem__ calls against cloudydict will accept a python string, file or another cloudydict
RemoteObject. If an assignment is made from a compatible remote object, cloudydict will perform the copy using the cloud's underlying bucket to bucket copy commands.
>>> d['c'] = d['b'] # Does not download d['b'] >>> str(d['c']) # Well, this does 'world'
Lastly cloudydict supports a number of non standard but cloud specific operations. Not all cloud vendors support these, but for those that do, you can make files public and retrieve their url.
>>> d['c'].make_public() >>> d['c'].url cloudydict-demo.s3-website-us-east-1.amazonaws.com/c
Cloudydict comes with a Django Storage Server. To free yourself from using your local file system to service content, just make the following small changes to your settings file:
DEFAULT_FILE_STORAGE = 'cloudydict.django_storage.StorageFromSettings' CLOUDYDICT_STORAGE_SERVER_SECRETS = ('service:container', ... credentials ... ) # For S3 this would looke like CLOUDYDICT_STORAGE_SERVER_OPTIONS_SECRET = ('s3:blab-blab', <aws user>, <aws secret key>)