Cloud based storage made super simple

Cloudydict is a simple python module that makes accessing cloudfiles as easy as using a dictionary. With cloudydict, gone are worries about proprietary APIS and vendor lockin. No matter whom your vendor, everything looks like a dictionary.

Installation

Its on Pypi. Just run this:

$ pip install --upgrade cloudydict

Tutorial

Assuming you've setup your .boto configuration file, working with S3 and cloudydict is as simple as:

>>> from cloudydict import cloudydict         
>>> d = cloudydict('s3:cloudydict-demo') # Choose your own bucket name

The returned cloudydict object acts like a dictionary. Items can be added.

>>> d['a'] = 'Hello world'         
>>> d['b'] = 'How are you?'          
>>> d.keys()         
[u'a', u'b']    
>>> 'c' in d
False
>>> 'a' in d
True
>>> 'd' in d
False
>>> d.update(dict(d='你好世界'))
>>> 'd' in d
True

Retrieval from cloudydict is lazy. Values are returned as cloudydict objects that can be treated either as strings or files.

>>> d['a']
<cloudydict.s3.RemoteObject instance at 0x2466b00>

>>> len(d['d'].read())
12
>>> import sys, codec
>>> sys.stdout = codecs.getwriter('utf8')(sys.stdout)   
>>> print d['d'].decode('UTF-8')
你好世界
>>> d.values()
[<cloudydict.s3.RemoteObject instance at 0x2362ea8>, 
 <cloudydict.s3.RemoteObject instance at 0x235cf38>] 
>>> d.items()
[(u'a', <cloudydict.s3.RemoteObject instance at 0x23c2a70>),
 (u'b', <cloudydict.s3.RemoteObject instance at 0x23c2560>)]

__setitem__ calls against cloudydict will accept a python string, file or another cloudydict RemoteObject. If an assignment is made from a compatible remote object, cloudydict will perform the copy using the cloud's underlying bucket to bucket copy commands.

>>> d['c'] = d['b']  # Does not download d['b']
>>> str(d['c'])   # Well, this does
'world'   

Lastly cloudydict supports a number of non standard but cloud specific operations. Not all cloud vendors support these, but for those that do, you can make files public and retrieve their url.

>>> d['c'].make_public() 
>>> d['c'].url
cloudydict-demo.s3-website-us-east-1.amazonaws.com/c

Cloudydict comes with a Django Storage Server. To free yourself from using your local file system to service content, just make the following small changes to your settings file:

DEFAULT_FILE_STORAGE = 'cloudydict.django_storage.StorageFromSettings'

CLOUDYDICT_STORAGE_SERVER_SECRETS = ('service:container', ... credentials ... ) 

# For S3 this would looke like
CLOUDYDICT_STORAGE_SERVER_OPTIONS_SECRET = ('s3:blab-blab', <aws user>, <aws secret key>)