#!/usr/analog/human === ElasticDrive README === ElasticDrive is a network block device based upon the Amazon S3 (Simple Storage Service). ElasticDrive provides a caching block device driver which pushes blocks to and from S3 as if they were being written to a local block device. ElasticDrive provides this service through a virtual NBD service. The NBD service translates from standard NBD to S3 packets transparently, so that the client (the kernel) sees a generic block device. NBD is supported on almost every linux kernel (including EC2). Simply modprobe nbd, and you are ready to use ElasticDrive. ElasticDrive is not intended to replace your existing hard drives or network filesystems directly. ElasticDrive is intended to provide seamless backup, RAID target devices, or backing stores for higher level distributed filesystems. === config.ini === The ElasticDrive configuration file is a standard INI file, just like in a Windows application. No XML here! You configuration file looks like this: [s3] AWS_ACCESS_KEY_ID = 1234567890ABCDEFGHIK AWS_SECRET_ACCESS_KEY = RXRXRXRXR123890713287525ABCDEFGGHIJKLzzz BUCKET=the.name.of.the.bucket.to.create #Stripe is the block device stripe size. 65536 is recommended. #This is NOT the same as the block size STRIPE=65536 #Volumesize is expressed in bytes, and should be a multiple of STRIPE VOLUMESIZE=16777216 [server] #PORT to listen on port=9999 #Show a ludicrous amount of debug data in the logs? debug=0 #cachesize is the max number of stripe elements to cache. 32 is about #2M here. We recommend large caches to prevent blocking the device. cachesize=1024 #The amount of time to store a local cache stripe. 60 is a good default. cachettl=60 Most of the configuration should be left as is. Just modify the VOLUMESIZE, BUCKET, AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. If you are creating a RAID Stripe, the VOLUMESIZE should match the size of your physical hard drive. === USE CASES === --- Direct Network Storage --- To use ElasticDrive directly as a storage device, simply run the elasticdrive binary from /opt/elasticdrive-x.x.x, then create a filesystem on it, and use it like a normal filesystem. After unmounting, the drive can be remounted on any system that can see S3. > cd /opt/elasticdrive-0.1.6 > ./elasticdrive > nbd-client bs=4096 localhost 9999 /dev/nbd0 > mke2fs -b 4096 /dev/nbd0 > mkdir /s3 > mount /dev/nbd0 /s3 --- Virtual Backup Tape --- You can also copy an existing filesystem directly to ElasticDrive: > cd /opt/elasticdrive-0.1.6 > ./elasticdrive > nbd-client bs=4096 localhost 9999 /dev/nbd0 > dd if=/dev/sda1 of=/dev/nbd0 bs=4096 count=sizeofyourdrive Note that this method of backing up preserves all the little things about your filesystem (like atime,mtime,hardlinks) which get missed by webdav or FUSE based storage. --- Software RAID (preferred) --- Software RAID is our favourite way to use ElasticDrive. By mounting a virtual RAID device, you can have pervasive remote storage which keeps a remote hot copy of your data, while using the local drive for high speed read/write access. The RAID mirror will catch up in the background with your filesystem changes. In the event of an unexpected failure, you have one side of a RAID mirror ready to recover with! This works especially well with EC2 instances ;) > cd /opt/elasticdrive-0.1.6 > ./elasticdrive > nbd-client bs=4096 localhost 9999 /dev/nbd0 > mdadm --create /dev/md0 -n2 --level 1 /dev/sda2 --write-mostly /dev/nbd0 > mke2fs -b 4096 /dev/md0 > mkdir /s3 > mount /dev/md0 /s3 In the event of a failure, on the rebooted machine (or a different machine): > cd /opt/elasticdrive-0.1.6 > ./elasticdrive > nbd-client bs=4096 localhost 9999 /dev/nbd0 > mdadm -A /dev/md0 /dev/sda2 /dev/nbd0 Note that the configuration files for ElasticDrive have to match in both cases (so back them up in advance to a remote location). --- Misc --- # Pricing Pricing is yet to be determined. # Additional Support Additional support and consulting services can be purchased from Enomaly Inc at www.enomaly.net or www.enomalylabs.com