Blogs

Backing up your files from your EC2 machine to amazon S3 bucket

June 19, 2012 by Arvind Das

Amazon S3 bucket are containers for objects (the files you store on s3). I would explain how to make back up of your files from EC2 machine to amazon s3 bucket.

  • Create a S3 account. If you already have an account ,move to step two. Else click here
  • Download your aws access key and aws secret access key.
    • You can download them from your security credentials menu.
  • Now download and install s3fs. You can Download
  • Install these prerequisites before installing.
  • yum install gcc 
    yum install libstdc++-devel 
    yum install gcc-c++ 
    yum install fuse 
    yum install fuse-devel 
    yum install curl-devel 
    yum install libxml2-devel 
    yum install openssl-devel 
    yum install mailcap 
    
  • Installation Steps
    • Unzip the downloaded s3fs-1.61.tar.gz and follow steps further
    tar xvzf s3fs-1.61.tar.gz 
    cd s3fs-1.61/ 
    ./configure --prefix=/usr 
    make 
    make install (as root) 
    
  • Create a file named .passwd-s3fs in your root directory.
  • Vi .passwd-s3fs
    
  • Write your access key id and secret access key in the format as aws_access_key:aws_secret_key
    • Change its mode as “chmod 600 ~/.passwd-s3fs “ so that no other group has access to it.
    • save and exit
  • In this method we will create a mount point for your bucket in your machine, So that everything you put in that directory(mount point) will be automatically moved to S3 bucket.
    • To mount your S3 bucket run
    • s3fs -o allow_other bucket_name /mnt/your_directory_name
      
    • This is the directory where you will put your files to move to S3
    • To copy files to your S3 bucket run rsync. Read the note below before executing this command
    • rsync -av  /you_directory_path_to_files  /mnt/your_directory_name
      
    • [Note] --delete command is to ensure that if you delete files from your bucket it will be deleted from your machine too.
  • If you have issues in building “make” or ./config your s3fs , then you may be running an older version of FUSE. Update that manually as
    • Download Latest Fuse
  •                    tar xvzf fuse-2.8.6.tar.gz
                        cd fuse-2.8.6
                       ./configure –prefix=/usr
                        make 
                        make install
    
  • To check if installation is correct, run
  • modprobe fuse
    

    If it runs clean, OK

    Else it may show an error

    FATAL: Could not load /lib/modules/2.6.16.33-xenU/modules.dep: No such file or directory

  • In such case, download http://s3.amazonaws.com/ec2-downloads/ec2-modules-2.6.16.33-xenU-x86_64.tgz extract
  • tar xvf ec2-modules-2.6.16.33-xenU-x86_64.tgz
    cd lib/modules
    cp -r * /lib/modules/
    

Comments

Leave a comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Follow Us

Recent Entries