Getting more out of your storage on a VM

I’ve got a VM over in the US; really useful to have due to good connectivity and the ability to run things on it that won’t be interrupted. One thing I run is undulatus, my twitter client. It logs all the tweets I see in a CouchDB database, and also builds a bunch of indexes so I can search them.

That takes up a lot of room. I’ve got around 4 million tweets on file; that takes up 21GB (including indexes.) Unfortunately my VM only has 25GB of disk space, so I was in trouble.

This handy article saved the day; I made a ZFS filesystem in a disk file, and then set up zfs-fuse. I set the filesystem size at 20GB, turned on transparent compression (in gzip-9 mode) and what was taking up 21GB is now taking up 5.5GB. Pretty neat; it’s actually quite fast too.

I’m not going to pretend it’s not an ugly hack, but it worked really well.

This entry was posted in Uncategorized. Bookmark the permalink.

One Response to Getting more out of your storage on a VM

  1. James French says:

    ZFS fuse is actually quite nice, I was pleasantly surprised when I tried it out last year. Has a few minor quirks but nothing that I can think of that’ll be a huge pain in this instance.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s