The future of online backup

Image representing Primary Technology as depic...
Image via CrunchBase

Primary Technology required a business case for investing more money into our online backup solution.  That was the idea anyway but I actually came to the conclusion that online backup is not a good area to invest.  In this blog post I am going to try to justify my decision and hopefully give you an insight to why online backup is great for the consumers but costly for the provider.

Scaling

Once you get to many terrabytes of data things become a bit more difficult to manage.  Elastic storage services do exist and if I’m honest storage isn’t that much of a problem.  One thing that is a problem for smaller online backup providers is bandwidth bottlenecks.  If my software only has a single point of entry then I have a) a single point of failure and b) an awful bottleneck.  To overcome this problem can cost quite a bit of money, money that would need to be reflected onto schools so scaling costs money and the notion that it saves money over a distributed model is actually mathematically proven to be untrue (only in this instance with a schools data requirements).  So don’t buy into the cloud just yet….

Encrypted data

Encrypting your data is a great idea but it makes it a real nightmare for the offsite backup provider as you can’t use a process called “de-duplication”..  De-duplication in it’s simplest form is the process of being able to store one copy of “Michael Jackson – Thriller.mp3” instead of 10 copies.  A provider can look at the file you are trying to upload and say “Whoa I already have that file, no need to upload it”.  This saves you bandwidth and the provider space..   As soon as you encrypt de-duplication is not possible as it’s not possible for the provider to look at the file and make logic based decisions.

Competitors and demand

I love the service dropbox/syncplicity etc. provider, they are clean, relatively low risk, fast and more importantly, FREE!  Great for teachers and students, terrible for IT admins looking to backup their databases.  The thing is when you think about data storage in general most of the data is being onto the internet and stored in web based applications such as PrimaryBlogger or PrimaryPad, so the value to a school of local storage is in decline.  Here is a comparison of online backup software with an idea of cost.  We considered writing our own but then we remembered that we create web based tools so we knew our own products in a way conflicted with this investment..  Another factor that is lowering demand for the service is the reduced risk of fires and burglaries.  I did a bunch of research into how these have changed, mostly using UK stats that are released by the Govt (and I’m greatful for this open data).

This is a diagram of a Wikipedia:Peer-to-Peer ...
Image via Wikipedia

The future

There’s no doubt in my mind that a peer to peer method is by far the best method for schools to be backing up data.  I can’t release the stats that I gathered but if you are considering writing a remote backup application you will want to use this method.  It has the obvious pitfalls and you will still need a single point of reference for this like emails to be distributed from but it is the lowest overall cost solution for the problem of online backup.  The basic log of peer to peer is that as hard disks prices drop and inter-school data links improve the real term cost of storing data between organizations falls through the floor.  Because it’s now so cheep to store data you can store your data in 5 other schools.  Obviously this data will need to be encrypted but that’s not a problem because we can afford the disk space.  

Want to know more about my research?

This post is just a brief overview of my research that I put into online backups, I have a way more verbose article that is owned by Primary Technology.  I may be able to come in and talk to you about my findings if you want to know more!

If you are still using tapes inside of your school then you need to really have a long sit down think about what happens if those tapes are forgotten on site or lost/stolen with the data or don’t actually work(as is nearly always the case).  We(PrimaryT) do offer an online backup service that costs ~£200 per server per year, often a great saving if something goes wrong..

Enhanced by Zemanta

Remote / Off-site self hosted backup comparison

Every 2 years Primary Technology re-engineers all of it’s services and this year I have been tasked with re-engineering our School offsite / remote backup service. Our current service works well but there are a few things we would change and we feel our current provider isn’t really giving us the support we need to make these changes.

The first part of any successful re-engineering job is to gather a comparison of all the available products on the market.

I reviewed these products/pieces of software:

  • Ahsay
  • Attix
  • Bacula
  • Crashplan
  • Dropbox
  • iDrive Backup
  • Intronis
  • One Safe Place
  • Remote-Backup.com
  • Rysnc.net
  • SOS Online Backup
  • Vembu
  • XZBackup
  • Amanda/Zmanda

And I reviewed them based on this following requirements of functionality:

  • Can the client transfer data and config over HTTPS?
  • Does a Windows client Exist?
  • Is the software brandable?
  • Can I perform a system state backup on the clients?
  • Can it export database bricks / incremental / differential backups
  • Does the client have a scheduler?
  • Can the client stop itself at a specified time
  • Can the server push changes to the client backup set?
  • Can the client easily select which files to upload?
  • Does the server save old versions of files?
  • Is the software open source?
  • Does the server have overview reporting?

And my notes which probably contain the most important informative information:

Enhanced by Zemanta

Syncplicity & its place in education (Software review)

Syncplicity is a small client application that periodicly uploads files to the internet so you can access them from another location. Accounts are free, limited to GB. Accounts are per email address.

Installation and configuration is unbelievably easy so there are no technical barriers.
I’m backing up my “my documents” as if I was a pupil at home on a netbook.
I think if syncplicity allowed shibboleth based logon to this service then they could be onto a winner. A school which had no servers could use this to backup and save pupils work.
There are of course a few advantages of having servers and those should not be ignored. I’m also not so sure on Becta’s stance on something like this, inevitably private information will be uploaded and the question has to be asked. Can simplicity be trusted any more or less than any other provider ie the remote backup services from Primary Technology? Primary Technology offer the same service but an annual charge of £200.
Visit syncplicity to download and check it out, I’m going to leave it running on my computer over the next few months just to see how it goes.