Data security,backup strategy and why it matters

I have been using a computer for a very very long time (The first one was back in 1977 à VT100 hooked to a PDP 11) in 1982 the First “home computer with 2 5” floppy disks. Back then the process was easy get a new floppy, copy, and then store, avoid any magnets or hole punchers and you where fine. Later in 1982 we to the first HD with 10MB suddenly that was plenty of floppy disks.

Fast forward from 1995 onwards I bought every year a new hard disk for 300$ Back then 1GB. Backup was always at some point a copy to another disk but it was not really organized or structured

Fast forward again to 2001. We purchased a Squeezebox music player and the slimserver music server. Suddenly I had to rip all my CD’s onto the PC and store them. Losing all the data was just to big of a Risk and the files where taking way too much space.

Error No1 RAID I created a home server running the music server and storing all the files music (and planned for all the other documents as well). I built a very nice RAID 1 array with 3x300MB diks with a Spare disk in the case of error and I though I was safe. Most of the time it is pretty safe but at some point in the upgrade, modification of the server I did manage to corrupt the raid partition. BYE BYE all files this was a big lesson and cost me 4 full day’s to go back and rip all the CD’s again.

Error No2 External HD Lesson learned I got a big external USB hard drive to store all the data just in case. This worked but it takes a long time as the volume grows. even for small changes such as music tags

Enhancement 1 As the volume of Data is growing and the use of the music in the family the data security is improved. 2 Disks in raid 1 for redundancy and replicated with Sync on the same server to a single spare disk. Now 1 disk can fail or the Raid can fail and I still have a copy

Enhancement 2 Added a 2nd server in my network to replicate the data onto a remote machine as well. Just in case the main server fails. Which eventually happened with the power supply dying on a Thursday night. With next day business support the part got here Monday night. Thank good there was another server to run the music server and alarm clocks off.

Enhancement / Error 3 Disasters can happen and still having the 2 servers close to each other I decided to put a remote server at my brothers place and replicate the data with Rsync. Technically possible but quiet a pain to manage and rather slow give our up links

Enhancement 4 As the amount of data with music and now photo’s is growing fast I am on the outlook for better solutions. I came across Crashplan. Their service is great and offers some free replication and paid on-line storage. After testing it I was hooked on how easy it is. Rsync is better if one needs access to the files in 2 places but for real backup it is great. It can manage version and everything else is automated and easy to define. Now I create 1 copy locally as Crahplan backup

Enhnacement 5 Using Crashplan I did go for their online storage offer. I can now store all my data online. All that is very easy. The limitation is my internet connection and it will take about 100 day’s to upload all the data but increments should be faster after that to update only increments

Summray so FAR  I should have it all covered right I have Raid 1 (2 copies on the same disks), 1 copy onto another dedicated disk(rsync every 6hours), 1 copy to remote server (rsync every 24hours). And now a partial backup online.This should be perfect but it is not. I still have a workstation with plenty of local data such as my work disk for current photo shoots and I have laptop for work.

Enhancement 6 & Disaster do happen Give the ease of use I implemented the backup of my workstation to my server using Crashplan. No setup of Rsync all is automated and works for my data disk. Disaster strikes where you are not ready 9 days before the end of my 3year warranty the main hard drive of my workstation was showing SMART errors and failed sectors. I ordered a spare disk and did clone it and was able to work and get the replacement disk from Dell a week later. Nothing Dramatic except I had to order a spare HD clone it 8 hours, make minor changes run it for a week and clone it again on the replacement disk another 5 hours. Maybe a Raid 1 would be solution for that system too.

Disaster 2 After the disk crash I was sure I had seen all the possible problems and the incident was not dramatic. I could probably live with the current weak points in the system. How wrong I was. I had already installed Crashplan on my laptop but not run the backup due to missing disk space on the server. And there DISASTER did strike again. On the train back from a Partner I try to start windows and it has lost my complete windows profile : Every document, browser setting, … everything stored in Document & settings was done. I have always been careful and most of the data was on a different partition and folder. still I lost some data that is painful to recover. That has now been added to my Crashplan backup to my home server.

Conclusion  KISS Keep It Simple Stupid

  1. Don’t trust the user (YOU) users will forget and are not reliable
  2. If the data is important keep multiple copies
  3. The number copies should increase if you need the fast in case of problems
  4. Use what exists don’t try to be smarter than all the people out there

Handling Compact Flash and Memory card

Camera Memory cards are another very risky asset. They all lookalike and it is easy to forget what you have done with it. It happened to me and I copied 2x the same card to my PC and I delete 1 that I had no copies of. And yes this was of the weeding of my cousin. Luck was on my side and I had a pretty good process in place

  1. In the camera bag I have 2 distinct spots for CF cards FULL and EMTPY. The Empty ones are also really empty (formatted in the camera)
  2. After the shoot I copy them ASAP from the Cards to a portable HDD / Card Reader
  3. The copied cards go to location 3 in my camera bag (full, copied but only once, maybe the HD could fail)
  4. When I get home I copy all the pictures onto my workstation work disk
  5. I delete all the un-sharp and bad ones (check the bin before empty just in case I delete a wrong pic. It happens when you manage 1,000s of pictures at once
  6. Copy all the remaining valid on to a folder on the server
  7. Erase the CF cards and format they go back to the free card location in the camera bag, ready for the next session
  8. Delete other not great pictures from the workstation
  9. Once only acceptable pictures remain copy them again onto the server (by then the server has made 2 copies(as above)
  10. Any modified picture that has some post processing goes into a dedicated folder
  11. Copy from workstation to server once processing is completed

Here I should mention that all the picture folders are managed by crashplan on the server and workstation to take extra copies. Yes I have been burned by this once and I may be overdoing it. I am happy with the current solution as it minimizes risk of accidentally deleting pictures. While on the go I may actually copy the same Card 2x just to be sure I have a copy on my portable HD and the Card.



Data security,backup strategy and why it matters — 1 Comment

  1. I thought I had planned for everything. I was wrong. Last night my whole network did collapse. I am still investigating why and what caused it.
    My guess is that i was :
    1. a bad cable somewhere
    2. a loop somewhere in the network (maybe a wire-wireless double connection
    3. A dying switch somewhere(some have been running for 10years)

    I had to restart all my switches and access points like 15times to cure it for now.
    If you have any suggestions on how to troubleshoot and prevent it. please tell me.

    What I won’t do is to create 2 physical netowkors

Leave a Reply

Your email address will not be published. Required fields are marked *