A strategy for efficient inclusion of JPGs with changed captions in incremental backups of all files RRS feed

  • Question

  • That's a mouthful of a subject. Let me explain:


    1. Frequent full backups may be the safest way to ensure recovery of recent files when a system has crashed but can be too onerous for the computer and user to do routinely.

    2. Incremental backups should be faster than full backups. Not many files should have changed between backups. It has been common guidance that one should do a full backup (weekly or monthly) and incremental backups (daily or weekly), depending on the intensity with which the computer is used.

    3. IPTC tag editors for image files, e.g. JPG, typically do not or cannot change the Last Modified time and may not change the file size. This is true of Google Picasa2 caption editing and many others. Without the Check File Contents option, SyncToy will miss that a file has changed. Picasa2's own Backup system also fails to detect caption changes.

    4. Incremental backup using the Check File Contents option is probably slower than a full backup because every file pair must be compared before each changed file is copied.

    5. Most applications and system files other than the aforementioned image editors do change the Last Modified time.


    As an alternative to frequent full backups, I suggest the following with SyncToy, and welcome any endorsements or criticism:


    1. Set up the destination folder under which all file types for a source drive or folder are to be placed.

    2. Define a Folder Pair in SyncToy to echo all files, excluding *.JPG (and any others you modify using apps that do not change the Last Modified time) with the Check File Contents option unchecked

    3. Define a Folder Pair in SyncToy to echo all *.JPG files only (and any others as above) with the Check File Content checked.

    4. Run both Folder Pairs. Both write to and compare the same folders without conflict and you end up with a backup set that is a mirror.

    5. Check that this is so by making another Folder Pair between the same folders, this time including all files, excluding none, and Preview. There should be no files in the backup set other than those changed by the system since step #4.  

    6. On your incremental backup schedule, run both folder pairs. This should maintain the full backup as a mirror.

    7. To confirm the mirror is maintained, occasionally rerun the Folder Pair Preview defined in step 5 with the Check File Contents turned on. This is going to be very slow but may flag other file types that may need to be moved to the exclusion list in the Folder Pair defined in #2 AND to the inclusion list for the one in #3.

    8. If you want to snapshot this backup set and start a new one (e.g. keeping a monthly snapshot to go back to), I think you could rename its root folder, recreate the target folder that the Folder Pairs use, and copy the SyncToyxxxxxx.dat files from the renamed folder to it.


    Now I have only just set this up on My Documents and it seems to work initially. I will expand its scope to include Documents & Settings, maybe the whole drive, as I gain more confidence with it.


    Speed would be enhanced by minimising the number of files to be compared byte-for-byte. If the changing files with constant Last Modified were confined to one folder (say, Working) and its sub-folders, maybe a better (faster) Folder Pair definition would include those file types in selected folders with Check File Contents on and the other definition would include all files in all folders with Check File Contents off. This will become apparent over time by using what I have initially defined and will depend on how one works with files and what apps are being used.


    What do you think?



    Saturday, August 30, 2008 8:45 PM

All replies

  • Sounds like a good plan to me. Great job capturing it Tom - would you be interested in working on synchronization technology? ;-). Be VERY careful with that step#8 above. It can mess things up if you don't do it properly. I would recommend against that step, but other than that, looks like a good setup to me.


    This issue with media editors not modifying timestamps is one of the top things for us to consider fixing in upcoming versions of the Microsoft Sync Framework (which provides the underlying file sync engine for SyncToy). Stay tuned for updates on that.


    Tuesday, September 2, 2008 4:01 AM
  • Thanks, Ashish. I'll let you know how this is working after a while.



    Tuesday, September 2, 2008 2:04 PM