locked
Robocopy and WHS RRS feed

  • Question

  • I'm hoping somebody familiar with the above can cast an eye over this for me.  I recently build a WHS and it works just fine.  I had an idea that I could store some work stuff on it.  I have a customer folder on my work server which has been there since 1999 and is now some 185GB.  As you can imagine it takes a long time to load over a network connection and back ups to a dat drive take forever.  Quite a lot of the customers folders have not been accessed for well over 12 months so I read about robocopy and thought why not shift everything older than 12 months to my WHS.  This will hopefully remove stuff I nolonger use and make the working directory much faster to access.  So reding all the docs on robocopy I've created a little batch job:

     

    Code Snippet

    ::
    :: Robocopy Job c:\AAAMOVE.RCJ
    ::
    :: Created by mike fish on Thur Apr 10 2008 at 20:05:32
    ::

    ::
    :: Source Directory :
    ::
     /SD:c:\robo :: Source Directory. This is where is copies from

    ::
    :: Destination Directory :
    ::
     /DD:f:\roboreceive\ :: Destination Directory. This is where it copies to

    ::
    :: Include These Files :
    ::
     /IF  :: Include Files matching these names
    ::  *.* :: Include all names (currently - Command Line may override) This means it copies all the files

    ::
    :: Exclude These Directories :
    ::
     /XD  :: eXclude Directories matching these names
    ::   :: eXclude no names (currently - Command Line may override) this means it won't exclude any directories

    ::
    :: Exclude These Files :
    ::
     /XF  :: eXclude Files matching these names
    ::   :: eXclude no names (currently - Command Line may override)
     /MINAGE:100 :: Excludes files with Last Modified Date - exclude files used since n.  this means any files newer than 365 days (1 years) will not be moved
    ::
    :: Copy options :
    ::
     /E  :: copy subdirectories, including Empty ones.
     /COPY:DATS :: what to COPY (default is /COPY:DAT).   This means copy permissions
     /MOVE  :: MOVE files AND dirs (delete from source after copying).
    ::
    :: Retry Options :
    ::
     /R:0  :: number of Retries on failed copies: default 1 million. This means it won't retry (problem files ignored to speed up the process)
     /W:0  :: Wait time between retries: default is 30 seconds.
    ::
    :: Logging Options :
    ::
     /LOG:aaalog
     /V  :: produce Verbose output, showing skipped files.
     /TS  :: include source file Time Stamps in the output.
     /FP  :: include Full Pathname of files in the output.
     /ETA  :: show Estimated Time of Arrival of copied files.
     /TEE  :: output to console window, as well as the log file.

     

     

    As you can imagine I'm not used to command line stuff and I don't want to get it wrong.  I will of course have a backup but if someone can advise if this is going to work I would be grateful.

     

    Mike

     

     


     

    Friday, April 11, 2008 7:45 AM

All replies

  • There's a very handy GUI for Robocopy which makes it very easy to create the scripts.

    Friday, April 11, 2008 10:05 AM
    Moderator
  • Hello Brubber

     

    I've had a look at that but it does not seem to have the MINAGE function so I think I'm stuck with command line.

     

    Mike

    Friday, April 11, 2008 10:23 AM
  • Hello

     

    You can do a first run with the "/L" option: This will essentially simulate without actually copying or deleting anything, and you can check the log file if it looks right.

     

    I would change:

    - Not copy the permissions (or if you do, then also copy the owner information)

    - Exclude a few files: SET /XF *.bak desktop.ini Thumbs.db

    - /MINAGE should probably be 365, not 100

     

    Regards, Martin

    Friday, April 11, 2008 11:00 AM
  • Hello Martin

     

    Thanks for the comments and well spotted with the 100/365 error I'd made.  I'm thinking with regard to permissions that I should change the attribute to read only, so I'll look at that next.  If I do a "/L" will it take just as long to run as if I were doing it for real.  Any guess on how long it will take to run per GB on a Gig link?

     

    Thanks

     

    Mike

     

    Friday, April 11, 2008 11:09 AM
  • Hello Mike

     

    On my system it does about 800 Megabytes per minute, so your data move should be done in a few hours. The /L option is a lot faster, because no files are moved across the network.

     

    Regards, Martin

     

    Friday, April 11, 2008 5:11 PM
  •  

    Hello again Martin

     

    Thanks for the info.  With /L is the output to the log the same?  The log is no doubt going to be very lengthy.  Does it group errors in one place say at the end?

     

    Thanks for your help, I think I'm going to run this on Sunday after my Saturday night backup.

     

    Mike

    Friday, April 11, 2008 7:40 PM
  • Hello

     

    I think the output is the same. There is a summary at the end of the log, which looks similar to this (sorry, it's in German):

     


    Code Snippet

                    Gesamt   Kopiert  Überspr. Nicht gef.  Fehler    Extras
     Verzeich.:       320         2       318         0         0         0
       Dateien:     17479       536     16943         0         0         1
         Bytes:  56.832 g   5.067 g  51.764 g         0         0    52.0 k
        Zeiten:   0:10:39   0:09:54                       0:00:00   0:00:44

    Geschwind.:             9146997 Bytes/s
    Geschwind.:             523.395 Megabytes/min

       Beendet: Sat Apr 12 18:14:24 2008

     

     

    "Fehler" means errors, so you see at least how many errors were encountered.

     

    The beauty about Robocopy is that it is "repeatable" (or "idempotent", in IT terms): If there were errors, just run the batch file again, and it will only copy the files that didn't work before.

     

    Regards, Martin

     

    Saturday, April 12, 2008 4:18 PM
  • Hello Martin

     

    Thanks for the info.  I'll let you know how it goes tomorrow.

     

    Danke für Ihre Hilfe (hope thats right :-))

     

    Mike

    Saturday, April 12, 2008 7:22 PM
  • UPDATE:

     

    I duly started the robocopy job this morning at just after 10:00 am.  I've just checked it and its still going having copied about 32GB so far.  At this rate it won't be finished before I start work in the morning!

     

    Mike

    Sunday, April 13, 2008 6:34 PM
  • Finally finished and logs reported no errors.

     

    If I run the job again will it delete from the destination those not in the source anymore?  If so how can I overcome that difficulty?

     

    Any help would be appreciated.

     

    Mike

    Monday, April 14, 2008 7:24 PM
  •  Mike Fish wrote:

    Danke für Ihre Hilfe (hope thats right :-))

    Down to the umlaut and all the caps at the right places ;-)

     

    Regards, Martin

    Monday, April 14, 2008 7:25 PM
  • Hi,

    I appreciate knowing that the ROBOCOPY MINAGE parameter can delete files that are older than 1 day or more, but is there a ways to delete files that are older than only 12 hours?

    Thanks in advance,
    Mary-Ann Erskine-Pourier
    (760) 804-2313
    Friday, November 14, 2008 9:58 PM
  • See e.g. here for a complete list of robocopy parameters, but no, MINAGE is a number of days, or a date.
    I'm not on the WHS team, I just post a lot. :)
    Friday, November 14, 2008 10:20 PM
    Moderator