locked
Backup Cleanup Issue RRS feed

  • Question

  • I've noticed that, if I click on "Cleanup Now" in WHS settings, rather than wait for the usual Sunday cleanup, my server will become inoperable after the cleanup is about half complete, and I have to hit the power button to get it going again. Then, to be safe, I use Toolkit to delete al the old backups & start over again.
    I'm not really looking for a solution...just will not try to do a manual cleanup anymore. One more WHS anomaly to deal with. There are no useful messages in the event logs.
    I'm sure this post will elicit the usual "this never happens to me" replies, but I'll bet I'm not the only one it does happen to. It's totally repeatable. Probably, most people never try to do this.
    Sunday, November 2, 2008 11:33 PM

Answers

  • Have you filed a bug report on Connect? If not, could you please do so, including the cab # for server logs submitted with the WHS toolkit add-in? Then post the Connect ID # back here?

    And if at all possible, don't delete the backup database afterward, at least not for a few days, in case additional information is required.

    I'm not on the WHS team, I just post a lot. :)
    • Edited by Ken WarrenModerator Monday, November 3, 2008 4:12 PM spelling
    • Marked as answer by Bobsie Tuesday, November 4, 2008 9:27 AM
    Monday, November 3, 2008 4:12 PM
    Moderator
  • I don't have a need to use manual cleanup on my production server. I have (wait for it...) never had a problem with backups. I did run one some time ago, but don't generally mess with a system that's working. :)

    I haven't done a recent manual cleanup on my test server either (during PP1 testing was the last time). When I have time (later this week, probably) I'll run some backups, flag some of them for deletion, then run a cleanup, assuming I can remember to do so. 

    I know you're experiencing a bunch of issues. I hate to bring this up, but have you considered that some/all of the issues you're experiencing with WHS may be related to your environment/hardware/software? For example, your slow backup issue: I can back up probably 2-3x the data in half the time it takes you, to (what I presume is) a slower and less capable server: an HP MediaSmart Server. We both have gigabit ethernet, too. when I see anecdotal evidence of that sort, the first thing I always ask is "what's different?"

    I'm not on the WHS team, I just post a lot. :)
    • Marked as answer by Bobsie Tuesday, November 4, 2008 9:27 AM
    Monday, November 3, 2008 6:21 PM
    Moderator
  • It has indeed been some time since the last time I did a full backup of a big machine to an empty database. But not as long as you probably think. I flip my main desktop machine (multiple drives in RAID 5, GB ethernet, fast Core 2 Duo) over to my test server when it comes time to test a new build, then I do a full backup. My test server runs an Allandale processor, 1 GB of RAM, and 2x 500 GB drives. So it's not much more powerful than an HP MediaSmart Server. That first backup is about 340 GB of data, and it runs overnight, but winds up taking about 5 hours. So yeah, there's an issue there on your end. To the HP MediaSmart, I think it was about 5.5 hours, but that was a year ago now and I have slept many days since then.

    Your laptops are a different issue. Get rid of the USB drives and backups will be a lot faster. Even if you have to get eSATA drives and an eSATA PC-Card (or ExpressCard) for each laptop. USB should never have been used as a way of connecting high-speed mass storage devices.

    Networking problems: Did you install the network? Or did you pay someone to? Was it done with the walls open, or did someone have to pull cable through the walls (and studs, sill plates, headers, joists, etc.) after everything was closed up? Old house or new? 

    When we had our house built, I inspected the wiring several times. I'm not a licensed electrician, or a certified network engineer, but I found a ton of stuff that was just plain wrong. As far as networking and entertainment goes, my favorite was the honking huge screw that went right through the middle of a structured wiring bundle, completely severing one coax and one Cat-5. Everyone screamed when I made them stop work on that wall until the network installer could get back out there and replace (and protect with a plate that he'd forgotten) that run. Then they screamed some more when he dinged them a couple of grand to fix that and other screwups that other trades had perpetrated, and I refused to pay for their subcontrators' incompetence.

    If you're likely to be moving soon, I would agree that you should wait on the whole "tune the network" deal.

    I'm not on the WHS team, I just post a lot. :)
    • Marked as answer by Bobsie Tuesday, November 4, 2008 9:27 AM
    Monday, November 3, 2008 10:34 PM
    Moderator
  • Bobsie said:

    I've noticed that, if I click on "Cleanup Now" in WHS settings, rather than wait for the usual Sunday cleanup, my server will become inoperable after the cleanup is about half complete, and I have to hit the power button to get it going again. Then, to be safe, I use Toolkit to delete al the old backups & start over again.
    I'm not really looking for a solution...just will not try to do a manual cleanup anymore. One more WHS anomaly to deal with. There are no useful messages in the event logs.
    I'm sure this post will elicit the usual "this never happens to me" replies, but I'll bet I'm not the only one it does happen to. It's totally repeatable. Probably, most people never try to do this.


    1. Did you ever try and give it some more time to finish? Depending on the amount of cleanup it has to do this process may take multiple hours and seemingly halted. Also please note HP mediasmart comes with 512MB RAM which is more then enough for most WHS boxes but simply not enough for this box. I also found mine was getting "unresponsive" in different situations, however giving the box some time it would always finish running processes and return to normal state. Upgrading RAM to 1 GB (I think this is currently also supported by HP) solved this problem. My home built WHS boxes run fine with 512 MB.

    2 The fact you find nothing useful in the event logs might well be an indication the process was still running (albeit slow) when you hit the power button (and thus probably corrupting the backup database). Also please have a look at the most recent backup log in C:\Documents and Settings\All Users\Application Data\Microsoft\Windows Home Server\logs, AFTER running manual cleanup and finishing either by completion or by manual reboot. You can also have a look at this log during cleanup. (best copy the file before opening). The log will tell you in detail what's happening, and is much more infomative then the event logs.

    3. I often run manual cleanups on all of my WHS boxes, and found this process to fail only when the backup database was (deliberately) corrupted. Failing cleanup due to this cause has been reported on the forums several times, however AFAIK in those cases it will fail quickly after starting because cleanup checks database integrity prior to the cleanup process. Even with one GB of RAM my Mediasmart because noticeably slower during large cleanup.

    So please try manual cleanup again, start late in the evening and allow some more time to finish, also you could temporarily disable backups when running this process
    • Marked as answer by Bobsie Tuesday, November 4, 2008 9:26 AM
    Tuesday, November 4, 2008 2:03 AM
    Moderator
  • Concerning your network problem, what are the actual transfer speeds between different workstations, workstations to WHS, both up and down? Are these OK after your intial problem?

    Also please have a look at the drivers (and router firmware). Best start with getting latest drivers for all of your hardware.

    Make sure driver settings are OK. You could try disabling / enabling JUMBO frames on all systems. Please note you may need to play around with settings here also (Slow Network Transfers - Fixed! (At least for Me))
    • Marked as answer by Bobsie Tuesday, November 4, 2008 9:27 AM
    Tuesday, November 4, 2008 2:37 AM
    Moderator
  • Oh. You meant "flash drive". I read USB drive as "USB connected hard drive". I don't think it's a good idea to back your flash drives up that way, TBH, but they're much smaller than than the average HDD, so they aren't as much of a problem. However, as you've now realized, they're backups of stuff that's getting backed up elsewhere. Why back them up too?

    As for the ExpressCard eSATA suggestion, I had something like this in mind, along with an eSATA HDD.

    I'm not on the WHS team, I just post a lot. :)
    • Marked as answer by Bobsie Tuesday, November 4, 2008 9:26 AM
    Tuesday, November 4, 2008 3:01 AM
    Moderator

All replies

  • Have you filed a bug report on Connect? If not, could you please do so, including the cab # for server logs submitted with the WHS toolkit add-in? Then post the Connect ID # back here?

    And if at all possible, don't delete the backup database afterward, at least not for a few days, in case additional information is required.

    I'm not on the WHS team, I just post a lot. :)
    • Edited by Ken WarrenModerator Monday, November 3, 2008 4:12 PM spelling
    • Marked as answer by Bobsie Tuesday, November 4, 2008 9:27 AM
    Monday, November 3, 2008 4:12 PM
    Moderator
  • Ken Warren said:

    Have you filed a bug report on Connect? If not, could you please do so, including the cab # for server logs submitted with the WHS toolkit add-in? Then post the Connect ID # back here?


    And if at all possible, don't delete the backup database afterward, at least not for a few days, in case additional information is required.


    Ken, I will do this if I ever again decide to do a manual cleanup. Not sure when, if ever, that will be because, being thrice-burned, I'm inclined to stay away from it. It wastes a lot of my time which needs to be devoted elsewhere.

    The reason I deleted the backup database those 3 times is that backups no longer seemed to run right after the manual cleanup episode.

    I'm fortunate to have my Ghost backups to rely on.

    Thanks for the reply; I know where you're coming from & I appreciate it.

    Just curious...have you personally tried the manual cleanup lately?
    • Edited by Bobsie Monday, November 3, 2008 4:45 PM
    Monday, November 3, 2008 4:42 PM
  • I don't have a need to use manual cleanup on my production server. I have (wait for it...) never had a problem with backups. I did run one some time ago, but don't generally mess with a system that's working. :)

    I haven't done a recent manual cleanup on my test server either (during PP1 testing was the last time). When I have time (later this week, probably) I'll run some backups, flag some of them for deletion, then run a cleanup, assuming I can remember to do so. 

    I know you're experiencing a bunch of issues. I hate to bring this up, but have you considered that some/all of the issues you're experiencing with WHS may be related to your environment/hardware/software? For example, your slow backup issue: I can back up probably 2-3x the data in half the time it takes you, to (what I presume is) a slower and less capable server: an HP MediaSmart Server. We both have gigabit ethernet, too. when I see anecdotal evidence of that sort, the first thing I always ask is "what's different?"

    I'm not on the WHS team, I just post a lot. :)
    • Marked as answer by Bobsie Tuesday, November 4, 2008 9:27 AM
    Monday, November 3, 2008 6:21 PM
    Moderator
  • Thanks for the ideas.
    Like you, I don't really have a reason to do the manual cleanup. Guess I just like to push a different button once in a while. Also, my backup window is not very large (6 hours) so I was thinking that it would help the Sunday backups to get done within the window if I did a manual cleanup earlier.

    As to the amount of time it takes to back up...naturally, after deleting all backups, the first backups take the longest. After that, the dailies go pretty quickly. But you're probably right that there's something in my environment that's not right. My main PC is a very fast, top of the line desktop. It took almost 6 hours to back up the 175GB on the 2 internal hard drives. The 2 notebooks in the house took 5 hours apiece to back up, with each one having about 110GB spread over 1 internal drive & 2 USB drives each. I'm sure your backups don't take that long, although it's probably been a very long time since you deleted all backups & started over.

    Even before adding the EX470 WHS to my network, I suspected networking  problems. I called Microsoft support and got a very eager & helpful Indian guy to start with. He could see the problem but couldn't do anything about it. (It's easy to demonstrate the problem by opening "Computer" on the desktop PC. It can take a minute or more to see(map) all the drives, even if you just did the same operation 5 minutes ago.)He got his Boss on with us. Still no solution. Finally, they patched in a guy they called the most knowledgeable networking specialist they had in their facility. He systematically ruled out every piece of hardware as the culprit, and could find nothing wrong in software. This was a phone call that lasted almost 7 hours, and I'm still really grateful to Microsoft for spending that much time on the problem. But the root cause never was identified.
    I later came to wonder if somehow the Cat 6 wiring in my new home was somehow flawed, but I think I'd rather move than try to troubleshoot that!! :-)
    Actually, my wife's career may cause us to move 2000 miles in the next 6-9 months, so I'm not motivated to spend a lot of time on the problem.

    My connection to the Internet via Comcast is unbelievably fast...22MB/s down and 1.6MB/s up. But moving big gobs of data around the house is definitely pokey.

    I'm guessing that if I plugged the WHS right into the router, rather than into the wall in the basement, it would go a lot faster. But my wife won't let me add one more gadget to the office where the router is.

    Thanks for listening and caring.

    Monday, November 3, 2008 7:09 PM
  • It has indeed been some time since the last time I did a full backup of a big machine to an empty database. But not as long as you probably think. I flip my main desktop machine (multiple drives in RAID 5, GB ethernet, fast Core 2 Duo) over to my test server when it comes time to test a new build, then I do a full backup. My test server runs an Allandale processor, 1 GB of RAM, and 2x 500 GB drives. So it's not much more powerful than an HP MediaSmart Server. That first backup is about 340 GB of data, and it runs overnight, but winds up taking about 5 hours. So yeah, there's an issue there on your end. To the HP MediaSmart, I think it was about 5.5 hours, but that was a year ago now and I have slept many days since then.

    Your laptops are a different issue. Get rid of the USB drives and backups will be a lot faster. Even if you have to get eSATA drives and an eSATA PC-Card (or ExpressCard) for each laptop. USB should never have been used as a way of connecting high-speed mass storage devices.

    Networking problems: Did you install the network? Or did you pay someone to? Was it done with the walls open, or did someone have to pull cable through the walls (and studs, sill plates, headers, joists, etc.) after everything was closed up? Old house or new? 

    When we had our house built, I inspected the wiring several times. I'm not a licensed electrician, or a certified network engineer, but I found a ton of stuff that was just plain wrong. As far as networking and entertainment goes, my favorite was the honking huge screw that went right through the middle of a structured wiring bundle, completely severing one coax and one Cat-5. Everyone screamed when I made them stop work on that wall until the network installer could get back out there and replace (and protect with a plate that he'd forgotten) that run. Then they screamed some more when he dinged them a couple of grand to fix that and other screwups that other trades had perpetrated, and I refused to pay for their subcontrators' incompetence.

    If you're likely to be moving soon, I would agree that you should wait on the whole "tune the network" deal.

    I'm not on the WHS team, I just post a lot. :)
    • Marked as answer by Bobsie Tuesday, November 4, 2008 9:27 AM
    Monday, November 3, 2008 10:34 PM
    Moderator
  • Although I was at the house every day it was being built, I paid no real attention to the home theater contractor who wired the Cat 6, phone & cable, except to make sure, on the first day, that he was really using Cat 6. He could very well have messed something up similar to what you found. Live & learn. (To answer your questions: the walls were open, the house was being newly built.)

    I ran all the speaker wiring myself & it works perfectly.

    You've got a point with USB drives on the laptops. One of them is used for "ReadyBoost" and backup storage of really critical data. The other is used for the Ghost backups. Why am I even backing them up to WHS???? Now that I think of it, it makes no sense. I'd switch to ExpressCard if I could, but those slots provide me with 4 extra USB ports which I seem to need. And I take the USB drives when I travel, so faster bootup & Ghost backup continues while I'm away from home.

    You know, besides stopping backup of the USB drives, it just hit me that I might be better served by running only manual WHS backups. That would stop the frustration over the WHS backup window and the inability to schedule the order in which the backups run. The latter is always causing inconvenience. Someday, if/when Microsoft gives us better control over the scheduling, I could go back to letting WHS do backups automatically.
    See, I've got family members who can & do work on any computer at any hour of the day or night. Makes it hard for me, the "Network/Database/IT Manager" to keep things running on an even keel.

    And, in my stream of consciousness, I just realized that, if I got ExpressCard eSata drives, I wouldn't need so many USB ports. So, a-shopping we will go.

    Ken, thanks for making me rethink my strategy.

    Monday, November 3, 2008 11:12 PM
  • Bobsie said:

    I've noticed that, if I click on "Cleanup Now" in WHS settings, rather than wait for the usual Sunday cleanup, my server will become inoperable after the cleanup is about half complete, and I have to hit the power button to get it going again. Then, to be safe, I use Toolkit to delete al the old backups & start over again.
    I'm not really looking for a solution...just will not try to do a manual cleanup anymore. One more WHS anomaly to deal with. There are no useful messages in the event logs.
    I'm sure this post will elicit the usual "this never happens to me" replies, but I'll bet I'm not the only one it does happen to. It's totally repeatable. Probably, most people never try to do this.


    1. Did you ever try and give it some more time to finish? Depending on the amount of cleanup it has to do this process may take multiple hours and seemingly halted. Also please note HP mediasmart comes with 512MB RAM which is more then enough for most WHS boxes but simply not enough for this box. I also found mine was getting "unresponsive" in different situations, however giving the box some time it would always finish running processes and return to normal state. Upgrading RAM to 1 GB (I think this is currently also supported by HP) solved this problem. My home built WHS boxes run fine with 512 MB.

    2 The fact you find nothing useful in the event logs might well be an indication the process was still running (albeit slow) when you hit the power button (and thus probably corrupting the backup database). Also please have a look at the most recent backup log in C:\Documents and Settings\All Users\Application Data\Microsoft\Windows Home Server\logs, AFTER running manual cleanup and finishing either by completion or by manual reboot. You can also have a look at this log during cleanup. (best copy the file before opening). The log will tell you in detail what's happening, and is much more infomative then the event logs.

    3. I often run manual cleanups on all of my WHS boxes, and found this process to fail only when the backup database was (deliberately) corrupted. Failing cleanup due to this cause has been reported on the forums several times, however AFAIK in those cases it will fail quickly after starting because cleanup checks database integrity prior to the cleanup process. Even with one GB of RAM my Mediasmart because noticeably slower during large cleanup.

    So please try manual cleanup again, start late in the evening and allow some more time to finish, also you could temporarily disable backups when running this process
    • Marked as answer by Bobsie Tuesday, November 4, 2008 9:26 AM
    Tuesday, November 4, 2008 2:03 AM
    Moderator
  • Concerning your network problem, what are the actual transfer speeds between different workstations, workstations to WHS, both up and down? Are these OK after your intial problem?

    Also please have a look at the drivers (and router firmware). Best start with getting latest drivers for all of your hardware.

    Make sure driver settings are OK. You could try disabling / enabling JUMBO frames on all systems. Please note you may need to play around with settings here also (Slow Network Transfers - Fixed! (At least for Me))
    • Marked as answer by Bobsie Tuesday, November 4, 2008 9:27 AM
    Tuesday, November 4, 2008 2:37 AM
    Moderator
  • Oh. You meant "flash drive". I read USB drive as "USB connected hard drive". I don't think it's a good idea to back your flash drives up that way, TBH, but they're much smaller than than the average HDD, so they aren't as much of a problem. However, as you've now realized, they're backups of stuff that's getting backed up elsewhere. Why back them up too?

    As for the ExpressCard eSATA suggestion, I had something like this in mind, along with an eSATA HDD.

    I'm not on the WHS team, I just post a lot. :)
    • Marked as answer by Bobsie Tuesday, November 4, 2008 9:26 AM
    Tuesday, November 4, 2008 3:01 AM
    Moderator
  • brubber said:

    Concerning your network problem, what are the actual transfer speeds between different workstations, workstations to WHS, both up and down? Are these OK after your intial problem?

    Also please have a look at the drivers (and router firmware). Best start with getting latest drivers for all of your hardware.

    Make sure driver settings are OK. You could try disabling / enabling JUMBO frames on all systems. Please note you may need to play around with settings here also (Slow Network Transfers - Fixed! (At least for Me))



    Thanks for all the good tips. The Microsoft guys I had on the phone did check all those things, however, so it's likely the fault lies with the wiring. I made note of a very long list of things they looked at, and your suggestions are near the top of the list.
    Tuesday, November 4, 2008 9:25 AM
  • brubber said:

    1. Did you ever try and give it some more time to finish? Depending on the amount of cleanup it has to do this process may take multiple hours and seemingly halted. Also please note HP mediasmart comes with 512MB RAM which is more then enough for most WHS boxes but simply not enough for this box. I also found mine was getting "unresponsive" in different situations, however giving the box some time it would always finish running processes and return to normal state. Upgrading RAM to 1 GB (I think this is currently also supported by HP) solved this problem. My home built WHS boxes run fine with 512 MB.


    I do have 2GB in the EX470.
    My server doesn't just become unresponsive during manual cleanup; it becomes unreachable. The Connector icon on each PC  turns from green to gray. Has yours ever done that during manual cleanup?
    Tuesday, November 4, 2008 9:30 AM
  • Ken Warren said:


    As for the ExpressCard eSATA suggestion, I had something like this in mind, along with an eSATA HDD.


    I'm not on the WHS team, I just post a lot. :)



    Thanks. Looks like the problem will be finding drives that are small enough & durable enough to be able to travel with us. The WD USB Passport is what we use now, & it's perfect. Will have to keep looking for something similar in eSata.
    Tuesday, November 4, 2008 9:38 AM
  • Ken Warren said:

    Oh. You meant "flash drive". I read USB drive as "USB connected hard drive". I don't think it's a good idea to back your flash drives up that way, TBH, but they're much smaller than than the average HDD, so they aren't as much of a problem. However, as you've now realized, they're backups of stuff that's getting backed up elsewhere. Why back them up too?


    After eliminating the USB & flash drives from the backup configuration, notebook backups took 7 minutes each. They used to run 1.5 hours even when they were not the first backup in the database for that PC.
    Ken, you were totally right & immensely helpful, as usual.
    Tuesday, November 4, 2008 3:11 PM
  • Keep the Passport and continue to use it, just don't back it up. You aren't likely to find anything in a 2.5" external drive with an eSATA interface, at least nothing that you'll like. I can find a couple of enclosures, and a couple of drives, but they're all expensive or just plain silly. (Leather covering on the enclosure? Ridiculous!)

    And I'm not surprised that taking the USB HDD out of the backup strategy helped. I would bet that backing up the flash drives isn't actually (much of) an issue...

    I'm not on the WHS team, I just post a lot. :)
    Tuesday, November 4, 2008 3:33 PM
    Moderator
  • It may not be pertinent to WHS directly, but I just have to make a followup on my networking issues which were described above.
    On a whim, I right-clicked on the Local Area Connection in the "Manage Network Connections" view of my Vista Ultimate PC and hit "Diagnose".
    As far as my notes show, Microsoft never asked me to do this during our long phone call.
    Almost fell off the chair when it said "Turn TCP On"!!!!! (I'm damn sure I never turned it off myself.)
    I did what it said and voila!, networking problem solved. Incredible.
    Friday, November 7, 2008 8:06 PM