locked
C: drive problem--space and fragmentation issues RRS feed

  • Question

  • I just got an HP EX495 two weeks ago, and I kept it as out-of-the-box as possible.  I methodically added 3 1TB drives and after gradually loading about 700GB of material over the course of a couple of days, all was right with the world.

    The day after I was done, the WHS console slowed to a crawl.  Over the course of the day it became completely paralyzed.  Even if I could get all the buttons and images to load, I'd be lucky if I could get one click that would respond within 10 minutes before the WHS Console completely stopped responding to any clicks (as in, click...wait an hour....nothing changes)

    I decided to remote into the server and look (but not touch!).  What I found was this:

    1) C:\ is the most fragmented I have ever seen a hard drive.
    2) C:\ was low on space

    I tried running Windows Disk Defragmenter on C:\, and naturally it took a LONG time to get those 20GB taken care of.  WHS console performance was better...but within a few hours not only did it go back to being slow, but the drive re-fragmented itself and the WHS then became so slow as to be unusable.

    Over the course of the next day, C:\ became completely full, despite the fact that I had added zero content.  I managed to install the WHS version of Diskeeper 2010, and it too reported C:\ being ungodly fragmented. 

    I told Diskeeper to defrag the entire machine on a reboot, went to bed, and in the morning the server was back up and running...but still the WHS console was slow.  A check of the diskeeper Analyze function and, once again, red everywhere--super fragmented hard drive. 

    I'm down to the point where WHS will connect but not load the opening page, C:\ has less than 1GB free, and is ridiculously fragmented.  HELP!!!

    Other factoids:
    - C:\Windows\Temp is practically empty
    - C:\Windows\Prefetch is pratically empty
    - System Properties indicated Windows manages the size of the page file and currently claims about 3.5GB
    - C:\Documents and Settings\All Users\Application Data\TwonkyMedia is a huge folder--over 2GB but I can't get it to ever finally log how big it is.

    Everything else runs perfectly....but if I can't get into the WHS Console....I can't administer large chunks of the server's functionality! 

    (Can I?)
    Saturday, December 5, 2009 5:42 AM

Answers

  • Update:

    HP has excellent tech support:  the kind that calls you, then works around your schedule.  The kind that takes your issue and tries to replicate it in their lab.  Given a lousy experience I'd had with HP laptops, this is saying something.

    Here's what they found: 

    Problem 1:  TwonkyMedia was problematic, but not the culprit...the reason this wasn't obvious is that once the damage was done, turning off Twonky didn't change the fact that I had literally one million tiny .wpl files murdering the hard drive's performance.  Using a command line it took 20 hours and multiple tries for the WHS to delete them all.

    I have from the HP rep that TwonkyMedia should only generate around 8-10 .wpl files, give or take, but definitely nowhere near a million.

    Problem 2:  Not every client was running the latest version of the WHS software.  This also wasn't obvious--I knew to uninstall and reinstall the WHS Connector, but I didn't know that you had to completely uninstall all HP software (or you get the "Windows Home Server Console is an older version" message) or that it might be necessary after major updates.  I had done the reinstall after Power Pack 3, but it looks like I'd needed to do it again a bit further down the road.

    So here's the good news:

    a) HP managed to replicate the issue in their labs and figure out how to walk people through cleaning up the mess

    b) HP worked with TwonkyMedia to get a more permanent fix to the file problem.  They passed an advance copy of this fix on to me, and last week (Wednesday, 2/24) we tried it.

    I did a complete uninstall, reboot, and reinstall on every client machine in my home.  So far everything's working.  I'm going to give it a good 10 days to be sure it doesn't slowly decay again, but my hopes are definitely up.

    I will post again after I confirm a fix, and if I can get a timeframe for when TwonkyMedia will release this update to the public I will do so.

    Thanks all of you for your help.
    Wednesday, March 3, 2010 1:55 PM

All replies

  • No, you can't administer the server if you can't access the console. :)

    Something is consuming space on C: on your server. The question is: What? You can log in using Remote Desktop (note: this is unsupported by Microsoft; there are tools available through the desktop that can seriously screw up your server) and browse through folders until you find the offender. Two places worth taking a look at are C:\WINDOWS\system32\LogFiles\W3SVC1 and W3SVC2 , and C:\Documents and Settings\All Users\Application Data\Microsoft\Windows Home Server\logs. If you find huge logs in the first location, it may indicate that someone (a neighbor with some malware on their PC, most likely) is constantly probing for open ports to infect other PCs. In the second location, it may indicate that there's something wrong with your server.
    I'm not on the WHS team, I just post a lot. :)
    Saturday, December 5, 2009 2:34 PM
    Moderator
  • Thanks Ken!  I've seen your helpful posts on some of the other threads I'd been reading to wrap my head around this problem.

    I'll check the log folders too--thanks for the tip.  I have found that the TwonkyMedia folder in C:\Documents and Settings\All Users\Application Data may also be the culprit:  about 12GB!  On a 20GB C:\ partition, that's bad.

    So...to clear room up long enough to get the fragmentation under control, I stopped the Twonky service, moved the files onto D, and am running Diskeeper.  Once things are nice and tidy, I figure I have three options:

    a) reboot and let Twonky build again, hoping it all goes back to normal
    b) throttle Twonky back with Disk Quota management first (something tells me this would be bad, just because WHS isn't a full-blown Server OS and because Twonky probably won't know what to do if it runs out of room to grow)
    c) reboot and see if I can't tell Twonky to start logging on D:\ and not C:\.  I see plenty on the Twonky forums on how to do this, but most appear to revolve around linux command-line instructions (bleah).

    Whichever I choose, I have a feeling that a defragged C:\ will get me my WHS Console access back, and I can stop using RDS. :)

    I'll post again when I can check the logs, too...
    Saturday, December 5, 2009 11:45 PM
  • Well, Twonky is your culprit, that's obvious. If I were you I would ask Twonky directly how to install and configure on Windows Home Server, because logging that much data to anywhere on C: is just not workable in this environment.
    I'm not on the WHS team, I just post a lot. :)
    Sunday, December 6, 2009 4:25 AM
    Moderator
  • As luck would have it, I found that although the Twonky information is no longer accurate as to where the configuration (.ini) file is , the current place Twonky puts it is that same crowded folder C:\Documents and Settings\All Users\Application Data\TwonkyMedia (which, it turns out, had literally a million .wpl files in it).  Once I'd moved the .wpls away, I pointed the catalog folder paths to D:\, saved to the file, and restarted Twonky.

    I've kept my eye on it, and Twonky appears to be running and gathering info...but thankfully is only growing the database file (which I'm fine leaving on C:\).  The .wpls it generates are now properly being stored on D:\ using Twonky's own default paths.

    If this works, I will post details here so that others may use the solution.  As for WHS' log files, they were quite reasonably sized at about a few MB total, so I left them alone.

    This leaves me with only one problem remaining:  fragmentation.  I told Diskeeper to do a boot-time scan and configured it to do defragment everything thoroughly.  I kicked that off by rebooting the server, went to bed...and in the morning the job completed, Diskeeper said the drive was defragmented...but I still see tons of red in the report.  And the program itself says that the drive is "suffering file performance issues due to fragmentation".  It says I should defragment the drive, but if I do, it'll run for a minute or two and declare itself done!

    Two options now:

    a) leave Diskeeper and assume it needs a few days to sort everything out with its automated background defrag

    b) yank Diskeeper and go back to WHS' Defrag utility.

    What would you do?

    Thanks again for your help!
    Sunday, December 6, 2009 10:35 PM
  • Okay, lesson learned...Diskeeper wasn't defragmenting all the way because its advanced settings for manual defragmentation offer a setting that only fiddles with large files the minimum necessary...uncheck that box, run a defrag, and it goes down deep & defrags the drive all the way.

    So now I get the WHS console, but it's still very, very slow.  I'm stumped!  Drives 100% defragged and everything.

    I installed WD Caviar 1TB volumes with a 32MB cache @ 7,200 RPM (and of course SATA II 3.0GB/S).  The default drive is a 1.5TB volume, probably with a 16MB cache and of course SATA II 3.0GB/s, most likely 7,200RPM as well.

    Would there be any disparity that is causing the console to be slow?

    Is a grand total of 4.5TB too much for the Ex495 to handle?

    These may be rhetorical questions, I'm afraid...so weird that the console is horribly slow, despite everything else chugging along nice and smoothly...
    Friday, December 11, 2009 4:39 AM
  • Hi,
    can you see anything in the event log?
    Defragmentation on Windows Home Server is not recommended by me, since the defragmentation software is taking ressources, may interact in a bad way with WHS and the main bottleneck for a server in low user count environments usually is the network.
    In task manager, can you see any process, taking up a high/increasing amount of CPU/memory?

    It may also be useful to uninstall add-ins, which you do not use.

    Best greetings from Germany
    Olaf
    Friday, December 11, 2009 9:23 AM
    Moderator
  • Thank you Olaf!  And greetings to you from Kentucky, USA. :)

    I installed Diskeeper because it will defragment in the background, it can be set to run in the middle of the night, and it can be configured to not take system resources.  I did it because any hard drive that becomes fragmented is automatically going to have problems with performance. 

    I also did it because the Windows Home Server defragmenter was not taking care of the problem.  So I'm not sure I want to give up defragmentation and go back to a hard drive that makes "thrashing" sounds constantly.

    I see nothing in the Event Logs that is out of the ordinary:  no warnings, no errors.

    I see nothing in the Task Manager taking up resources:  everything is very stable and typically "idles" at 4% CPU usage.

    I definitely uninstalled the other add-in I had installed:  Avast Antivirus.  I am a fan of Avast, but I uninstalled it to troubleshoot this problem.

    At the time, the fact that Twonky filled the hard drive definitely seemed to be the issue:  there was no room and the Page File became fragmented. 

    Now that I have Twonky using D: for the catalog, I have plenty of free space and the C: drive is very clean and unfragmented.

    The WHS itself performs very fast!

    It's just the Console that is very slow.  I can't figure out why everything else runs and has no issues, but the console is a slow mess.

    Let me know what else you think I should try, and I will definitely give it a shot.
    Friday, December 11, 2009 12:49 PM
  • What do you mean with "Slow"? Startup of the console may take a while due to all the stuff to load.
    Is Remote Desktop also that slow or a console session, started within the Remote Desktop session?
    If so, it could be an Add-In, which behaves bad.
    Uninstall one Add-In and check again, until you have found the culprit.
    Which version number is shown under Settings/Ressources?

    Best greetings from Germany
    Olaf
    Friday, December 11, 2009 1:00 PM
    Moderator
  • Hrm.  I no longer have any add-ins on the WHS.  Except Diskeeper, which I installed *after* having the problem.

    By "Slow" I mean this:

    Click the WHS Console icon--quickly get the login screen.
    Click the login button (the circle with an arrow in it) and connection starts
    It can take 2 or 3 minutes for the first page on the WHS Console to load in the WHS Console window.
    Clicking anything--any one button--one time--will take 5-10 minutes before the screen refreshes and shows me the screen I wanted.
    Every step is like this, but some steps never respond at all:  I can click "Settings" and in 5 minutes I get the Settings window, but some of the left-hand icons (like "Twonky" and "Remote Access") may never respond.  An hour after I click they may still be trying to respond.

    And yet...

    A Remote Desktop session runs just fine.
    Accessing media runs just fine.
    Browsing files across the network runs fine.
    Opening files using Windows Explorer (like double-clicking an MP3 after browsing to the folder) runs fine.

    It's just the WHS console that sucks.  And for the first week I had the server, it also ran perfectly.  It didn't slowly degrade over time:  I just logged in one day and it was horribly slow, and within an hour or two it was as slow as it is now.

    I will try to get to Settings/Resrouces, but I may never get there.  Resources is one of the buttons I can click that doesn't respond.  Thank you for the suggestion, though.  I will try it.
    Friday, December 11, 2009 10:17 PM
  • Well, that's really slow.
    On the logon screen, below the password field, you can find the menu Options. Within is an item "Reset Windows Home Server console".
    Did you try this already?
    And again - which Add-Ins are installed?
    Best greetings from Germany
    Olaf
    Saturday, December 12, 2009 1:49 PM
    Moderator
  • Hi
    I have some suggestions.
    1. Did you change the router recently? Some, e.g. Dlink DIR615 do not like WHS. They don't like VoiP either but that is a whole other story :-(
    2. If things are slow on a machine it is one of two things, either high disk activity or high CPU load. for this I use a free enhanced Task Manager called Extended Task Manager by Extensoft. It is free and can show you which process is taking up cpu and/or disk load. Use this over RDP while logging on to the Console from another PC (don't start the console form within RDP for this).
    3. When you run the cosole from within RDP, make sure you log off to exit, don't jus close RDP. closing will leave the Console running, thus consuming resources.
    4. Try logging on to the Cosole with your router disconnected from the WAN. If this works quickly, then add your server's name t Windows/sys32/drivers/etc/hosts. PP# apparaently does this but check to be sure. This avoids the server trying to resolve your server name over DNS. (which is what the DIR615 doesn't handle well at all)

    There are reports of demigrator.exe taking up most of cpu and disk access. I have not seen that on my H340, and I presently have 8TB of duplicated data across 6 drives. This is even with 2 DVD drives writing to the server via gigabit net and 1 XBMC system viewing a movie concurrently with music playing to another PC.

    Gerrit
    Saturday, December 12, 2009 5:56 PM
  • Argh - apparently my reply disappeared into the aether...

    Anyway, I did try resetting the WHS Console.  It didn't help.

    And again - I do not have any add-ins installed except Diskeeper.  That's it.

    I don't think Diskeeper is the problem, since I put it on the Server to try and SOLVE the problem.  The drives stayed fragmented and the Server wasn't defragging them.

    I hope Germany is enjoying the same nice weather as we have here - sunny and 6 deg. C :)
    Thursday, December 17, 2009 8:03 PM
  • Hello z80yyz, and thank you for replying.  :)

    1) No I did not change the router.  It is a Linksys WRT610N Ultra and the server ran fine on it for days before running into this problem.

    2) High disk activity - my C:\ drive was horribly fragmented.  Now it's not fragmented.  Still I have the slow WHS console problem.

    No high CPU load - no load at all!  The CPU can be fluctuating between 0-2% used and 98-100% free and still I will have the problem.  Task Manager doesn't show me anything eating resources, and if you see above, anything ELSE I do on the server BESIDES the WHS is really fast and responsive.  It's just the darn WHS Console that wants to freeze up. :)

    3) I have to administer about 8-10 servers in my job.  I definitely never disconnect from an RDP session without doing the manual "Start"-->"Log Off" (or in some cases, "disconnect") method.  Good tip, though:  everyone should be aware of that.  :)
     
    4) I will try disconnecting the router from the WAN and hitting hte console again.  I am not sure what in particular you are wanting me to add to the hosts file to refer to my server's name:  the resolved network name?  FQDN within my home network?  Let me know what you have in mind and I'll definitely give it a try.

    Demigrator (or any other programs) don't seem to be hitting the CPU/Disk at all.  The Server is idling along with minimal resources being used, it's easy to Browse to, quick to respond to a playback request, and even fine over RDP.  It's just the darn WHS Console that wants to freeze up.

    Let me put this out to anyone reading:

    Is there a possibility that Windows Search 4 sucks?  The server installed this on itself as an update, but I've had it really ruin performance on XP machines before, back in the days when you couldn't even uninstall it.

    Should I try removing Windows Search 4?
    Thursday, December 17, 2009 8:14 PM
  • An update:

    I tried disconnecting the router from the WAN and it didn't change anything:  the WHS console is still slow.  I did this with only one machine on my network, so nothing else could possibly be hitting it.

    I tried physically connecting the WHS to one machine instead of through the Router--still had the problem.

    I then decided to do a complete reinstall:  use the WHS disc and reinstall the Server OS the recommended way (on an HP MediaSmart EX495, this is a setup wizard that walks you through the rebuild).

    Once it finished, I had a nice clean rebuild and I told it not to download any updates yet:  let's just see how it works out of the box.  I figured if I could at least finish setting things up the way I wanted, I could live with a frozen WHS console since everything else runs fine.

    I never got to the console:  even with a clean rebuild it would take 5-10 minutes to load the WHS screen, and then stop responding completely (I clicked "Settings", waited an hour, and still had no response).

    Guys/gals, this is getting ridiculous.  I came at the idea of a Windows Home Server with my "joe user" hat on.  I never wanted to tinker with a single thing:  just "set it & forget it".  I'm baffled that one day it just crapped out and stopped responding like this.  And that nothing I've done--slowly escalating from "try this simple thing" to "remote into the box and start looking deeper"--has fixed this one simple thing.

    Could if be a problem with the fact that the WHS Console is Flash driven?  Is this an Adobe issue?

    Wednesday, December 30, 2009 9:55 PM
  • An update:

    I tried disconnecting the router from the WAN and it didn't change anything:  the WHS console is still slow.  I did this with only one machine on my network, so nothing else could possibly be hitting it.

    I tried physically connecting the WHS to one machine instead of through the Router--still had the problem.

    I then decided to do a complete reinstall:  use the WHS disc and reinstall the Server OS the recommended way (on an HP MediaSmart EX495, this is a setup wizard that walks you through the rebuild).

    Once it finished, I had a nice clean rebuild and I told it not to download any updates yet:  let's just see how it works out of the box.  I figured if I could at least finish setting things up the way I wanted, I could live with a frozen WHS console since everything else runs fine.

    I never got to the console:  even with a clean rebuild it would take 5-10 minutes to load the WHS screen, and then stop responding completely (I clicked "Settings", waited an hour, and still had no response).

    Guys/gals, this is getting ridiculous.  I came at the idea of a Windows Home Server with my "joe user" hat on.  I never wanted to tinker with a single thing:  just "set it & forget it".  I'm baffled that one day it just crapped out and stopped responding like this.  And that nothing I've done--slowly escalating from "try this simple thing" to "remote into the box and start looking deeper"--has fixed this one simple thing.

    Could if be a problem with the fact that the WHS Console is Flash driven?  Is this an Adobe issue?

    It could be some sort of hardware issue.  You should try contacting HP for support.
    Thursday, December 31, 2009 3:24 AM
    Moderator
  • I had an unfortunately nasty experience with HP support on another matter, shortly after I bought the WHS.  Now I'm loath to contact them, but I'll give it a try.  I imagine they'll tell me to blow it away (again) and then not actually know why the WHS console is so slow.

    [rant]
    This is ridiculous.  I actually want to do everything the out-of-the-box way, and manage everything through the console.  Why in the world is everything else about this machine perfect, but the WHS console such a piece of garbage?
    [/rant]

    Are most people using WHS getting along just fine with the console?  I've tried 6 computers, 2 different routers, and six versions of Windows (XP Pro, XP x64, and all three flavors of 7).  I'm baffled.

    Thanks all of you for your suggestions.  I *do* try them, and it *has* helped to have someone else come up with ideas, now that I've run out of them.

    Thursday, January 7, 2010 1:44 AM
  • I've had pretty good experiences with HP's WHS support staff, much better than other support divisions.  If you've done a reinstall and are having major problems, you really should contact HP Support.

    Twonky is terrible.  It kills CPU cycles and doesn't stream very well even when it's done indexing media files.  HP should never have added it to this product, it's just a marketing ploy that suckers like me fell for.  I've posted on sevferal forums and no one has yet to say - "Gee, you're wrong, Twonkey works great on my system."  So, I recommend trying another fresh install of WHS OS.  Uninstall Twonky immediately.  Do any updates, then if it still doesn't respond well, call HP.

    HP should really be taken to task over Twonky.

    I'm looking at WebGuide as an alternative.
    Thursday, January 7, 2010 6:49 PM
  • Gary, thanks!  This is encouraging. 

    I will go ahead and open a support ticket with HP next week and see what they say.  I figured it was either that or I'd be trying to install a proper Server OS on the box...I'd rather keep this simple. :)

    Thanks again.
    Thursday, January 14, 2010 10:42 PM
  • Short update - HP is helping, but this may take a while.  They'd mentioned that there are known issues with TwonkyMedia, but right now they're thinking that may not be the problem.  I'll post back here when I have a good definitive answer.
    Tuesday, February 16, 2010 6:00 PM
  • Update:

    HP has excellent tech support:  the kind that calls you, then works around your schedule.  The kind that takes your issue and tries to replicate it in their lab.  Given a lousy experience I'd had with HP laptops, this is saying something.

    Here's what they found: 

    Problem 1:  TwonkyMedia was problematic, but not the culprit...the reason this wasn't obvious is that once the damage was done, turning off Twonky didn't change the fact that I had literally one million tiny .wpl files murdering the hard drive's performance.  Using a command line it took 20 hours and multiple tries for the WHS to delete them all.

    I have from the HP rep that TwonkyMedia should only generate around 8-10 .wpl files, give or take, but definitely nowhere near a million.

    Problem 2:  Not every client was running the latest version of the WHS software.  This also wasn't obvious--I knew to uninstall and reinstall the WHS Connector, but I didn't know that you had to completely uninstall all HP software (or you get the "Windows Home Server Console is an older version" message) or that it might be necessary after major updates.  I had done the reinstall after Power Pack 3, but it looks like I'd needed to do it again a bit further down the road.

    So here's the good news:

    a) HP managed to replicate the issue in their labs and figure out how to walk people through cleaning up the mess

    b) HP worked with TwonkyMedia to get a more permanent fix to the file problem.  They passed an advance copy of this fix on to me, and last week (Wednesday, 2/24) we tried it.

    I did a complete uninstall, reboot, and reinstall on every client machine in my home.  So far everything's working.  I'm going to give it a good 10 days to be sure it doesn't slowly decay again, but my hopes are definitely up.

    I will post again after I confirm a fix, and if I can get a timeframe for when TwonkyMedia will release this update to the public I will do so.

    Thanks all of you for your help.
    Wednesday, March 3, 2010 1:55 PM
  • The twonkymedia update is meant to be out in the HP 3.0 Release DVDs that everyone can use to upgrade their EX470/5  EX485/7 models.  I guess the update for the EX490/5 will come through a downloadable patch.  Source of this is Alex Kuretz on the 26th Feb Windows Home Server Show Podcast:
    http://homeservershow.com/the-home-server-show-78.html

    More info on the DVDs here:
    http://mswhs.com/2010/02/25/hp-3-0-software-shipping/

    In the UK for £10:
    https://www.best2serve.com/p,website.landing/lang,en/show_flow,RCD/start_flow,RCD



    --
    Wednesday, March 3, 2010 3:17 PM
  • Al, thanks for posting that!  Anyone with this problem will definitely find that useful to know.

    And those that do have to reinstall your client software, don't forget to pull the HP stuff off as well as the Windows Home Server stuff.  It'll save you headaches and versioning messages down the road...
    Thursday, March 11, 2010 8:43 PM
  • Okay, last post and I'll stop chattering.

    Just FYI, we're two weeks out from when I ran the update.  Everything appears to be running rather smoothly.  I get both Twonky and Windows Media servers exposed to me and the WHS Console, while occasionally a bit sluggish, is nowhere NEAR as bad as it was.  I can even access my personal web page for the WHS very easily from elsewhere.

    I've seen a drop in video playback quality on the PS3, but I blame this on the PS3's bandwidth because any computer with Windows Media Center and the WHS plugin for it can stream things very nicely.

    C:\ drive still sits at 60% free.  Diskeeper reports everything staying defragmented, I never hear the box making thrashing noises, and I can see from WHS that storage is balanced.  I even turned on the automatic video conversion and found it to "rip" a full DVD in far less time than my copy of CloneDVD Mobile did.

    I'm ready to call this good, and thank goodness.  Thanks all of your for your helpful suggestions, and thanks HP for great support.
    Thursday, March 11, 2010 8:57 PM
  • Why don't you try one of the following tools:

    Defraggler:

    http://www.piriform.com/defraggler

    MyDefrag

    http://www.mydefrag.com/

    Tuesday, January 8, 2013 8:27 PM