locked
File Corruption RRS feed

  • Question

  •  

    Rather than add anything more to the data corruption discussion/fire/pile, I would instead like to add a suggestion:

     

    Maybe we need a problem fix rather than a bug fix until the bug fix is worked out. 

     

    In many cases on the server side, MS introduces tools/fixes/patches to get us over a problem.  The current offering -- do not use a file server as a file server reminds me of the joke... doc it hurts when I lift my  right arm...well then don't lift your right arm...[rimshot]  Just off the top of my head I can think of several in the W2K era around Terminal Server (installl one processor...install windows...uninstall ram...install sp1...install second processor..reinstall ram...) and even more around NT4 to W2K FAT/NTFS conversions. 

     

    As a user, at this point I would like to know, from MS and the WHS team, exactly, in technical detail, what is causing the problem (can you say DE?).  Perhaps share a list of tested scenarios and outcomes with the community.  My assumption here is that DE is the problem, as it is from many others.  Can MS WHS team at least confirm this?

     

    So far MS has only released one safe condition-- single drive. 

    Are drives outside of the DE drive pool safe?

    Are unduplicated shares in the drive pool safe?

    If DE is disabled, is there any corruption?

     

    Since we are months into this, and probably months before an end of this, maybe we need an approved no DE tool/patch rather than a fix to DE.  I would be more than happy with a known good configuration and much more patient than running a known ticking time bomb waiting for me to stumble upon a new combination that will corrupt my files.  And maybe this approach is what is lacking from the discussion or from the WHS team. 

     

    Which leads me back to an old topic...  Still no resolution to the NTbackup problem?  I am guessing now these are very related. 

     

    And to bring up just one more sore subject... Maybe we do need WHS on MSDN/Action Pack subscription.  Sure would come in handy right now for testing. I have a great test box ready and available...  I think the WHS is missing out on a great opportunity here to align themselves and tap into a large resource community.  I would be more than willing to run test scripts, follow check lists etc to provide data.  I am sure many others here would as well. 

     

     

    Ken

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

    Thursday, February 28, 2008 1:05 AM

All replies

  • Microsoft is working hard on a fix, Ken. I can't share any details of the source of the issue, but based on what I know, it's not in the class of issues that could be fixed temporarily with a hotfix. So you will have to wait for a solution to be available.

    As for Microsoft providing more information, where needed I pass these threads back to the team. In this case, I don't need to, as I'm leaning on the team to get more information out to the community as soon as reasonably possible, and have been right along. The problem is that this is all a very large, very public, public relations nightmare, and Microsoft (reasonably, I suppose) doesn't want to add fuel to the fire. So while I'm pressing them for more public disclosure, I don't offer any great hope that they will actually deliver. And of course, they will never please everybody, unless they just drop the code on us in a huge pile of bits and bytes. (Which wouldn't please me one bit, BTW. I have enough code I'll never look at sitting around already. Smile )

     fierrpawz wrote:
    So far MS has only released one safe condition-- single drive. 

    Are drives outside of the DE drive pool safe?

    Are unduplicated shares in the drive pool safe?

    If DE is disabled, is there any corruption?

    • Single drive is safe.
    • Drives outside the pool are safe.
    • Unduplicated shares are not safe.
    • You can't realistically disable Drive Extender. Drive Extender is not just the DE Migrator service, though that provides an important function. There are additional components that are also involved in the whole concept of the "storage pool", and those components are involved in the issue as well.
     fierrpawz wrote:

    Which leads me back to an old topic...  Still no resolution to the NTbackup problem?  I am guessing now these are very related.

    Ntbackup doesn't work because it can't back up everything and restore it, unless you want to shut your server down every time you back it up, boot off other media, and run the backup with the server itself off-line. Then you can get an image of the entire server. (Do you have anything to put your possibly multi-terabyte image on, BTW?)

     fierrpawz wrote:
    And to bring up just one more sore subject... Maybe we do need WHS on MSDN/Action Pack subscription.  Sure would come in handy right now for testing. I have a great test box ready and available...  I think the WHS is missing out on a great opportunity here to align themselves and tap into a large resource community.  I would be more than willing to run test scripts, follow check lists etc to provide data.  I am sure many others here would as well.

    What do you want to test that you can't test using the 120 day evaluation version, which you can get for a few bucks? The WHS team has decided that they don't want their product on MSDN right now, for whatever reason. Making a 120 day eval available for the cost of the media plus shipping is a pretty reasonable alternative.


    Thursday, February 28, 2008 5:14 AM
    Moderator
  • I was under the impresion the best work around is to put your corruptable files on a new non pool drive and then backup manualy from that drive as needed. 

    Thursday, February 28, 2008 4:43 PM
  • Ken, thank you for your reply, as has been the case you seem to be put in the position of spokesperson for the WHS team, rather than someone from the team speaking direct to the community. 

     

    Your use of phrases like "can't share details" "where needed" shows an active filter is being applied and it is clear there is PR management in play as you state.  Part of the reason for my post was a clue to the WHS team that lack of communication is now the problem.  The PR nightmare is now out in the general public.  Not having an official technical position beyond "some other software" causes problems with WHS, when it is fairly clear the problem lies in DE, is now more of a problem than whatever technical guidance that should be delivered.  Ok I can get with maybe a month or so of PR cover, but we are now staring down 6 months of PR cover with no substantive tech guidance?

     

    As has been shown by the less polite posts on the issue, lack of communication from the WHS team I think is the issue.  Case in point,  if this is the number one issue at WHS, then why are not the blogs full of discussion on the issue?  I should be reading about testing proceedures, the cup of coffee that got spilled at 3AM on the test box under a desk, why the team cannot reproduce itunes corruption... 

     

    Too much spin, not enough tech.  I was really hoping for better from WHS team. 

     

    Thank you for your response on what is safe and what is not.  It helps, but again, as you have stated several times, you are not on the WHS, only a dedicated user like us.  Where is the WHS team offical answer stating exactly what you just said?  Did that list come from the WHS? or is it your opinion based on reading comments here?  I too have read all of the comments on here, and sifted through what I think are real and what is hysteria, but why is that the best available answer? 

     

    You are correct a patch is not going to fix this problem.  But, a guidance, a data migration tool, a DE disabling option, would.  It would remove users from the harmful condition while a fix is in the works.  The current guidance has already been proven faulty, and does not remove the danger of corruption.  Again there are examples stacked high in the server world of exactly these type of fixes.  They have their day, and then are made obsolete by the next service pack or the passage of time. 

     

    I think the lack of MSDN/ActionPack comes from a similar place the problem above comes from.  There is obviously an insular and guarded mentality at WHS that does not bode well for the product, or market they are wanting to capture.  This could be the most interactive, involved community at MS, and some of the early work around user developed add-ons hinted at how this could be.  I hoped that when a problem did arise, the WHS team would continue this interaction with the community, and even leverage that community beyond what one would expect on the professional side.  WHS users are going to be invested, and passionate about the product, they are going to be evangalists with family and friends, they "get" it.  Part of bringing servers to the masses, is working with the masses, and sometimes hands will get dirty.  Rather than hiding ivory tower style, this would be a perfect chance for the WHS team to reach OUT not close in. 

     

    Sorry for the long winded post, but I think this event, not the technical problem, is setting a tone for all future problems, and it is not the tone the WHS set out with, and it is not what I expected.  I would expect at LEAST what would be available on technet, that is, an accurate scope of the problem, a fix that fixes the problem, or guidance/tool that I can follow to avoid the problem.  The current KB article is a PR piece, not a tech post.  I think this is what most are reacting to, where a lack of facts drives rampant hysteria.  The WHS team could be working actively to share the facts, and tone down the hysteria that only grows with each passing month. 

     

    Ken

     

     

     

    Thursday, February 28, 2008 6:22 PM
  • Thank you, I too am "under the impression" that non-pooled drives are safe.  I was looking for more official guidance than that though. 

     

    If you were to amass all of the "impressions" nothing would be safe.  (tech really isn't democratic)

    If you follow the exact guidelines of the KB article, that is not safe either, corruption is reproducable outside the known problem apps, and beyond just editing. 

     

    And that is my point.  In the absence of an official accurate known good, all is suspect, as others have tried to point out. 

     

    Due to the KB update, we now have one known good configuration - single drive. Hmm.. I dont have a 2TB drive. damn.

     

    At 4 months and counting, the silence is deafing, and the "wait and wow em with a fix" strategy is not working.

     

    Ken

     

     

     

     

     

     

     

     

     

     

    Thursday, February 28, 2008 6:36 PM
  • Ok consider this a "what I would do" musing--

     

    I am only a tech, I am not a developer, and do not have 1/10th the expertise I am sure exists in every developer on that team, nor have a spent months of my life wrestling with these problems, but if you would, let me sketch for a second:

     

    Using the same mission statement the team worked under --

     

     " ...helping families with multiple PCs connect their digital experiences, providing a familiar and reliable way to store, access, share and automatically protect what is most important"  from Charlie's blog post here-- http://blogs.technet.com/homeserver/archive/2007/02/13/why-doesn-t-windows-home-server-do-foo.aspx

     

     

    WHS has the same roots and under carriage as W2k3SBS, and this is a good thing.  This is a known good, based on a history of other known goods.  Since NT4sp6a or there abouts, Windows Server has been extremely reliable and easy to use way to access, share and protect important data.

     

    WHS diverged from this core competency, in order to deliver ease of use -- pooled storage.

     

    I think the team could deliver better on the mission with a different configuration that is still just as easy to use for the home user.  There was an assumption that pooled storage is easier to use than volumes.  I disagree. 

     

    Same as any other version of windows, disks are added or subtracted as seperate volumes and assigned drive letters.  It is also possible to span the volumes across multiple disks, but lets avoid that complication and say that is possible with a simple wizard question "would you like this disk added to your C: drive?"

     

    Add the concept of a backup disk, or even a network share. Both are possible today in NTbackup and even better in some cheap 3rd party backup solutions.

     

    I think the complexity of disks and volumes can be masked from the user using just the console wizard.

     

    I see a new tab, storage manager, that would allow some defaults to be set

     

    My System Drive =  C:  (not configurable)

    My Storage Drives = D: and E:,  WHS is free to use these for share storage.

    My Backup Drives = F: and \\pc\backupshare  WHS is free to use these for backups.

    My Removeable Drives = G: H: I:  Do not use for storage or backup.

     

    Or all volumes are available, and backup folders are just another managed share.

     

    Imagine the share tab wizard.  A simple set of questions, similar to the setup of SQL, asking where to put the share would be easy enough for any user.  Even suggest that the largest disk, with the largest free space, is the best option. 

     

    Would you like this share backed up?  Where to ?  Here are the available backup disks.

     

    When adding a new disk, a simple question of is this disk for storage, or backup, neither, or both?  I think any user can understand this. 

     

    Scripts can be created to add/move/delete/permission shares just as are in use today between the volumes.   The user does not need to see this, other than "this disk is too full, would you like me to move one or more shares?"  A space calculator could even be created to best distribute the shares on the drives.  "Would you like me to optimize your storage?  this could take a while..."  Let the calculator determine the space available on storage drives, and determine that my pictures, and private docs fit best on one drive, while my movies and music fit on another.  (with overhead and growth statistics taken into account).  Allow a manual override, no, really, I want my movies on my 10KRPM drive...

     

    We can argue duplication is not a neccesity in this environment, nor does it fit the mission statement.  Backups ARE a neccesity, and do fit the mission statement.  Duplication only protects from one failure, loss of hard drive.  Backups protect against many failures.  I am not buying the benefit of duplicaton.  Leave that to the HW manufacturers and allow them to use RAID1, or even something similar to Dell's backup drive option.  I am also sure there was debate on the use of DFS to accomplish the same tasks. 

     

    The SBS backup wizard is still a bit complicated for the average user, but is close.  I think it can be simplified for the average user to "do you want this share backed up?" and "where to?"   This would be an improvement as today there are no backups of shares (duplication is not a backup).  I even think the average user would be OK with weekly backup of shares if the backup windows are to congested.

     

    Out of the box thinking, since we already have synchronized user accounts, and PCs now days have oversized HDs, a wizard could create a network share for backups on PCs, and have the backup wizard utilize this space.  I am not sure how this folder could be excluded from the PC backup, but that is why there are smarter guys than me. 

     

    WHS has monitors, and can be flagged to watch the following conditions:

    • Drive > 85% full. 
    • Drive failures.
    • Backup success or failures.

     

    On drive 85% full offer the following options:

    • Move a share to a drive that has more space.
    • Add a new drive.
    • Compress.
    • Nag alerts until the situation is solved.

    On drive failure notices:

    • Red alert Nagging.
    • Offer to move shares on failing drive to other drive with space.  Use monitors to best calculate available space and distribution of the shares.
    • Enter Credit Card number to have drive shipped from new egg today...
    • No, really, go buy a drive now.

    Backups have been failing, not enough backup storage, would you like to:

    Add/Configure more backup disks?

    Configure space on a network PC for backup?  WHS will now look for available HD space on your PCs...  WHS found 100GB free on your OFFICEPC, would you like WHS to create a backup share here?  Based on our calculations, the following shares will be backed up here... SharedMusic and JOE user folder.  Is this OK?

     

    Sample screen for storage manager:

     

    Storage Disks:

     

    E:         200GB      100GB Free.  Healthy  Current Shares -- Shared Docs Type: SATA, internal

    F:         1000GB     800GB Free  Healthy  Current Shares -- Music, Videos Type: SATA internal

     

    Backup Disks

    G:     500GB         100GB Free   Healthy   Shares backed up PC2, Music, Video Type USB2 External

    \\PC\backup    250GB  100GB Free  Healthy Shares backed up -- SharedDocs, PC3.

     

     

    What would you like to do?

    Add new disk.

    Remove disk.

    Check health of disk.

    Change use of disk.

    Let WHS optimize your storage. 

     

    Sample Shares Screen:

     

    Shared Docs   Location  E: Drive. 20GB.  Backed up to \\PC\Backup yesterday.  EDIT?

    Music             Location F:    200GB  ALERT: Backed up to G: 2 days ago  EDIT?

    Videos            Location F:    500GB Backed up to G: yesterday EDIT?

    New                Location F:  20mb Backup not configured.  EDIT?

     

    What would you like to do?

    Change permissions on a share

    Relocate a share

    Backup a share

    Create a new share

     

     

     

    With the removal of DE, would we not get back volume shadow copy and NTBackup functionality?  Does this not also eliminate the large primary drive problem? 

     

    I think there is also an opportunity to introduce offsite storage to the mix, again by share level decision making.

     

    Target audience is a user with multiple PCs and a network.  I don't see anything above that a user in this group would not understand.   If Joe sixpack can figure out how to hookup the drive to the machine, I think Joe can follow wizards.    Grama Edith would not be a WHS user (yet).  I think you have at least 2 revs to hone this before she is in the mix.  Nor do I see this as any more complicated for the user than pooling. 

     

    Did I miss any of the key functionalities?  Would this not keep WHS in step with SBS and other known goods?  Would this not provide more reliability, more protection of data than the current product? 

     

    I think, since we are talking console and wizards to access already available tools, this should not be too hard to develop. 

     

    Just my 2cents.

     

    Ken

     

     

     

     

     

     

     

     

     

    Thursday, February 28, 2008 8:40 PM