locked
Drive Extender compromise method RRS feed

  • General discussion

  • Can you tell me if this would be a possible way of implementing the Drive Extender concept?...

    It would work by accepting the limitations (and flexibility) of NTFS and working with it, rather than trying to layer stuff on top of it.

    The basic idea is that when formatting a drive during WHS setup, the user has the option to pre-allocate extension space in the MFT - essentially to format a drive as if it were larger than it really is. It would mean making a distinction between allocated space and realized space (hoping that is appropriate terminology). The allocated space is the formatted "space" - equal to or some multiple of the actual size of the volume being formatted. Realized space is the size of the volume (real), and non-realized space is the formatted space minus the realized space.

    Ok, now suppose a new NTFS attribute called $NON_REALIZED. Create a folder on the formatted drive called "<Drive>\Non Realized Space" and give it the $NON_REALIZED attribute, with folder size set to the extent (size) of non-realized space. NTFS ignores any folder with the $NON_REALIZED attribute when servicing I/O requests - that is, it does not regard it as containing free space. However, as soon a new drive is added to the computer or NAS, NTFS reduces the size of "<Drive>\Non Realized Space" by an amount equal to the formatted capacity of the new drive.

    So the compromise is that the user has to indicate at setup time, what growth he/she expects in total volume size, and accept that adding pre-allocated space larger than the physical drive would mean forgoing space for files on the existing drive(s), due to the increased size of the Master File Table.

    Questions:

    1. Does this make sense?

    2. Is it technically feasible?

    3. Would people accept having to plan their volume growth at setup time?

    4. Would people accept losing more of their initial drive space as a trade-off for growth potential?

    5. Would end users understand the concept well enough to make and informed decision at setup time?

     

    Tuesday, March 8, 2011 1:46 AM

All replies

  • You couldn't do that with vanilla NTFS and would still have to layer
    something on top to handle it all...  Something's got to keep track of
    what data is on which disk and how and where to write new data to. 
     

    Bob Comer - Microsoft MVP Virtual Machine
    Tuesday, March 8, 2011 2:44 AM
  • "Something's got to keep track of what data is on which disk and how and where to write new data to."

    Another attribute: $LOCATE

    That attribute contains disc location data. So a folder is ignored if attribute $NON_REALIZED exists, or is (partially) referenced by $LOCATE otherwise. Sure NTFS would need some work to get this going, however i don't see why this can't be handled entirely within NTFS.sys, and then you have something that can work on any product.

    Tuesday, March 8, 2011 4:32 AM
  • Looks like something neer that in the StableBit Drivepool

     

    See:

    http://blog.covecube.com/2011/03/stablebit-drivepool-technical-overview/

    Tuesday, March 8, 2011 10:48 PM
  • I'm not sure how similar it is to my concept, but it looks interesting.

     

    Tuesday, March 8, 2011 10:59 PM
  • If $LOCATE containted LBN ranges for the drives in the allocated space set (updated as necessary), couldn't the disk location of a folder be determined simply by taking its starting LBN, and doing a select case on the LBN ranges stored in $LOCATE? That is, just calculate the location of each folder on the fly, rather than adding an extra index. A tiny performance hit on a media drive is inconsequential.

    However keeping track of what disks data is on and where it should go is achieved, surely this is not so difficult as to make the scheme impractical? What i'm trying to get to is why the ability to extend drives is so difficult to achieve reliably.

     

    Tuesday, March 8, 2011 11:21 PM