locked
Sync from blob to local with fillter? RRS feed

Answers

  • Zafar,

    It really doesn't look that hard to do. Copy the code into a FilterBlobs() method.
    Just ask yourself: What do you want to filter on?

    Then just wrap the following lines inside an If block with the criteria you need, and that should be it.
    if (<your condition here>)
    {
        ItemFieldDictionary dict = new ItemFieldDictionary();
        dict.Add(new ItemField(ItemFields.CUSTOM_FIELD_NAME, typeof(string), o.Uri.ToString()));
        dict.Add(new ItemField(ItemFields.CUSTOM_FIELD_TIMESTAMP, typeof(ulong), (ulong)blob.Properties.LastModifiedUtc.ToBinary()));
        items.Add(dict);
    }

    Tuesday, February 9, 2010 1:45 PM

All replies

  • Zafar,

    I have no hands-on experience with the File Sync Providers, nor with Azure Blob Storage, but in essence you would want to modify the below method in AzureBlobSyncProvider.cs to report only the blobs you are interested in:
            // EnumerateItems is called by the framework to look for changes (inserts, updates, deletes) in the data store.
            // It should enumerate all of the items currently in the store and return the properties that were specified by the MetadataSchema property.
            public override void EnumerateItems(FullEnumerationContext context)
            {
                List<ItemFieldDictionary> items = DataStore.ListBlobs();
                context.ReportItems(items);
            }
    So build a new 'filtered' list from the items that were returned and pass that filtered list to context.ReportItems.

    Alternatively, you could change the DataStore.ListBlobs method and apply the filtering there. Or you could add a new method to the DataStore class, allowing you to filter the items and use that method in the EnumerateItems method in the SyncProvider.

    HTH,
    Rudi
    Tuesday, February 9, 2010 1:26 PM
  • Hi thanx you are right that instead if ListBlobs() there should be a method of FilterBlobs() but i have also no experience in File Sync Providers, nor with Azure Blob Storage.

    Any one who can write this method who has experience?     required very urgent.

    Thanx

    regards,
    zafar
    Tuesday, February 9, 2010 1:34 PM
  • Zafar,

    It really doesn't look that hard to do. Copy the code into a FilterBlobs() method.
    Just ask yourself: What do you want to filter on?

    Then just wrap the following lines inside an If block with the criteria you need, and that should be it.
    if (<your condition here>)
    {
        ItemFieldDictionary dict = new ItemFieldDictionary();
        dict.Add(new ItemField(ItemFields.CUSTOM_FIELD_NAME, typeof(string), o.Uri.ToString()));
        dict.Add(new ItemField(ItemFields.CUSTOM_FIELD_TIMESTAMP, typeof(ulong), (ulong)blob.Properties.LastModifiedUtc.ToBinary()));
        items.Add(dict);
    }

    Tuesday, February 9, 2010 1:45 PM
  • Hi thanx for your soon reply. i will implement it then i will let you know ok.

    In my case i want to syn specific pseudo folder which was in the container. e.g i have container name "photogallery" and within this i have lot of  pseudo folder with user name like "user1" , "user2" etc and i just want to syn user1 with local files.

    Regards,

    Zafar
    Tuesday, February 9, 2010 2:06 PM
  • In this case, if the implementation can go through, then the client will have 5 files out of 20.

    My question: if the local store sync with another blob store, we may end up with non-convergent data.  Here is the possible scenario.

    0. Assume the blob store, the local store and a 3rd store are in sync.
    1. The blob store create 6 new files and 5 of them are in the filter.
    2. Sync from blob to the local store and 5 newly created file in the filter go to the local store.  However, since we do not construst the filtered change batch, in this case the local store will learn all changes from the blob which means it applies 5 changes but "pretends" learning all 6 changes.
    3. Now you can see that when the local store sync to the 3rd store, it will pass the knowledge of all 6 changes to the 3rd replica.  But since it only have 5 new files so it pass to 3rd store 5 new files.  (missing the 6th one here).
    4. Sync from the blob store to the 3rd replica, since the knowledge on the 3rd replica contains all 6 changes (from the local store), then the 6th file is no longer coming.  Now we have non-convergent data.


    If files are not shared between users, then you could setup sync betwen blob_storage\each user to the loca store of each uesr without doing "filtered sync".


    Thanks.


    Leo Zhou ------ This posting is provided "AS IS" with no warranties, and confers no rights.
    Wednesday, February 10, 2010 9:04 AM
    Answerer
  • Hi  Rudi - Euricom

    You are right i have implement the FilterBlobs() instead of ListBlobs() according to my requirement.

    Thanx for you help

    Now i can sync blob to local according to my requirement.


    Regards,
    Zafar
    Friday, February 12, 2010 7:36 AM