none
Apply Round Robin Approach to upload Images Into azure blob storage RRS feed

  • Question

  • Hi,

    I have a problem related to uploading images into blob storage due to throughput or storage capacity.

    So, My business has suggest me to use the Round Robin Approach  to uploading images into azure blob storage.

    Business has denied  to use of premium storage.

    Kindly suggest me how to handle this problem or any alternative solution for the same.

    Thanks,

    Chandra

    9213967390

    Wednesday, May 8, 2019 6:48 AM

All replies

  • Hi Chandra,

    Thank you for posting here.

    Since this thread is related to azure blob, you could post in the following forum.

    https://forums.asp.net/1247.aspx/1?Azure+and+ASP+NET

    The Visual C# forum discusses and asks questions about the C# programming language, IDE, libraries, samples, and tools.

    Best Regards,

    Jack


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Wednesday, May 8, 2019 7:02 AM
  • Round robin algorithms process one item at a time and then move to the next one. I don't know that this really solves your throughput problem though as throughput generally is limited and RR doesn't have anything to do with throttling. But the gist of the approach is that you'll enumerate through each image and then upload each one. Once the upload is complete go to the next one. In a pure RR algorithm where processing can take an arbitrary amount of time then you would also need to put a limit on execution time but that doesn't make sense in this case because uploads shouldn't be that long unless you're talking really large files.

    foreach (var image in images)
       UploadImage(image);
    
    

    This can dramatically slow things down if you have a lot of images since they are now being uploaded sequentially. Additionally you'll still have a throttling problem if you can only, say, upload 60 images every 10 minutes but the actual upload takes 2 seconds. If you need to control how much is sent at any one time then you'll need to implement a batching system instead. Irrelevant your code will need to be able to identify which ones have been uploaded and which ones haven't but that is a separate problem.

    If you really need to limit the execution time as well then this can get quite a bit more complicated. You could probably hack something together with tasks and waitable joins but, again, this seems like overkill. It might just be easier to use a limited concurrency task scheduler (one is available in MSDN) that allows you to send all the images to it but it only runs a limited quantity at any one time. This isn't RR but it is a lot easier and allows parallel uploads with a throttled count.


    Michael Taylor http://www.michaeltaylorp3.net

    Wednesday, May 8, 2019 2:10 PM