none
How to setup a persistent data services on azure cloud RRS feed

  • Question

  • I have a business layer that does a client calculation which may take 3 seconds. This could be called by five or six widgets on a page and I would want to persist the data (maybe 30,000 records of calculations and summaries) between calls so they can all summarise the data differently.

    This requires SQL for base data but then extrapolates in memory. Don't really want to store or searlise as both have a cost.

    How would I set up a service like a traditional persistent Windows service but in the cloud?

    The base data for most clients is a few hundred items and this is extrapolated out over time to many thousands of resulting events. This is all point in time as when the client alters anything everything is recalculated so does not need to be continuously persisted. When I make a request from a client page I want to be able to access the pre-populated object in memory and from that fill in the different searialisations of the viewed data and not pull from a database.

    I could set up a VM and make a windows service available. but this sounds like an old pattern.

    Technology stack

    I am using SQL server to store initial data and extrapolating this daily over 30 years. I am using a Azure C# webapp that is efficiently calculating the potentially 100,000 events that occur and then want to server that to multiple client side widgets. This would be ajax calls for different graph data different month summarise, a set of line items, basically the same date for different views on the page.

    At the moment I am recalculating on each call as storing all of the interactions is overkill. At the moment three are three of four calls per page and it is a frustrating pattern to see the same calculation running three or for times for a page load.

    Where a web service layer by design is stateless I am looking for something similar to a windows service which after calculation can preserve the data for different interface calls.



    Thursday, September 7, 2017 1:41 AM

All replies


  • Based on your mention of "At the moment three are three of four calls per page and it is a frustrating pattern to see the same calculation running three or for times for a page load.", you really shouldn't need to have multiple AJAX calls per page running the same calculation on the web server. Perhaps your page could download this extrapolated data once and store it on the page (in the client side) itself and then have the various widgets work off that single copy of the downloaded data. 

    Based on your mention of " This is all point in time as when the client alters anything everything is recalculated so does not need to be continuously persisted.", it doesn't seem like you need the extrapolations  to be stored in SQL. You could potentially have something like Azure Web jobs run periodically and re-generate extrapolations for any base data in SQL that has changed since the last set of re-generations. The extrapolations can potentially be stored in an in-memory cache (like Azure Redis cache). Your web page can fetch the data from the cache if it is available and only fall back to SQL if the data isn't available in the in-memory cache. You will need to serialize this extrapolated data into the cache or use one of the inbuilt data structures provided by the cache service. 

    Wednesday, September 13, 2017 6:29 AM