Research: SQL Server and high performance applications RRS feed

  • General discussion

  • There are three papers related to SQL Server and high performance applications (though not computational clusters specifically) that you might find interesting. These came out of collaboration between Jim Gray (Microsoft Research), Gerd Heber (Cornell Theory Center) and others. The specific example used is a FEA application re-architected to utilize SQL Server.


    From the summary:

    "The first article laid out the requirements for a modern Finite Element Analysis system. It argued that one needs an integrated system to design the model, create the mesh, run and steer the simulation, manage the data outputs of each of these steps, and then manage the visualization and analysis of the resulting data. The second article described how we mapped the data management problem onto a modern relational database system, Microsoft SQL Server 2005, and discussed the benefits and limitations of that approach. [The third] article discussed the actual analysis of the resulting simulation data and the use of an off-the-shelf visualization system OpenDX. It showed OpenDX lets us interactively analyze a multi-terabyte database resulting from a week-long simulation run. This system has been deployed and is being used by a dozen people at Cornell and in the SIPS project."


    "We believe our experience with FEA of polycrystalline metals has broader applicability. Most of the FEA simulations we see seem to have similar data management and visualization problems. It seems likely that the techniques described here could be applied in other disciplines."


    Supporting Finite Element Analysis with a Relational Database Backend; Part I:

    There is Life beyond Files

    Gerd Heber; Jim Gray

    Word 237 Kb; PDF 174 Kb

    We show how to use a Relational Database Management System in support of Finite Element Analysis. We believe it is a new way of thinking about data management in well-understood applications to prepare them for two major challenges, - size and integration (globalization). Neither extreme size nor integration (with other applications over the Web) was a design concern 30 years ago when the paradigm for FEA implementation first was formed. On the other hand, database technology has come a long way since its inception and it is past time to highlight its usefulness to the field of scientific computing and computer based engineering. This series aims to widen the list of applications for database designers and for FEA users and application developers to reap some of the benefits of database development.


    Supporting Finite Element Analysis with a Relational Database Backend Part II:

    Database Design and Access

    Gerd Heber; Jim Gray

    Word 802 Kb; PDF 586 Kb

    This is Part II of a three articles on using databases for Finite Element Analysis (FEA). It discusses (1) db design, (2) data loading, (3) typical use cases during grid building, (4) typical use cases during simulation (get and put), (5) typical use cases during analysis (also done in Part III) and some performance measures of these cases. It argues that using a database is simpler to implement than custom data schemas, has better performance because it can use data parallelism, and better supports FEA modlularity and tool evolution because database schema evolution, data independence, and self-defining data.


    Supporting Finite Element Analysis with a Relational Database Backend; Part III:

    OpenDX – Where the Numbers Come Alive

    Gerd Heber; Chris Pelkie; Andrew Dolgert; Jim Gray; David Thompson

    Word 5456 Kb; PDF 2943 Kb

    In this report, we show a unified visualization and data analysis approach to Finite Element Analysis. The example application is visualization of 3D models of (metallic) polycrystals. Our solution combines a mature, general purpose, rapid-prototyping visualization tool, OpenDX (formerly known as IBM Visualization Data Explorer) [1,2], with an enterprise-class relational database management system, Microsoft SQL Server [3]. Substantial progress can be made with established off-the-shelf technologies. This approach certainly has its limits and we point out some of the shortcomings which require more innovative products for visualization, data-, and knowledge management. But, overall, the approach is a substantial improvement in the FEA lifecycle, and probably will work for other data-intensive sciences wanting to visualize and analyze massive simulation or measurement datasets.



    Friday, January 18, 2008 10:15 AM