On the use of services to support numerical weather prediction

The challenges of building an effective grid-based problem solving environment that truly extends and embraces a computational scientist's traditional tools are multifold. It is far too easy to build simple stovepipes that allow fixed use patterns, that don't extend a scientist's desktop, and fail to encompass the full range of patterns that a scientist needs to find such a problem-solving environment as a liberating and enabling tool. In the LEAD project, we have focused on the most challenging users of numerical weather prediction, namely, the atmospheric science researchers, who are prone to use their own tools, their own modified versions of community codes such as the Weather Research and Forecasting (WRF) model, and are typically comfortable with elaborate shell scripts to perform the work they find to be necessary to succeed, to drive our development efforts. Our response to these challenges includes a multi-level workflow engine, to handle both the challenges of ensemble description and execution, as well as the detailed patterns of workflow on each computational resource; services to support the peculiarities of each platform being used to do the modeling (such as on TeraGrid), and the use of an RDF triple store and message bus together as the backbone of our notification, logging, and metadata infrastructure. The design of our problem-solving environment elements attempts to come to grips with lack of control of elements surrounding and supporting the environment; we achieve this through multiple mechanisms including using the OSGI plug-in architecture, as well as the use of RDF triples as our finest-grain descriptive element. This combination, we believe, is an important stepping stone to building a cyber environment, which aims to provide flexibility and ease of use far beyond the current range of typical problem solving environments.