Simulation is one of the cornerstones in scientific research. The aim of the SimEnv project is to develop a toolbox oriented simulation environment that enables the modeller to handle model related quality assurance matters (Saltelli et al., 2000 & 2004) and scenario analyses. Both research foci require complex simulation experiments for model inspection, validation and control design without changing the model in general.
SimEnv aims at model evaluation by performing simulation runs with a model in a co-ordinated manner and running the model several times. Co-ordination is achieved by pre-defined experiment types representing multi-run simulations.
According to the strategy of a selected experiment type for a set of so-called factors x which represent parameters, initial or boundary values, or drivers of a model M a numerical sample is generated before simulation. This sample corresponds to a multi-run experiment with the model. During the experiment for each single simulation run the factors x are adjusted numerically according to the sample and the factors’ default values. Each experiment results in a sequence of model outputs for selected state variables z of the model M in the space of all addressed factors {X}. Model outputs can be processed and evaluated after simulation generally on the state space and experiment-type specifically on the factor space.
| The following experiment types form the base of the SimEnv multi-run facility: | Example for {X} = (x1,x2) x = sample points o = factor's default value |
| Global Sensitivity Analysis - Elementary Effects Method Qualitative ranking of a large number of factors x with respect to their sensitivity on model output at random trajectories in the factor space {X}. For determination of the most important factors. ( --> = trajectories)
|
 |
| Deterministic Factorial Design Inspection of the model’s behaviour in the factor space {X} by a discrete numerical sampling with a flexible inspection strategy for sub-spaces. For model verification, numerical validation, deterministic error analysis, deterministic control design, scenario analysis and spatial patch model applications. |
 |
| Local Sensitivity Analysis Determination of model (state variable’s z) local sensitivity to factors x. Is performed by finite difference derivative approximations from M. For numerical validation purposes, model analysis, sub-model sensitivity. |
 |
| Uncertainty Analysis - Monte Carlo Analysis Factor space {X} sampling by perturbations according to probability density functions. Determination of moments, confidence intervals and heuristic probability density functions for state variables in the course of experiment post-processing. For error analysis, uncertainty analysis, verification and validation of deterministic models. (only a sub-sample is shown)
|
 |
| Global Sensitivity Analysis - Variance Based Method Orthogonal variance decomposition of model output to first order and total effects of factors by a Monte Carlo re-sampling experiment For decomposition of variance at model output to input factors. (+ = second sample; only sub-samples are shown)
|
 |
| Bayesian Calibration Reduce uncertainty about factor values by deriving a representative sample from factor prior distributions while having measurement values from the system avail-able for model – measurement comparison. For uncertainty analysis and reduction with a Bayesian technique. (Trace plot of a MCMC chain)
|
 |
| Optimization - Simulated Annealing Determination of optimal factor values x by a simulated annealing method for a cost functions derived from z. For model validation (system - model comparison), control design, decision making. (only a sub-sample is shown)
|
 |
SimEnv makes use of modern IT concepts. Model preparation for interfacing them to SimEnv is based on minimal source code manipulations by implementing interface functions into Fortran, C/C++, Python and Java model source code and at shell script level for the addressed factors and for model output. Additionally, interfaces for Matlab, Mathematica and GAMS models are available.
In experiment preparation an experiment type is selected and equipped numerically by sampling the factor space. Experiment performance supports local, remote, and parallel / distributed hardware architectures to distribute work load of the single runs of the experiment.
Experiment specific model output post-processing enables navigation in the complex factor - model output space and interactive filtering of model output and reference data by application of operator chains. SimEnv supplies built-in operators and enables specification of user-defined and composed operators.
Result evaluation is dominated by application of pre-formed visualization modules. SimEnv model output as well as experiment post-processing offer data interfaces for NetCDF, IEEE compliant binary and ASCII format for a more detailed post-processing outside SimEnv.
