Successful integration of computer simulation with wet-lab research requires the relationship between simulation and the real-world system to be established. Spartan, described in our paper in PLoS Computational Biology, is a package of statistical techniques specifically designed to understand this relationship and provide novel biological insight. These techniques help reveal the influence that pathways and components have on simulation behaviour, offering valuable biological insight into aspects of the system under study.
Spartan is open source, implemented within the R statistical environment, and freely available from both the Comprehensive R Archive Network (CRAN) and on this page below. Use of the package is demonstrated via the tutorial published in the R Journal. Example simulation data for each technique described are available in the tabs below.
Consistency analysis operates by contrasting distributions of responses from stochastic simulations, all generated using the same fixed set of parameter values and containing identical numbers of simulation samples. By varying the number of samples comprising the distributions, the analysis determines the number required to obtain statistically consistent distributions, where the response can be attributed to the parameter values and not affected by randomness in the simulation.
The robustness of a simulation response to parameter alteration can be determined through the use of this approach. A set of parameters of interest are identied, and a range of potential values each parameter could lie within is assigned. The technique examines the sensitivity to a change in one parameter. Thus, the value of each is perturbed independently, with all other parameters remaining at their calibrated value, and responses compared to determine the impact of parameter value change.
Though Robustness Analysis elucidates the effects of perturbations of one parameter, it cannot show any non-linear eects which occur when two or more are adjusted simultaneously. A Global Sensitivity Analysis technique is needed to identify such effects, and to give an indication of the parameters which have the greatest influence on the simulation output. Using this technique, a subset of parameter values are perturbed simultaneously, allowing correlations between parameter value and response to be calculated for each parameter.
Similarly to latin-hypercube sampling, this technique perturbs a selection of parameter values simultaneously. Parameter sampling and response analysis is conducted using the eFAST Approach (extended Fourier Amplitude Sampling Test). Values for each parameter are chosen using fourier frequency curves through a parameters potential range of values. A selected number of values are selected from points along the curve. Though all parameters are perturbed simultaneously, the method does focus on one parameter of interest in turn, by giving this a very different sampling frequency to that assigned to the other parameters. As this is the case, this technique is fairly complex, and we would recommend that those applying the technique study the references in the tutorial.
From Version 1.3, we have added the functionality to combine the statistical analysis available in Spartan with simulations generated in Netlogo.
This integrates Netlogo’s parameter sweep function, Behavior Space, with an extended version of Spartan, enabling local and global sensitivity analyses to be performed on Netlogo models. With the addition of SPARTAN, the researcher can automatically create Netlogo experiment files for both local (individual parameter) and global (latin-hypercube and Fourier frequency) analyses, run these experiments in Netlogo, and receive detailed statistical information on the influence a parameter has on simulation response: vital information for translating a simulation result to a hypothesis grounded in the system being studied. The tutorial on using this technique utilises a slightly modified version of the Virus transmission and perpetuation model that is available in the Netlogo model library (available for download below).
Version 2.3 adds functionality to compare the results of spartan analyses at selected simulation timepoints. This new functionality is described in a new paper currently in review.
The data below accompanies this paper. More information will be added when the paper is released.
Spartan 2.0 offered substantial additional features that significantly altered each of the included techniques. If you wish to use a previous version of Spartan, version 1.3, the tutorials for this version, and suitable tutorial data, are available below:
1. Consistency Analysis:
Tutorial: Consistency Analysis Tutorial (PDF , 277kb)
Tutorial: Robustness Tutorial (PDF , 316kb)
3. Latin-Hypercube Analysis
Tutorial: LHC Tutorial (PDF , 357kb)
4. eFAST Analysis
Tutorial: eFAST Tutorial (PDF , 334kb)
5. Netlogo Analysis:
SpartanV is a Java-Based GUI for Spartan, permitting the specification of the analysis using a wizard style data entry system.
At the current time, SpartanV is only compatible with versions 1.0-1.3 of Spartan. Work is ongoing to make this compatible with Spartan 2.0