Through a successful bid for University pump-priming funds, the Robotics Lab are developing RoboSpartan, an extensive new infrastructure that will permit an increased understanding and optimization of the behaviours of robotic systems, and aid targeted experimentation in hardware. This project transfers expertise developed at the University in analyzing biological simulations, that led to the development of the spartan package of statistical techniques. The platform will possess the capability to:
RoboSpartan is open source, and implemented within Shiny and R. The developing platform is supported by example data and video demonstrations of functionality, detailed in the tabs.
RoboSpartan is currently available via our Github page: https://github.com/kalden/robospartan
We suggest running the download in RStudio. Successful application of RoboSpartan is dependent on a number of R packages:
We are hoping to host RoboSpartan online in the near future. For the moment, the following RoboSpartan apps are available on ShinyApps.io:
- spartan, version 3.0.2 or greater
- packages spartan is dependent upon: lhs, gplots, XML, plotrix, mlegp, ggplot2, neuralnet, psych, mco, randomForest, e1071
- shiny, shinyjs, and shinyBS
- xml2, readr, and DT
Prior to undertaking each analysis, we suggest examining the spartan package itself for a full description of each technique, supported by detailed tutorials and example data:
Relevant Papers on Using Spartan:
Here, and in supporting papers for this tool, we show how RoboSpartan can be used to understand 5 parameters that influence the Omega algorithm used in swarm robotics. We examine five parameters, and two simulation responses.
We are going to generate parameter value sets that change the values of 5 parameters:
In the github repository, the file epuck_omega_algorithm.argos is the simulation that we will modify using RoboSpartan, collating the executed results prior to analysis using the second RoboSpartan app. In the descriptions in the other tabs, we reference other files in the repository that can be used to show the functionality in RoboSpartan.
RoboSpartan can generate parameter value sets for three sensitivity analysis techniques, one local (that changes the value of one parameter at a time), and two global (that change all simultaneously). We have not duplicated the detail of each technique here, instead we refer the reader to the vignettes for the spartan package: (https://cran.r-project.org/web/packages/spartan/vignettes/sensitivity_analysis.html)
If running locally, in the main roboSpartan folder, open app.R in RStudio. On the toolbar above the file editor should be a button labelled "Run App". Click the down arrow beside "Run App", and choose "Run External". Then press "Run App", and the parameter sampling app will open in a web browser. Alternatively you can run this app online at https://robospartan.shinyapps.io/sampling/. In this app you can:
If you now press the "Create Sample" button, a sample is created and shown in the panel on the right hand side of the application. Below the samples you have two buttons: one to download the sample ("Download Data"), and one to download the settings used in generating these samples ("Download settings"). The settings is handy to download, as you can enter this file into the next app that analyses these samples, to save having to input the same parameter and measure information again.
Finally, bottom of the left hand panel is a section to upload and generate ARGoS simulation files. This takes an ARGoS simulation file and alters the values of each parameter in the file, generating one file to match each sample shown on the right hand side. You can try this by entering the parameters and measures for the omega algorithm, described in the second tab, and specify the ARGoS file to modify as the "epuck_omega_algorithm.argos" file included in the git repository. With the file uploaded, pressing "Download Modified ARGoS Files" will download a zip file of all simulation configuration files for the generated sample. To generate scripts to run this experiment on a sun grid engine, press the "Generate SGE Cluster Script" button. This downloads two files: one to run the parameter sets on an SGE, and the second to post-process the results into a format that can be input into the analysis app.
Once you have your parameter value sets and performed the executions, you can use the second RoboSpartan app to analyse the data. To aid demonstration of this process, in the github repository there is a folder, "Test_Settings_and_Results", that contains RoboSpartan settings files and simulation execution results for all three sensitivity analyses. From RStudio, open the app.R file that is contained in the "analysis_platform" folder, again running externally as detailed for the sampling app.
|Robustness Analysis||Sampling-Based Analysis (LHC)||Variance-Based Analysis (eFAST)|
With the app open:
Again to save duplication we do not detail how one should interpret each graph here, instead we refer you to the spartan package vignette that contains all the analyses used by RoboSpartan (https://cran.r-project.org/web/packages/spartan/vignettes/sensitivity_analysis.html)
We have recently shown how machine learning algorithms, trained on a simulation dataset, can speed up and permit execution of intensive statsitical analyses, by predicting simulation output. RoboSpartan App 3 permits training of five machine learning algorithms from an LHC or eFAST dataset, and generation of a ensemble that makes a prediction informed by the predictions of all five, weighting each on algorithms predictive performance on the dataset. For a full description of this demonstration, see https://ieeexplore.ieee.org/document/8374844. To aid demonstration of this process in RoboSpartan, in the github repository there is a folder, "Test_Settings_and_Results", that contains RoboSpartan settings files and simulation execution results.
From RStudio, open the app.R file that is contained in the "Machine_learning_emulator_app" folder, again running externally as detailed for the sampling app. With the app open:
These predictive models can then be used in place of the simulator to perform sensitivity analyses and approximate bayesian computation, and explore the parameter space using a GA. A description of how to do just that can be seen here: https://cran.r-project.org/web/packages/spartan/vignettes/emulation_ensembles.html