Online international collaboration shows reliability of quantum simulations for materials design

Posted on Thursday 31 March 2016

Through an online collaboration, they demonstrated that the most recent generations of codes agree well, in contrast to earlier generations. The study, which is the first systematic investigation of the reproducibility of quantum simulations, appears in the journal Science.

It is a corner stone of science that independent yet identical experiments should produce identical results. Only in this way can science identify ‘laws’, which lead to new insight and new technologies. Several recent studies have identified, however, that such reproducibility does not always come spontaneously. In scientific areas as diverse as psychology research and genetic research, the studies identified occasions where repeating previous experiments led to significantly different results.

Predictions by computer codes also require caution, since the way in which theoretical models are implemented may affect simulation results. This can affect any field of research that critically depends on computer simulations. In the study and design of materials, for instance, several independent software packages based on quantum physics are available.

Dr Matt Probert and Dr Phil Hasnip, of the Department of Physics at York, are two of the lead authors of CASTEP, a leading UK-developed package. Scientists now use these codes increasingly often in automated procedures with limited human supervision. It is therefore essential to know to what extent predicted materials properties depend on the code that has been used.

Despite the need for reliable property predictions of materials, the reproducibility of quantum simulations had not been investigated systematically before. This is mainly because there is no single person sufficiently skilled in all existing codes.

Scientists from the University of York joined forces with more than 60 colleagues, bringing together the expertise of over 30 prominent institutions. The researchers investigated 40 different methods to describe the influence of pressure in 71 different crystals. The international composition of the team meant discussions and collaboration were mainly conducted via online tools – similarly to the way people collaborate to write Wikipedia.

The team demonstrated that, although a few of the older methods clearly yield deviating results, predictions by recent codes are entirely equivalent. They defined a quality criterion that allows the verification of future software developments against the extensive database they have established.

Dr Probert said: "One of the key features of science is reproducibility - two different groups doing the same experiment should get the same result. There are a number of computer packages like CASTEP which all use Density Functional Theory (DFT) as their theoretical foundation. Last year over 20,000 papers were published that used these techniques - and yet the issue of reproducibility of DFT results had not been successfully tackled - until now."

Dr Hasnip added "This paper demonstrates a major step forward for the field, with many of the developers of the leading European codes working together on this community project. This will bring tangible benefits to many different fields of science."

New test data are continuously added to a publicly available website: http://molmod.ugent.be/DeltaCodesDFT

The researchers involved hope their work will contribute to higher standards for materials property simulations, and that it will assist the development of improved simulation codes and methods.