Enabling Remote Data Analysis During the COVID Pandemic and Beyond

The Covid-19 pandemic brought about challenges to the way ISIS Neutron and Muon Source (ISIS) analysed data, but their collaboration with STFC Scientific Computing enabled them to overcome this with a transformative new platform and a long-lasting twist to remote working.
Around the start of the Covid-19 pandemic, the ISIS Neutron and Muon Source (ISIS) had a long shutdown planned for several major upgrade projects. This, combined with the uncertainty the pandemic caused, led to the scientific and technical teams finding new ways of working. Something that was identified early was the need to ensure ISIS users could interact with their experiment data just as they would have done if they had been physically present.
This included seamless access to the raw data and calibration files, the ability to have multiple experimental team members look at the data together with the instrument scientists, powerful computing resources for cases when data processing and analysis is computationally expensive, and access to all the right software tools. This all needed to be available through an easy-to-use platform, ideally a web browser.
IDAaaS has since provided thousands of sessions, with the users of these ’virtual machines’ being able to cover research across the full range of the ISIS portfolio – from battery materials to barocalorics, spin liquids to solar cells, chiral magnets to cancer-causing proteins, and much more besides. This resource has allowed hundreds of individual users to access the compute resource their need to view their experiments and analyse their data at any one time.
ISIS Data Analysis as a Service (IDAaaS)
Before the pandemic struck, ISIS and Scientific Computing (SC) had already been collaborating on an STFC Cloud-based data analysis platform called IDAaaS. This was being developed and used for data from a subset of the ISIS instruments that had particularly large computing resource requirements.
Once the decision had been made to attempt remote user experiments across ISIS, the platform had to be expanded significantly over a relatively short timescale. The Scientific Computing and ISIS teams defined what the new IDAaaS environments needed to look like and implemented a much-expanded service that would be robust enough to meet demand from a hugely increased number of remote users.

IDAaaS has been a game changer in how we package and deploy software for the user community. It has provided a consistent, repeatable interface to the facility and event acts as a great test bed for interoperability of different codes and approaches to delivering software. The IDAaaS team have really catered to the needs of ISIS as a facility and have taken the time to understand our requirements and deliver them on relatively short timescales. Access to this resource has greatly simplified what could have been an even more taxing set of user cycles during the pandemic.
Lamar Moore, ISIS Neutron and Muon Source Scientific Software Group Leader
Providing a service that is available 24/7 has also allowed greater collaboration, enabling users from different time zones to run remote experiments. These benefits will continue to be available for years to come and will undoubtedly shape future hybrid approaches for providing access to research facilities.

I have received lots of positive comments from users about how straightforward IDAaaS has made the experience of viewing and analysing their data when running experiments remotely, which aids their ability to optimise their precious beamtime. ISIS scientists have found IDAaaS to be invaluable, and it has really offered a step change in how we think about what can be done with collaborative data analysis. A particularly positive point is that users can continue to make use of IDAaaS after their experiment has finished, and it is to be hoped that this will help them to analyse their data more rapidly, leading to greater scientific success for ISIS.
Russell Ewings, ISIS Neutron and Muon Source Excitations Group Leader
The system exceeded expectations during the pandemic and has proved to be so popular with users and instrument scientists that it continues to be maintained and expanded.

This is a great example of how teams can work together across the organisation to produce a platform that has changed the way our facility users can do their work. The combination of the expertise in ISIS, development of the IDAaaS platform in SC and the utilisation of the STFC Cloud has meant we can offer the users access to their data, the software they need for analysis, and vast amounts of processing, all through a browser from anywhere in the world.
Gordon Brown, Scientific Computing Head of Software Infrastructure Division
