The National Computational Infrastructure (NCI or NCI Australia) is a high-performance computing and data services facility, located at the Australian National University (ANU) in Canberra, Australian Capital Territory. The NCI is supported by the Australian Government's National Collaborative Research Infrastructure Strategy (NCRIS), with operational funding provided through a formal collaboration incorporating CSIRO, the Bureau of Meteorology, the Australian National University, Geoscience Australia, the Australian Research Council, and a number of research-intensive universities and medical research institutes.
Access to computational resources is provided to funding partners as well as researchers awarded grants under the National Computing Merit Allocation Scheme (NCMAS).
The current director is Andrew Rohl.
Contents
Notable staff
Lindsay Botten – former director
Chris Pigram – former CEO of Geoscience Australia and acting director after the retirement of Lindsay Botten.
Sean Smith – former director
Andrew Rohl - current director
Facility
The NCI building is located on the ANU campus in Canberra and uses hot aisle containment and free cooling to cool their computers.
Computer systems
As of June 2020, NCI operates two main high-performance computing installations, including:
Gadi, meaning 'to search for' in the local Ngunnawal language. a 9.26 PetaFLOP high-performance distributed memory cluster consisting of:
145,152 cores (Intel Xeon Scalable 'Cascade Lake' processors) across 3024 nodes
160 nodes containing four Nvidia V100 GPUs
567 Terabytes of main memory
20 Petabytes of fast storage
47 Petabytes of storage for large data files
50 Petabytes of tape storage for archival
HDR Mellanox Infiniband in Dragonfly+ topology (up to 200 Gbit/s transfer)
Tenjin, a 67 TeraFLOP bespoke high-performance partner cloud, consisting of:
1600 Intel Xeon Sandy Bridge cores
25 Terabytes of main memory
160 Terabytes State Disk
Data services and storage
NCI operates the fastest filesystems in the Southern Hemisphere. 20 Petabytes of storage is available for fast I/O, 47 Petabytes is available for large data and research files, and 50 Petabytes is available on tape for archival.
Datasets
NCI hosts multiple data sets that can be used on their computation systems including:
Aboriginal and Torres Strait Islander Data Archive (ATSIDA) which provides Australian Indigenous research data
Australian Astronomy Optical Data Repository (ODR) including:
Anglo-Australian Telescope (AAT) current and selected historical datasets
Southern Sky Survey, using the ANU's robotic SkyMapper telescope at Mount Stromlo Observatory
Australian National Geophysical Collection (300 TB in 2015) including:
Airborne geophysics data
Gravity data set
Seismic survey
High-resolution 'raw' Indian Ocean sea floor data was generated as part of the search for Malaysia Airlines Flight 370.
Research
Research conducted or underway includes:
Southern Sky Survey, using the ANU's robotic SkyMapper telescope at Mount Stromlo Observatory
The Australian Community Climate and Earth System Simulator (ACCESS)
COVID-19 mitigation research
Medical and materials research
History
NCI Australia is a direct descendant of the ANU Supercomputing Facility ANUSF, which existed from 1987 through to 1999. At the turn of the new millennium, the Australian Government pushed ahead with a process to form the Australian Partnership for Advanced Computing (APAC), the foundation of which would be built around a new national computational infrastructure. With its heritage in supercomputing, it was decided that the APAC National Facility would be located at The Australian National University, with the facility ultimately commissioned in 2001.
In 2007, APAC began its evolution into the present NCI collaboration.
The table below provides comprehensive history of supercomputer specifications present at the NCI and its antecedents.
Vayu
The Vayu computer cluster, the predecessor of Raijin, was based on a Sun Microsystems Sun Constellation System. The Vayu system was taken from Sun's code name for the compute blade within the system. Vayu is a Hindu god, the name meaning "wind". The cluster was officially launched on 2009-11-16 by the Government of Australia's Minister for Innovation, Industry, Science and Research, Senator Kim Carr, after provisional acceptance on 2009-09-18.
Vayu was first operated in September 2009 with one-eighth of the final computing power, with the full system commissioned in March 2010. Vayu had the following performance characteristics:
Peak performance: 140 TFLOPS
Sustained: 250K SPECfp rate
Resources: 110M hrs p.a.
The system comprised:
11936 CPUs in 1,492 nodes in Sun X6275 blades, each containing
two quad-core 2.93 GHz Intel Nehalem CPUs
24Gbyte DDR3-1333 memory
24 GB Flash DIMM for swap and job scratch
total: 36.9 TB of RAM on compute nodes
Dual socket, quad-core Sun X4170, X4270, X4275 servers for Lustre fileserving
approx 835 TB of global user storage
The power consumption of the full 11936 CPU system was approx 605 kW, but all the power was intended to be from green energy sources.
System software for the Vayu cluster includes: