More than a month after a flooded data center at Stanford University knocked out data processing operations for two NASA sun-studying spacecraft, solar scientists are starting to regain access to information and images beamed to Earth by the instruments.
“We have begun processing most near-real-time [Solar Dynamics Observatory] data products,” JSOC team members at Stanford — short for Joint Science Operations Center — which is spearheading repairs, said in a Jan. 7 update.
A restored database server is “functioning adequately” with back-up servers ready for use, the statement reads. “These data are being processed from the time of the flooding incident going forward.”
That incident occurred on Nov. 26, when a 4-inch-wide (10-centimeter-wide) pipe burst and sprayed water from floor to ceiling inside a Stanford University building, severely damaging roughly 20% of the computer systems in the lab that process and distribute data from SDO, as well as from the Interface Region Imaging Spectrograph (IRIS) spacecraft.
Archives from two of SDO’s three science instruments — the Helioseismic and Magnetic Imager (HMI) and the Atmospheric Imaging Array (AIA), which study the sun in different wavelengths of light — became inaccessible, requiring scientists to rely on alternate sources for research.
SDO has been studying how solar activity affects life on Earth since 2010, and the IRIS probe since June 2013. “Restoration of the database servers that manage the storage of the billions of files in the JSOC repository is our first and foremost challenge — this is our highest priority,” the team said at the time of the flood.
By Dec. 20, the first stage of the processing pipeline for incoming data could be restarted, albeit with a significantly reduced capacity.
Imagery courtesy of the Solar Dynamics Observatory (SDO) is finally back online after the JSOC outage since November. Great news! https://t.co/aqK4Q6XdAY pic.twitter.com/OFZenDBRm2January 8, 2025
Deliveries of ordered replacements to other damaged data servers remained uncertain during the winter break, but “are expected to arrive soon,” according to the JSOC statement.
Those arrivals would help the team assess what can be recovered from damaged disk drives and restore online datasets. “Until then, we will not be able to provide access to the archived SDO data.”
Meanwhile, the spacecraft themselves continued to operate normally in Earth orbit during the outage, so “no science data has been lost,” the statement reads.