NASA's Science Mission Directorate now manages more than 150 petabytes of data spanning Earth science, astrophysics, heliophysics, planetary science, and biological and physical sciences. At that scale, matching the right computing environment to the right scientific problem is not just a technical detail – it is mission-critical. Get it wrong, and simulations get bottlenecked or collaborative science grinds to a halt; get it right, and missions such as Artemis II run on infrastructure built for exactly what they need.
Andrew Mitchell, Deputy Chief Science Data Officer within NASA's Science Mission Directorate (SMD), is a program manager and technical leader in scientific data systems and computing services, overseeing initiatives that enhance data management and computing strategies within NASA's Office of the Chief Science Data Officer. Here, he tells SCW how the agency's hybrid strategy pairs high-end computing for tightly coupled simulations – supporting work such as Artemis II and wind tunnel testing at Ames – with cloud infrastructure for interactive, collaborative science.
What does NASA's Chief Science Data Officer role involve?
Andrew Mitchell: Within the NASA SMD, we have five science-focused divisions: Earth science, astrophysics, biological and physical sciences, heliophysics, and planetary sciences. Currently, we manage access, distribution, and analysis for around 150 petabytes of scientific data. Within the Office of the Chief Science Data Officer, we coordinate that work, providing guidance to the science divisions on how to manage those data systems.
How do you design a computing strategy that can support NASA’s different users across the scientific areas that you oversee?
AM: We use a flexible hybrid computing strategy. It combines our high-end computing, our managed cloud environments, such as our science cloud, along with our integrated data services and advanced networking to support mission needs across the entire agency. A good example is with our engineers preparing for the recent Artemis II mission. They relied heavily on our high-end computing to run tightly coupled computational fluid dynamics (CFD) simulations.
Meanwhile, our Earth scientists studying wildfire damage might use our science cloud to analyse satellite observations or collaborate through cloud-native research environments. Rather than forcing every workload into the same infrastructure, we match the computing environment to the scientific problem we are trying to solve. The goal is to give researchers an environment that best supports the work they are trying to accomplish, whether that is an astrophysicist processing telescope data, a planetary scientist, or an engineer running structural simulations.
How is data growth changing NASA's computing and storage needs?
AM: NASA missions are collecting more data at higher resolution over longer periods of time than ever before. Within the Science Mission Directorate, we manage over 150 petabytes of scientific data representing 50,000 unique science datasets. Our scientists expect to work with that information more interactively than they did previously when things were on-premises. Because of that, we are working toward an integrated computing environment where compute, storage, AI capabilities, and network services are not just designed together, but managed together rather than as separate systems. Another big game-changer for us is the use of AI, machine learning, and advanced analytics in our scientific workflows. Recently, we released two foundation models, one for Earth observation and one for solar physics. These are helping researchers identify patterns, prioritise observations and accelerate scientific advancements.
How does NASA make petabyte-scale data interactive for researchers?
AM: That goes back to the hybrid infrastructure we are building. On the cloud side, we heavily use services like S3 (Amazon’s Cloud Object Storage), and we use deep storage options like Glacier (a low-cost cloud storage service from Amazon Web Services (AWS) built for long-term data backup, digital preservation, and archiving). For our on-premises high-end computing, we utilise scratch storage. It is a hybrid approach to managing that 150 petabytes, which will soon grow to exabyte scale.
Is it difficult to keep pace with data growth? How do you convey this challenge to NASA management?
AM: If you had asked me that a couple of years ago, I would have said yes. But management, industry and the world are coming around to the importance and need for data and data storage. This is not a NASA-specific problem. Some of the other things we are looking at include on-the-fly or on-demand processing, so we do not have to store all levels of our datasets at the exact same time. That hybrid approach spans not just the infrastructure itself but also how we manage that data and make it available. We have also been working with our international partners on those strategies.
How does NASA decide which infrastructure fits each workload?
AM: Our decisions are driven by intended outcomes. We evaluate how researchers and engineers will use the data, considering multiple factors: whether they need low latency, large memory, GPU acceleration, AI capabilities, how they will scale in the cloud, and proximity to the data. Some users prefer not to move data. Our measure of success is how effectively users accomplish their mission, not necessarily the technology they use.
For example, in our wind tunnels at the Ames Research Center, engineers performing computational fluid dynamics simulations needed an environment closely tied to the wind tunnels, rather than just the fastest available computer. By connecting our high-end computing systems directly to the wind tunnel workflows, researchers could compare computational results in near real time. This used to take weeks or months, but now they accomplish it within hours. We prefer a right-sizing approach.
How do you determine which workloads go to high-end computing versus the cloud?
AM: First, we begin by understanding what types of computing the mission requires. High-end computing remains the best choice for large-scale, tightly coupled simulations that demand extremely high performance, low latency, and predictable execution, such as CFD, spacecraft design, and climate modelling.
Cloud environments serve researchers who need flexibility, rapid scaling, or collaboration. The cloud market also offers shared data services and modern AI technologies. Some projects use both: a team might prepare data in the cloud, run a large simulation on high-end computing resources, and return results to the cloud for analysis, visualisation, and sharing.
We have a systems engineering-led team that works directly with end users to determine fit-for-purpose based on mission requirements. That can change over time as the research progresses, so it is not stagnant.
Do you see HPC and cloud as complementary parts of a single research ecosystem?
AM: Definitely. Different missions require different computing environments, and some leverage both. Integrating HPC and cloud environments is neither easy nor automatic, presenting real challenges. First, managing data movement costs across environments at petabyte and exabyte scales. Second, aligning security accreditation processes built for different infrastructure models, with the industry still catching up. Third, building a workforce with expertise across both niche environments. We manage this through our hybrid computing strategy, optimising for performance, scalability, and security.
On workforce development, does NASA develop those skills and training, or do you search for external talent?
AM: Both. NASA currently has a major push to in-source capabilities germane to our core mission. On the flip side, where we can leverage industry expertise, we rely heavily on partners. We look to partner with industry rather than compete, including addressing workforce demand.
How does NASA plan long-term infrastructure amid fast-changing science?
AM: We make infrastructure investments that remain viable over time while recognising that scientific priorities and technologies continue to evolve. A good example is our newest supercomputer, Athena, launched at Ames Research Center.
As NASA's largest system, with over 20 petaflops of peak performance, it significantly reduces the agency's supercomputing utility costs. We managed this by housing Athena in a modular supercomputer facility designed for flexibility, accommodating future needs while keeping operating costs low. Under our hybrid computing strategy, we rely on cloud resources for quick access to emerging technologies or rapid scaling to meet changing scientific needs, while balancing the multi-year procurement cycles of high-end computing.
Does that agility allow you to test new technologies in the cloud before bringing them in-house, or do you use dedicated testbed systems?
AM: Both. In addition to testing in the cloud and on our HPC systems, we utilise industry partnerships. We test on emerging infrastructure provided by industry partners, domestic agencies, and international partners, working with organisations such as IBM and Microsoft to access cutting-edge technologies.
How is AI reshaping NASA's scientific computing infrastructure?
AM: One of the biggest changes is that we are no longer designing infrastructure solely to run simulations; we design it to support the complete scientific workflow combining observation, simulation, AI and visualisation. AI has become part of nearly every stage of the research lifecycle, helping prepare data, identify patterns and discover relationships.
We are doing significant work in agentic AI across the agency, which changes infrastructure demands at scale. Where traditional scientific workflows focused on CPUs, there is now increasing demand for GPU acceleration, training systems, massive storage and high-speed networks. AI has become central to our scientific process, but we must ensure those models are verified, validated and applied to support sound science rather than simply speed.
Andrew Mitchell is the Deputy Chief Science Data Officer within NASA’s Science Mission Directorate Office of the Chief Science Data Officer.
