Speaker
Description
The Space Weather and Heliophysics research community, supported by the Community Coordinated Modeling Center (CCMC, https://ccmc.gsfc.nasa.gov), provides a collaborative platform for space weather models and comparisons with data. The growing demand and increasing complexity of the modern models puts a practical limit on a manual diagnosis of model faults in otherwise automated pipelines, including CCMC’s Runs-on-Request (ROR) and Instant Runs (IR) services. To reduce this bottleneck, we have developed a prototype agentic AI system designed to streamline the error analysis and triage of simulation runs. Our preliminary tests show that the system can correctly recognize and suggest remediation for a number of complex simulation errors including numerical instability, grid mismatch, input configuration issues, and others.
The system utilizes a Local Large Language Model (LLM) equipped with a suite of curated, diagnostic tools to investigate simulation failures. The system employs a multi-stage reasoning pipeline consisting of an Investigator (to build an evidence packet with exact file and line citations), a Critic (to attack unsupported causal claims and preserve contradictory evidence), and a Planner (to determine the root cause and suggest a discriminating solution). Furthermore, we implemented a custom context-budget manager that summarizes aged tool results, allowing the agent to parse massive log files without exceeding token limits.
This presentation will detail the current triage architecture, demonstrate its operational effectiveness, and discuss the future transition from static metadata curation toward active, multi-agent scientific workflows.