LLM Applications in High Energy Astronomy

1. LLM Applications in High Energy Astronomy

Implementing ‘CTAgent’
Summer Student Project
Elisa Jones
Berlin, 03.09.2025

2. Purpose and Theory

What is the plan for CTAgent?
Inspired by Gammapygpt
• CTAgent is a Pydantic model generator
• Streamline data modelling
• Can intake various text formats to generate equivalent code
• Uses a feedback agent
Microsoft AutoGen
• Multiagent Framework
• AssistantAgent allows to configure prompts, API keys, and behaviour
• Tool registration
• Controls the workflow
Models used:
Qwen3 30B A3B
GPT-o3
GPT-5
DESY. | LLM Applications in High Energy Astronomy | Elisa Jones, 03.09.2025
Page 2

3. Agent Workflow

Structure pipeline
JSONExpert
"You are a Python developer specializing in JSON data modeling.
Given JSON data/schema, create a complete, working Pydantic
BaseModel class. Preserve field names and handle nested
objects, arrays, and optional fields. Output only valid, complete
Python code that can be executed immediately."
CodeImprover
"You are a Python code improvement specialist focused on Pydantic
models. When given broken or incorrect Pydantic code, fix all
issues:
1. Add missing imports (from pydantic import BaseModel, typing
imports)
2. Fix syntax errors and type hints
3. Ensure proper BaseModel inheritance
4. Make sure field types match the original data structure
Return only the corrected, complete Python code without
explanations."
DESY. | LLM Applications in High Energy Astronomy | Elisa Jones, 03.09.2025
Page 3

4. Pipeline Example

amc_calib.json
DESY. | LLM Applications in High Energy Astronomy | Elisa Jones, 03.09.2025
Page 4

5. Pipeline Example

amc_calib.json
DEBUG __mp_main__: Uploaded: amc_calib.json, mimetype: application/json, size: 1026
DEBUG __mp_main__: File: amc_calib.json, mimetype: application/json, size: 1026
Ingesting file: amc_calib.json | MIME: application/json | Size: 1026 bytes
Detected as JSON
…
DEBUG ctagent.agents: Starting pipeline with content length: 1303, type: json
DEBUG ctagent.agents: Selected expert: JSONExpert
…
'You are a Python developer specializing in JSON data modeling. Given JSON data/schema, create a complete, working Pydantic BaseModel
class. Preserve field names and handle nested objects, arrays, and optional fields. Output only valid, complete Python code that can be executed
immediately.’
...
INFO ctagent.agents: Code generation successful on attempt 1
DESY. | LLM Applications in High Energy Astronomy | Elisa Jones, 03.09.2025
Page 5

6. Feedback

CodeImprover agent in action
Problems: Syntax error: Syntax error: invalid
syntax (<unknown>, line 1)
Please provide corrected Python code that fixes
these issues.’
DEBUG ctagent.agents: Code has issues, trying to
improve...
…
INFO ctagent.agents: Code generation successful
on attempt 2
DESY. | LLM Applications in High Energy Astronomy | Elisa Jones, 03.09.2025
Page 6

7. Benchmarks

Testing GPT-5 vs GPT-o3 reasoning performance in Gamma-Ray Astronomy
TevCat includes information about sources such as their coordinates, type of source, and discovery date.
Asked the model to return the coodinates for a specified source, with two different reasoning efforts.
TevCatQuestion, TevcatQuestionPossible and TevCatQuestionImpossible were used to test the hallucination rate of the
model
DESY. | LLM Applications in High Energy Astronomy | Elisa Jones, 03.09.2025
Page 7

8. LLM Theory

Kumar, Ramavath Shiva. "Hyperplanes in Higher-Dimensional Spaces: A Practical Explanation with
Example and Implementation in SVM with 2-D and 3-Dimensional Space." Medium, 5 Nov. 2024,
https://shivakumarramavath.medium.com/hyperplanes-in-higher-dimensional-spaces-a-practicalexplanation-with-example-and-implementation-0cdd5caa2c62.
DESY. | LLM Applications in High Energy Astronomy | Elisa Jones, 03.09.2025
Page 8

9.

DESY. | LLM Applications in High Energy Astronomy | Elisa Jones, 03.09.2025
Page 9

10.

Thank you!
Contact
Deutsches ElektronenSynchrotron DESY
www.desy.de
Elisa Jones
DV
Summer Student
elisa.jones@desy.de