when i joined the data intelligence lab, i had never worked in a research environment before. i expected to spend most of my time training models, but i quickly realized how much research depends on good software. before anyone can experiment with machine learning, someone has to collect the data, organize it, clean it, and make sure everyone is working from the same foundation. that’s where i spend most of my time.
most of my work is focused on building software for researchers, not on the models themselves. i work with noaa solar observations, help automate data pipelines, and build flare-indexer, an open-source python package that turns solar image timestamps and a flare catalog into labeled datasets for flare prediction. it started as a set of scripts, and turning it into a package meant giving it a small public api, labeling for both the whole sun and individual active regions, support for image sequences, and tests, so a change to the labeling rules couldn’t quietly change a dataset. since then it has grown to handle real-data loaders, configurable matching rules, and time-based sequences. a lot of the actual work is less glamorous than that sounds: reproducibility, documentation, testing, and making sure a script i wrote once could still be trusted six months later.
that was the part that surprised me most. i always imagined research as people training models and reading papers, and i didn’t realize how much engineering happens before any model exists. building software other people could actually trust turned out to matter just as much as building anything accurate.
it also made me realize i like building the tools other people build on top of, more than i like the finished product itself. working in the lab has changed how i see the department, too. instead of only seeing professors in a classroom, i get to collaborate with researchers and see how academic work actually happens behind the scenes.