← projects

Solarflare Labeler

research software · reproducibility · data pipeline

github ↗︎ documentation ↗︎

i built solarflare-labeler during my software engineering internship at the data intelligence lab at texas christian university. the lab works with solar activity data from noaa, and one of the recurring problems was preparing consistent datasets for machine learning experiments.

the project started as a collection of scripts, but i wanted to turn it into something researchers could actually reuse. i redesigned it as a python package with a small public api, configurable labeling strategies, and reproducible dataset generation so different experiments could share the same pipeline instead of maintaining separate preprocessing code.

before: scripts separate preprocessing per experiment after: package shared api, reproducible pipeline

matching solar events turned out to be the most challenging part. observations come from different datasets and don’t always line up perfectly, so the package matches events, handles duplicate records, validates inputs, and produces training datasets in a consistent format before they’re used for machine learning.

noaa data event matching dedup + validate dataset generation

i also spent a lot of time making the package easier to use. instead of exposing a collection of scripts, the package provides a documented api, clear error messages, automated tests, and a predictable interface that can be reused across different research projects.

FullDiskSingle one frame, whole sun FullDiskSeries time series, whole sun ActiveRegionSingle one frame, one region ActiveRegionSeries time series, one region ↓ all four feed DatasetBuilder

the public api stays small on purpose, one entry point per labeling strategy:

from solarflare_labeler import DatasetBuilder, ActiveRegionSeries

dataset = DatasetBuilder(strategy=ActiveRegionSeries()).build(events)

one thing i wanted from the beginning was reproducibility. given the same inputs and configuration, the package always produces the same dataset. that made it much easier to compare experiments and reduced the amount of manual preprocessing researchers had to do before training models.

same inputs + same config deterministic pipeline same dataset, every time

solarflare-labeler is still used as part of our research workflow at the data intelligence lab, and it’s been one of my favorite projects because it taught me how different research software is from building products. the focus isn’t adding features. it’s making sure results are reliable, repeatable, and easy for someone else to build on.