Sometimes the best fix is removing the complex thing
A memory problem in a Go + cgo + GStreamer pipeline, and a fix that consisted of not having the pipeline.
Recently we spent quite a lot of time trying to solve a memory issue in a video-processing pipeline built with Go + cgo + GStreamer.
Sounds simple. In practice we were dealing with:
- cgo boundaries;
- GStreamer object lifetimes — buffers, samples, refs and unrefs;
- memory that the Go GC does not really control;
- random spikes inside the bindings that were hard to reproduce and harder to explain.
At some point the question stopped being how do we fix this and became do we need this complexity inside our Go process at all?
The answer was no. We replaced the in-process GStreamer pipeline with a plain ffmpeg subprocess call.
ffmpeg is not a magic solution for every case. But for ours, process isolation turned out to be worth far more than theoretical elegance.
It was a good reminder. Engineering is not always about making the complex thing work. Sometimes it’s about removing the complex thing completely.
