Validating LLM output like any other API response

It's tempting to treat an LLM API call as a black box that either works or doesn't. In practice, it behaves more like any other third-party API: sometimes it returns exactly what you asked for, sometimes it returns something close but malformed, and sometimes it returns something confidently wrong.

Building an AI-assisted clinical entity extraction pipeline made this concrete. The task was straightforward to describe turn a consultation note into structured fields like symptoms, diagnosis, and medication but the note itself might be in more than one language, informally phrased, or incomplete.

The naive version of this pipeline sends the note to the model and writes whatever comes back straight into the patient record. That's a mistake for the same reason it would be a mistake to write an unvalidated third-party API response straight to a database: you have no guarantee the shape is what you expect.

The version that held up added a validation layer between the model call and the write. Output gets checked against an expected schema before it's persisted — missing fields, unexpected types, or clearly malformed structure get caught and handled explicitly, rather than silently corrupting a patient's record.

One subtle failure mode worth calling out: on audio-based input, transcription quality dropped sharply when the upload didn't declare an explicit mime_type. It wasn't a total failure it was a quiet, hard-to-notice one, which is the worst kind. The fix (declaring mime_type explicitly) was simple once found, but it's a good reminder that AI pipeline bugs aren't always in the model call itself.

The underlying discipline is the same one that applies to any backend integration: define the contract, validate what comes back against it, and design for the case where the other side doesn't hold up its end.