What's your rule for when an LLM call should NOT be retried?
Everyone has a retry story. Mine: an agent that retried a malformed request eleven times before anyone noticed, each attempt replaying the full conversation. The request was never going to succeed — a 400 doesn't heal — but the retry policy didn't care, because it only looked at "did it throw?" Our
Everyone has a retry story. Mine: an agent that retried a malformed request eleven times before anyone noticed, each attempt replaying the full conversation. The request was never going to succeed — a 400 doesn't heal — but the retry policy didn't care, because it only looked at "did it throw?"
What I haven't settled: provider 529s / overloaded errors during peak. Retry in place, or fail over to a different model immediately and accept the quality change?
What's your rule set — and which error taught it to you?