Chatbots im Vergleich: Bewertungs-Pipelines für Bewerbervorauswahl aufbauen

Use a chatbot service comparison as a way to choose a replaceable scoring component, not as the hiring decision. For a media company scoring candidates against a job rubric, compare OpenAI ChatGPT, Claude, and Gemini by the contract around the model: preserved evidence, explicit uncertainty, rejected malformed output, and a request that can move between providers without changing its meaning.

The tempting implementation sends a resume and rubric to one service, asks for a score from 1 to 10, and stores the number. That ships quickly. It is difficult to audit and painful to replace. A better first version produces four bounded states per criterion, validates the result, and keeps each provider request behind one adapter. Measure human agreement, invalid-output rate, latency, and token use before turning a service comparison into a product decision.

This is the useful answer to the OpenAI ChatGPT versus Claude versus Gemini question. Their names belong in the test matrix, not in the hiring rule.

A score without evidence is a misleading number. Ask for a criterion-level result with one of meets, partial, does_not_meet, or insufficient_evidence, plus evidence, a rationale, and confidence with a defined meaning. Confidence describes evidence strength; it does not promise job performance.

For a senior audio producer, a rubric might require a relevant project and role, an editorial decision, and a cross-functional example. The final score can be a deterministic function of those states. The model extracts and classifies evidence; application code applies weights and decides when a human must review. If a prompt revision changes a weight, the stored score should not change unless the scoring version changes too.

type CriterionStatus = | " meets " | " partial " | " does_not_meet " | " insufficient_evidence "; type CriterionResult = { criterionId: string; status: CriterionStatus; evidence: string []; rationale: string; confidence: number; }; type CandidateAssessment = { rubricVersion: string; results: CriterionResult []; needsHumanReview: boolean; }; Enter fullscreen mode Exit fullscreen mode Reject empty evidence for meets and partial, confidence outside 0 through 1, and unknown criterion IDs. Reject extra fields if storage cannot preserve them. A fallback that stores half-valid JSON creates a quiet audit problem.

Portability is usually described as moving an API call. The harder move is preserving semantics. OpenAI, Claude, and Gemini can sit behind a common interface, but request shape, response handling, model lifecycle, safety controls, and structured-output behavior still need separate treatment. A shared TypeScript function does not erase those boundaries.

The adapter accepts domain data and returns the domain contract. It should not leak a vendor response object into scoring or persistence.

This boundary is where a small implementation earns its keep. The scoring worker can retry a transport failure without knowing which service produced it, while the reviewer can see that an assessment was retried and still inspect the same evidence fields. Persistence does not need a new migration for every provider-specific response property. The trade is a little adapter code up front, paid back whenever a model identifier, response format, or operational policy changes.

type ScoreRequest = { rubricVersion: string; rubric: string; candidateText: string; }; type ScoreResponse = { assessment: CandidateAssessment; metadata: { provider: string; model: string; requestId?: string }; }; interface CandidateScorer { score ( request: ScoreRequest ): Promise < ScoreResponse >; } Enter fullscreen mode Exit fullscreen mode Each implementation translates the request and normalizes the response. The interface stays boring. That makes a provider swap a controlled experiment instead of a rewrite of persistence, review screens, and background jobs.

The trade-off is real: a lowest common denominator can hide useful capability exposed by one service. Use an explicit capability record rather than provider-specific branches throughout the application. An adapter can report whether it supports the response mode required by the current contract, and the job can stop before a candidate receives an assessment.

Compare the same five things for OpenAI, Claude, and Gemini: contract support, refusal behavior on sensitive input, usable context for the actual resume set, latency distribution, and evidence extraction. Do not compare a feature list to a hiring outcome.

Start with a fixed set containing easy matches, clear non-matches, sparse resumes, contradictory dates, and adjacent experience. A set of 30 reviewed examples can expose contract errors; it cannot establish broad model quality. Keep it versioned and remove identifying candidate data from fixtures.

Run the same rubric and prompt versions through each adapter. Distinguish three failures: no response, invalid contract, and a valid assessment that disagrees with the human label. Transport errors belong in a queue policy. Invalid structure belongs in adapter telemetry and a bounded retry or review path. Substantive disagreement belongs in rubric review. Retrying the third case only makes the system slower.

Record a correlation ID, rubric version, prompt version, provider, model, validation outcome, and latency. Avoid logging full resumes by default. Candidate material is sensitive, and a debug log can become a second, poorly governed copy of the hiring dataset.

A hiring rubric has its own vocabulary and distribution. The right comparison is the one that survives the application’s real inputs.

Use an asynchronous path. A reviewer submits a candidate, a job records rubric and prompt versions, the scorer returns a validated assessment, and the review interface shows evidence before any aggregate score. A timeout produces a pending state, not a fabricated zero. An invalid response produces a reviewable error state, not an empty assessment.

Keep the decision policy outside the model. needsHumanReview can be true when a criterion is insufficient_evidence, when candidate text contains conflicting dates, or when validation required a retry. The exact policy belongs to the hiring workflow and should be tested like any other business rule.

Prompt injection is part of this design problem. A resume can contain instructions aimed at the model rather than facts about the candidate. Treat candidate text as untrusted data, delimit it clearly, and instruct the scorer to extract evidence rather than follow commands found in the text. OWASP’s LLM guidance is a useful threat-modeling reference; the relevant control is the boundary between instructions and imported content.

The practical decision is to choose the service that passes the same contract and review set for the workload you actually have, then keep the adapter replaceable. Do not let a chatbot name become a hiring rule. A portable pipeline leaves room to change providers, prompts, or review thresholds without losing the evidence trail that makes a score defensible.