Building a Production RAG API in Go with pgvector and Gemini

Lessons from building the Restaurant Suggestion System, a Go/Gin backend with a conversational recommendation API.

A RAG demo takes an afternoon: embed some documents, run a similarity search, paste the results into a prompt. A RAG API you can put in front of real users takes longer, because most of the work is not the LLM call. It's everything around it.

I built the Restaurant Suggestion System (RSS) as a Go/Gin backend where users chat to get dish and restaurant suggestions. The answers come from RAG over real menu data in PostgreSQL with pgvector and Gemini embeddings. Because the recommendations have to respect allergies, "the model usually gets it right" was never good enough.

This post walks through the shape of that pipeline and the Go patterns that made it dependable. The snippets are simplified and focused on the ideas, not copied from the private repo.

Every chat request goes through the same steps:

The rule behind all of it: the LLM explains and converses. It does not decide what is safe or what exists.

It's tempting to put everything into the embedded text and let similarity sort it out. Don't do that for anything safety-related. Allergens belong in a column you can filter on exactly.

CREATE EXTENSION IF NOT EXISTS vector; CREATE TABLE menu_items ( id BIGSERIAL PRIMARY KEY, restaurant_id BIGINT NOT NULL REFERENCES restaurants ( id ), name TEXT NOT NULL, description TEXT NOT NULL, allergens TEXT [] NOT NULL DEFAULT '{}', price_cents INT NOT NULL, is_available BOOLEAN NOT NULL DEFAULT TRUE, embedding vector ( 768 ) NOT NULL ); CREATE INDEX ON menu_items USING hnsw ( embedding vector_cosine_ops ); CREATE INDEX ON menu_items USING gin ( allergens ); Enter fullscreen mode Exit fullscreen mode The dimension (768 here) must match the embedding model's output configuration. Pick one, store it in config, and fail fast at startup if they disagree. A silent dimension mismatch is a miserable bug to find in production.

RSS uses Gemini embeddings in two places: when indexing menu items, and during onboarding to capture user preferences as a vector. Wrap the provider behind a small interface so the rest of the code doesn't care which SDK you use:

type Embedder interface { Embed ( ctx context. Context, text string ) ([] float32, error ) } type GeminiEmbedder struct { client * genai. Client model string dims int32 } func ( g * GeminiEmbedder ) Embed ( ctx context.

Context, text string ) ([] float32, error ) { ctx, cancel:= context. WithTimeout ( ctx, 5 * time. Second ) defer cancel () res, err:= g. client.

Models. EmbedContent ( ctx, g. model, genai. Text ( text ), & genai.

EmbedContentConfig { OutputDimensionality: & g. dims }, ) if err!= nil { return nil, fmt. Errorf ( "embed: %w", err ) } if len ( res. Embeddings ) == 0 || len ( res.

Embeddings [ 0 ]. Values )!= int ( g. dims ) { return nil, errors. New ( "embed: unexpected embedding shape" ) } return res.

Embeddings [ 0 ]. Values, nil } Enter fullscreen mode Exit fullscreen mode Two habits worth copying: a hard timeout on every external call, and a shape check on the response. The interface also makes tests easy, since a fake embedder returns fixed vectors.

This is the most important design decision in the system. Allergen and health filters run in the retrieval query, so excluded items never reach the model. A prompt like "don't suggest peanuts" is a request. A WHERE clause is a guarantee.

type Candidate struct { ID int64 Name string Allergens [] string PriceCents int Distance float64 } const retrieveSQL = ` SELECT id, name, allergens, price_cents, embedding <=> $1 AS distance FROM menu_items WHERE is_available AND NOT (allergens && $2::text[]) ORDER BY embedding <=> $1 LIMIT $3` func ( r * Repo ) Retrieve ( ctx context. Context, q [] float32, exclude [] string, k int ) ([] Candidate, error ) { rows, err:= r. db. Query ( ctx, retrieveSQL, pgvector.

NewVector ( q ), exclude, k ) if err!= nil { return nil, err } defer rows. Close () var out [] Candidate for rows. Next () { var c Candidate if err:= rows.

Scan ( & c. ID, & c. Name, & c.

Allergens, & c. PriceCents, & c. Distance ); err!= nil { return nil, err } out = append ( out, c ) } return out, rows.

Err () } Enter fullscreen mode Exit fullscreen mode One pgvector gotcha: with an approximate index such as HNSW, filtering happens after the index returns its nearest neighbours. With a strict filter you can get fewer than k rows back. pgvector 0.8+ supports iterative index scans ( SET hnsw.iterative_scan = relaxed_order; ), which keep scanning until enough rows pass the filter. On older versions, raise hnsw.ef_search or over-fetch and trim.

Either way, write a test where most rows are filtered out and check you still get results.

Vector distance is one signal, not the ranking. RSS ranks filtered candidates with a deterministic scoring engine, so the same input gives the same order and every recommendation can be explained.

type Weights struct { Similarity, Preference, Budget float64 } func Score ( c Candidate, prefSim float64, budgetCents int, w Weights ) float64 { s:= w. Similarity * ( 1 - c. Distance ) + w.

Preference * prefSim if budgetCents > 0 && c. PriceCents <= budgetCents { s += w. Budget } return s } func Rank ( cands [] Candidate, scores map [ int64 ] float64 ) { sort.

SliceStable ( cands, func ( i, j int ) bool { si, sj:= scores [ cands [ i ]. ID ], scores [ cands [ j ]. ID ] if si!= sj { return si > sj } return cands [ i ].

ID < cands [ j ]. ID // stable tie-break }) } Enter fullscreen mode Exit fullscreen mode Note the tie-break on ID. Without it, equal scores can shuffle between runs, and your tests become flaky.

The signals here are illustrative; the point is that ranking lives in plain Go you can unit-test, not in a prompt.

Only the top candidates go into the prompt, each with its ID. A pattern I recommend on top of that: ask the model for structured output that references those IDs, and validate the response before returning it:

type LLMAnswer struct { Message string `json:"message"` ItemIDs [] int64 `json:"item_ids"` } func validateAnswer ( a LLMAnswer, sent [] Candidate ) error { allowed:= make ( map [ int64 ] bool, len ( sent )) for _, c:= range sent { allowed [ c. ID ] = true } for _, id:= range a. ItemIDs { if! allowed [ id ] { return fmt. Errorf ( "model referenced unknown item %d", id ) } } return nil } Enter fullscreen mode Exit fullscreen mode If the model mentions something you didn't give it, that's a hallucination by definition, and you can detect it in a few lines.

LLM APIs time out, hit rate limits and have outages. RSS handles that by degrading gracefully: if the LLM call fails, the API still returns the filtered, scored candidates from vector search. In the sketch below, an answer that fails validation takes the same path:

func ( s * Service ) Recommend ( ctx context. Context, req ChatRequest ) ( ChatResponse, error ) { cands, err:= s. retrieveAndRank ( ctx, req ) if err!= nil { return ChatResponse {}, err // no data, no answer } genCtx, cancel:= context. WithTimeout ( ctx, 8 * time.

Second ) defer cancel () ans, err:= s. llm. Answer ( genCtx, req. Message, cands ) if err == nil { err = validateAnswer ( ans, cands ) } if err!= nil { s. log.

Warn ( "llm unavailable, using vector-only fallback", "err", err ) return fallbackResponse ( cands ), nil } return ChatResponse { Message: ans. Message, ItemIDs: ans. ItemIDs, Source: "llm" }, nil } Enter fullscreen mode Exit fullscreen mode Because the safety filters ran before the LLM, the fallback is just as safe as the full path.

It's less chatty, not less correct. Returning a Source field also helps the frontend and your dashboards see how often you're degrading.

The RAG pipeline is maybe a third of the system. The rest is ordinary backend work that still matters:

Go suits this well: strict types for model output, context deadlines on every call, and a single binary that's easy to run next to Postgres.

Pradip Parmar is a Go backend and AI agent engineer who has been writing Go since 2018. He builds RAG systems, MCP servers and LLM agents for businesses, and previously led the Python-to-Go migration of the Flights backend at MakeMyTrip. If you need a Go AI agent developer for a RAG or agent project, you can reach him on LinkedIn, Upwork or Contra, or see his code on GitHub.