Map your AI stack by layer before you pick vendors

Posted on Oct 22 • Originally published at chainofthought.show

It is tempting to start with a model name or a hot framework. I think that order is backwards. I'd start by drawing your stack in layers, the same way the Chain of Thought ecosystem map groups the companies guests represent on the show. Each layer answers a different question, and vendors are answers to those questions.

Foundation models and labs are where everything else sits. On the map that includes Google DeepMind, Cohere, Liquid AI, Writer, Poolside AI, and Scaled Cognition. The decision here is which capability tier you need per task, and what you will pay per token or per request.

That is where the Pareto frontier (cost-performance curve) helps. Each model is a point: capability on one axis, cost on the other. Models on the frontier are the best trade-offs at that moment. A model below the frontier is dominated: something else matches or beats it on both price and capability, so there is no task where it wins on those two axes alone. Labs ship multiple sizes in one family (a larger tier and a smaller, cheaper one) partly because each size pushes the frontier in a different direction. For builders, the curve is an argument against always picking the newest or largest option. I'd map your workflows first, then place a model tier on the curve for each one.

Agent frameworks and orchestration are for building and coordinating agents. The map lists CrewAI, LlamaIndex, and Lyzr. This layer forces a workflow question: how do steps chain together, who holds state, and where do humans approve or override? My worry with picking a framework before you know those patterns is ending up with abstractions that do not match production. Sketch the orchestration you need (single agent, crew, tool loop, retrieval step) and only then shortlist frameworks that match that shape.

Data, retrieval and vector tools ground models in your data. The ecosystem page groups Weaviate, MongoDB, Neo4j, Elastic, Oracle, Databricks, Unstructured.io, Airbyte, Cohesity, and Redpanda here. The decision is how context reaches the model: chunk and embed, structured stores, pipelines, or a mix.

Vector retrieval is good at finding text about a topic. A knowledge graph takes a different shape: nodes (entities) and edges (relationships), so the system can answer connection questions and traverse from one fact to related ones. For grounding, graphs and vectors are complementary; pairing them (sometimes called graph RAG) is one approach to giving the model structured context instead of loosely related chunks. The map lists graph, vector and pipeline companies side by side in this layer. Do not collapse RAG into one product slot until you know whether similarity, connectivity, or both drive your queries.

Evaluation, observability and reliability is where you measure and trust AI in production; Snorkel AI, Honeycomb and Parlance Labs are among the companies on the map. Before you finalize vendors above, decide what "good" means (accuracy, latency, safety, cost per success) and what you will log when the model drifts. Compute and infrastructure (NVIDIA, AMD, Cisco, Fireworks AI) is the serving and hardware layer: where workloads run and how you scale inference. Developer tools and AI coding (Vercel, Graphite, Postman, ChatPRD, OutSystems) is where AI meets the SDLC. Those layers force ops and team questions as well as model questions.

Applied and enterprise AI on the map (Block, IBM, Intercom, Superhuman, Twilio, ServiceNow, and others) reflects companies shipping AI into products and workforces. Use them as reference patterns rather than a shopping list. The launch and production examples on the site collect interviews, clips, and production work that show how those patterns show up in real launches. When a layer on your own map is empty, I would at least ask whether that is a deliberate choice.

For each layer, list two or three candidates from your research (the map is a starting set of names, and listing is not an endorsement). Score them on the same axes: cost at your expected volume, quality on your eval set, integration cost with adjacent layers, and operational burden. For foundation models, plot tiers on the cost-performance curve instead of debating brand. For retrieval, split "similarity" and "relationship" needs before you compare Weaviate-style vector stacks to graph stores. For agents, prototype one critical path in a framework rather than running a six-month platform shootout.

The Chain of Thought podcast is where builders reason through changes in AI infrastructure, agents, and evals; use episode guests as depth on a layer you have already placed on your map rather than as a reason to add a layer you do not need.

# Sketch: layer checklist, not production code LAYERS = [ " foundation_models ", # tier per workflow on cost-performance curve " agent_orchestration ", # state, tools, human gates " eval_observability ", # metrics, traces, regression tests " data_retrieval ", # vectors, graphs, pipelines " compute_serving ", " developer_tools ", " applied_patterns ", # reference architectures only ] def gaps ( stack: dict ) -> list [ str ]: return [ layer for layer in LAYERS if not stack. get ( layer )] Enter fullscreen mode Exit fullscreen mode Checklist Draw seven layers (foundation, agents, eval/observability, data/retrieval, compute, dev tools, applied patterns) and write one decision question per layer. Assign a model tier per workflow using the cost-performance curve; avoid defaulting to the largest model. Split retrieval needs (similarity vs relationships) before picking vector, graph, or paired approaches.

Shortlist vendors only after adjacent layers are defined so integration work is explicit. Fill empty layers from production examples and eval plans before you scale traffic. More on this topic, with the related episodes, is on Chain of Thought.

Subscribe to the Chain of Thought newsletter for new episodes and write-ups like this one.

Drafted with AI assistance from the episode transcripts.