Empty AI observability panels: a product teardown of SigNoz

I’ve been building ReplayLens, a project around AI agent quality and regression reporting. That got me interested in an engineering question: how do we make complex technical systems easier to understand when something isn't working as expected?

While reading SigNozs public AI observability documentation I noticed a product-design question.

When a developer opens an observability dashboard and sees panels how quickly can they figure out why?

This is a documentation-based teardown. I have not reproduced a bug in a running SigNoz instance so I'm treating this as a hypothesis to investigate than a confirmed defect.

SigNozs AI Observability features help developers inspect LLM and agent workloads, including token usage, cost, latency, errors time to first token and tool calls.

The documentation explains that these panels depend on telemetry arriving in the expected format.

gen_ai.request.model is used by LLM panels and the model filter.

gen_ai.usage.input_tokens and gen_ai.usage.output_tokens support token and cost panels.

gen_ai.tool.name is used by tool-call panels.

AI Observability requires traces; integrations that send metrics and logs do not appear in these views.

SigNoz also provides Attribute Mapping to normalize attributes emitted under names by instrumentation libraries.

These are engineering requirements. Standardized attributes make telemetry more consistent across frameworks and providers.

The question I want to explore is how directly the product helps a developer identify which requirement is missing.

Imagine instrumenting a LLM application and opening AI Observability to understand its token consumption.

Traces arrive,. The expected GenAI attributes aren't present.

The instrumentation uses attribute names.

Model pricing isn't configured, so cost calculations aren't available as expected.

A self-hosted installation is missing a required traces-pipeline processor.

The documentation describes the prerequisites and configuration options. My product hypothesis is that developers would benefit from seeing more of this information directly in the workflow.

An empty panel shouldn't force a developer to guess which layer is responsible.

I have not verified how every one of these cases currently appears in the product. That would require a test.

A small diagnostic panel could report whether the system has detected:

Required processors for self-hosted setups

Each status could link to the appropriate setup instructions.

The goal would be to reduce the time between seeing a panel and identifying the next useful debugging step.

Instead of a generic empty chart a panel could explain the requirement it needs.

“We haven't found matching spans with the attributes this panel uses. Inspect a span. Review your attribute mapping.”

The exact message should depend on what the system has checked. It shouldn't tell someone an attribute is missing if that hasn't been verified.

SigNozs Attribute Mapping documentation already describes testing mappings against a sample span and inspecting the resulting attributes.

I'd explore bringing that validation flow to the moment a developer encounters missing data:

Highlight expected attributes that're present or absent.

Explain which panels the resulting attributes support.

The idea isn't to replace the existing configuration. It's to make the connection between configuration and outcome easier to understand.

I'd test this against a running instance before filing an issue or calling it a bug.

A trace containing the expected GenAI attributes.

An application sending metrics and logs but no traces.

A self-hosted setup with the required processors configured correctly and incorrectly.

For each case I'd record the UI state, available diagnostics steps needed to resolve the problem and whether a developer could identify the cause without repeatedly consulting documentation.

That would establish whether this is an usability gap, an already well-handled case or an opportunity for further improvement.

I wouldn't change the underlying commitment to OpenTelemetry conventions.

Standard attribute names make it easier to compare telemetry across applications and attribute mapping addresses the reality that different instrumentation libraries can emit names.

The documentation also makes important prerequisites explicit.

I'd build on those strengths by making the diagnostic experience as clear as the setup guidance.

Developer tools need to do more than display the data when everything is configured correctly.

They should also help developers understand why the expected data isn't there.

For AI observability making the gap between instrumentation and useful dashboards easier to diagnose is a product question.

I'd be interested, in how the SigNoz team thinks about telemetry readiness and empty-state diagnostics.

This is a documentation-based product teardown. No live benchmark or confirmed bug reproduction is claimed.