Logs and errors

Grafana Loki

Connect Grafana Loki, self-hosted or on Grafana Cloud, to TraceX with a logs:read token and an optional stream selector.

Search Loki logs, self-hosted or on Grafana Cloud, with a stream selector you choose.

Why connect Grafana Loki

Logs are where a failure leaves its trail: the timeouts, the retries, the request that returned a 500. Without them, TraceX can tell you what the data looks like but not always what happened. With Grafana Loki connected it can see the failure itself, how often it happened and exactly when it started, which is often the strongest evidence in a report.

Before you start

Grafana Cloud: create an access policy token with the logs:read scope, and note your Loki user id (on the Loki data source details page).
Self-hosted: use the address and basic-auth credentials of your Loki gateway.

Connect

Open Integrations → Grafana Loki and fill in:

Field	Value
Loki address	`https://logs-prod-012.grafana.net`
User	Grafana Cloud user id (optional)
Password or token	needs `logs:read`
Stream selector	optional, e.g. `{namespace="prod"}`

The stream selector limits searches to production streams. TraceX checks the credentials with a real read, then stores them encrypted.

How TraceX uses it

While investigating, and in Conversations, TraceX searches your logs in two ways:

around the time the problem started, to see what changed in the log stream;
for the identifiers in the issue: a customer, an order, a request id, an error message.

Thousands of log lines are hard to read, so TraceX groups them into patterns with counts, errors first. A burst of the same failure, such as 1,240 × PaymentGatewayTimeout since 09:02, stands out immediately, and the report links to example lines.

Example

Customers in the EU can’t upload files since 14:30.

TraceX searches for errors from the upload service after 14:00, groups them, and finds 3,812 × AccessDenied: s3:PutObject starting at 14:27, right after a deploy that changed the bucket policy. The report cites example log lines and the change.

Troubleshooting

Problem	What to check
The credential check fails	The key or token needs read access to logs; check the scope or role listed above and that it hasn’t expired.
Searches return nothing	Check the region or site, and that the log groups, index or filter you entered contain the service’s logs.
Logs are found but not the right ones	Narrow the filter (log groups, index pattern, stream selector) to production services.

Remove access

Disconnect Grafana Loki in Integrations, then revoke the key or credentials on the Grafana Loki side.