# Bug OP-20622 - OpenTelemetry logs "invalid span attribute value type Hash" for every S3 request made through fog

Project: OpenProject
Status: tested
URL: https://community.openproject.org/work_packages/OP-20622

## Open points

_No work packages_

## Details

| Attribute | Value |
| --- | --- |
| Date |  |
| Priority | Medium |
| Module | Modules and project features / Attachments |
| Target versions | 17.9.5 |
| Observed in versions | 17.9.3 |
| Sprint |  |
| Backlog bucket |  |
| Bug found in version (deprecated) |  |
| Found in mobile app version |  |
| Follow-up feedback e-mail |  |
| Requires doc change | No |
| Accessibility | No |
| Pull preview |  |
| List |  |
| Labels |  |

## People

| Attribute | Value |
| --- | --- |
| Assignee |  |
| Accountable |  |
| Product manager |  |
| Developers |  |
| Designer |  |
| Ordered by |  |
| Signed off by |  |
| Interest by |  |

## Planning

| Attribute | Value |
| --- | --- |
| Relevance |  |

## Costs

| Attribute | Value |
| --- | --- |
| Votes |  |
| Work |  |
| Labor costs |  |

## Description

**Summary**

With OpenTelemetry enabled, the Excon instrumentation logs an error for every S3 request made through fog-aws (attachment storage). The span is still recorded, but without its `url.query` attribute.

**Steps to reproduce**

1.  Enable OpenTelemetry (`opentelemetry_enabled` setting).
2.  Configure fog/S3 as attachment storage.
3.  Open or download an attachment, or do anything else that makes fog issue a request to S3.
4.  Check the application logs.

**Actual behavior**

Every fog request logs an error, and the resulting span has no `url.query` attribute. The request itself succeeds.

**Expected behavior**

No error is logged. The span carries `url.query` as a string when the request has a query.

**Environment**

  - OpenProject 17.9 (seen on production, current branch `release/17.9`)
  - `opentelemetry-instrumentation-excon` 0.29.1 (via `opentelemetry-instrumentation-all` 0.95.0)
  - `fog-aws` 3.33.x, `excon` 1.5.0

**Logs and errors**

    openproject E, [2026-10-26T13:38:01.328758 #8] ERROR -- : OpenTelemetry error: invalid span attribute value type Hash for key 'url.query' on span 'GET'

**Additional context**

Cause: the Excon tracer middleware copies `datum[:query]` into the span attributes unchanged (`attributes['url.query'] = datum[:query] if datum[:query]`). Excon accepts the query as a String or a Hash. fog-aws always passes a
Hash, even an empty `{}`, so every request is affected. The OTel SDK rejects Hash attribute valuegs the error. This is unrelated to AppSignal. As of 2026-10-26 it is still unfixed upstream onopentelemetry-ruby-contrib `main`.

Fix: <https://github.com/opf/openproject/pull/26360> prepends a patch onto the Excon `TracerMiddleware` (stable and dup semconv variants). When the query is a Hash, the patch serializes it with `Excon::Utils.query_string`, the same function Excon uses to build the request. The request on the wire stays byte-identical, so fcted.

Frequency: always, on every fog request while OpenTelemetry is enabled.
