Skip to content

Unit test comparator silently ignores TIMESTAMP(Microsecond) and ARRAY/STRUCT columns, so wrong expect values always pass #16649

Description

@dataders

Current Behavior

A wrong value in a TIMESTAMP (or ARRAY/STRUCT) column of a unit test's expect block is never flagged as a mismatch. The test passes when it should fail, hiding real regressions.

The unit test comparator only renders a subset of Arrow types to strings for comparison (ints, floats, strings, booleans, decimals, Timestamp(Second), Timestamp(Nanosecond), Date32). Any other type, notably Timestamp(Microsecond) (BigQuery's default precision), Timestamp(Millisecond), List/LargeList/FixedSizeList, and Struct, falls through to a catch-all that renders the literal string [unsupported]. Both the actual and expected cells render identically, so the equality check always reports a match for that column.

Expected Behavior

Every Arrow type a model can produce (at minimum all timestamp precisions an adapter can emit, plus List/Struct/nested variants) should be faithfully stringified for comparison. Failing that, the comparator should raise a hard error rather than silently reporting a match for a type it can't render.

Steps To Reproduce

Reproduced on dbt Fusion 2.0.6 with DuckDB:

-- models/dim_teams.sql
select 1 as team_id, timestamp '2026-01-01 00:00:00' as created_at
# models/_unit_tests.yml
unit_tests:
  - name: probe_ts_wrong
    model: dim_teams
    given: []
    expect:
      rows:
        - team_id: 1
          created_at: '2030-01-01 00:00:00'   # wrong on purpose
$ dbt test
Passed unit_test main_dbt_test__audit.probe_ts_wrong

This should fail, since created_at differs by four years. When a different column in the same test does mismatch, the diff table shows the masking value:

+---------+---------------+----------------------+
| team_id | created_at    | customer_tos_seconds |
+---------+---------------+----------------------+
| 1       | [unsupported] | 301.0 -> 300.0       |
+---------+---------------+----------------------+

The same behavior was observed on BigQuery for TIMESTAMP columns and for ARRAY columns (including arrays of structs). Array reordering is not detected either. The generated comparison SQL is correct (running it directly in BigQuery shows the real mismatch). The difference is lost in Fusion's own comparison step after the query.

Environment

  • dbt: 2.0.6
  • Adapters: BigQuery (original report), DuckDB (minimal repro)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    adapter:bigqueryConcerns the BigQuery adapter / SQL dialect.engine:v2Concerns the dbt v2 engine.status:has-reproA reliable reproduction has been provided.type:bugA defect: dbt behaves incorrectly versus expected/reference behavior.unit_tests

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions