Skip to content

Add Athena data-diff table summaries - #2335

Open
terzioglub wants to merge 1 commit into
mainfrom
fix-athena-data-diff-summary
Open

Add Athena data-diff table summaries#2335
terzioglub wants to merge 1 commit into
mainfrom
fix-athena-data-diff-summary

Conversation

@terzioglub

Copy link
Copy Markdown
Collaborator

Summary

  • Implement GetTableSummary for Athena so bruin data-diff can compare Athena tables.
  • Add an Athena type mapper for numeric, string, boolean, datetime, binary, and JSON types.
  • Collect row counts and per-column statistics for full comparisons, while preserving schema-only behavior.
  • Add sqlmock coverage for schema-only, full stats, invalid table names, and Athena type mapping.

Fixes #1224.

Tests

  • CGO_ENABLED=0 go test ./pkg/athena ./pkg/diff

Note: go test -tags=no_duckdb_arrow ./cmd -run 'TestDataDiff|TestCompare|Test.*Diff' could not complete in this runner because the container ran out of disk while compiling unrelated cloud provider dependencies.

@greptile-apps

greptile-apps Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor
Prompt To Fix All With AI
Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
pkg/athena/db.go:523-525
**Catalog Names Are Rejected**

When `bruin data-diff` receives an Athena table as `catalog.schema.table`, the CLI passes that identifier through unchanged, but this branch rejects it before any Athena query runs. Fully qualified Athena tables therefore fail with the format error instead of being compared.

Reviews (1): Last reviewed commit: "Add Athena table summaries for data diff" | Re-trigger Greptile

Comment thread pkg/athena/db.go
Comment on lines +523 to +525
tableNameOnly = tableComponents[1]
default:
return "", "", "", fmt.Errorf("table name must be in table or schema.table format, '%s' given", tableName)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Catalog Names Are Rejected

When bruin data-diff receives an Athena table as catalog.schema.table, the CLI passes that identifier through unchanged, but this branch rejects it before any Athena query runs. Fully qualified Athena tables therefore fail with the format error instead of being compared.

Rule Used: What: Comments should be concise and targeted to s... (source)

Prompt To Fix With AI
This is a comment left during a code review.
Path: pkg/athena/db.go
Line: 523-525

Comment:
**Catalog Names Are Rejected**

When `bruin data-diff` receives an Athena table as `catalog.schema.table`, the CLI passes that identifier through unchanged, but this branch rejects it before any Athena query runs. Fully qualified Athena tables therefore fail with the format error instead of being compared.

**Rule Used:** What: Comments should be concise and targeted to s... ([source](https://app.greptile.com/bruin/-/custom-context?memory=6adfbd2b-1b23-4560-9e00-6254e7cb7c70))

How can I resolve this? If you propose a fix, please make it concise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add data-diff Support for Athena

1 participant