Skip to content

Added create_native_delta_table to dynamodb_target, for glue crawler module - #38

Open
pkapoor-blackline wants to merge 4 commits into
cloudposse:mainfrom
pkapoor-blackline:glue-crawler-delta-native-table-support
Open

Added create_native_delta_table to dynamodb_target, for glue crawler module#38
pkapoor-blackline wants to merge 4 commits into
cloudposse:mainfrom
pkapoor-blackline:glue-crawler-delta-native-table-support

Conversation

@pkapoor-blackline

Copy link
Copy Markdown

what

  • Added support for create_native_delta_table in delta_target block inside main.tf.
  • Updated the delta_target variable definition in variables.tf to include create_native_delta_table as a bool field.

why

Changes

  • Code updated to support new argument in main.tf
  • Variable schema extended in variables.tf

references

Comment thread modules/glue-crawler/variables.tf Outdated
@oycyc

Copy link
Copy Markdown

/terratest

@oycyc yangci (oycyc) left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the addition pkapoor-blackline

@oycyc yangci (oycyc) added enhancement New feature or request minor New features that do not break anything and removed triage Needs triage labels Aug 7, 2025
@pkapoor-blackline

Copy link
Copy Markdown
Author

yangci (@oycyc), Hi, thank you for the review and +1

The terra-test seems to be failing because of some S3 issue, can you help with that in order for this change be merged to repo.

TestExamplesComplete 2025-08-07T02:26:22Z retry.go:99: Returning due to fatal error: FatalError{Underlying: error while running command: exit status 1; ╷
│ Error: creating S3 Bucket (eg-ue2-test-glue-a2c1wp-destination) ACL: operation error S3: PutBucketAcl, https response error StatusCode: 400, RequestID: 702K8K1WTCNK6ETV, HostID: pDI8jq9TUAiJphlAxTAA5Qsx71t8DyJFOcYxQCbV/6fT/4NyDVjbY6pa6fPnxZljYoHvXoUcRTA=, api error AccessControlListNotSupported: The bucket does not allow ACLs
│ 
│   with module.s3_bucket_destination.aws_s3_bucket_acl.default[0],
│   on .terraform/modules/s3_bucket_destination/main.tf line 143, in resource "aws_s3_bucket_acl" "default":
│  143: resource "aws_s3_bucket_acl" "default" {
│ 
╵
╷
│ Error: creating S3 Bucket (eg-ue2-test-glue-a2c1wp-source) ACL: operation error S3: PutBucketAcl, https response error StatusCode: 400, RequestID: 702QEGYBCEXNDC6K, HostID: F+5VDGTV3+sEK4G7D/qkd7W4GhNXEmN6w+g2t1rqnxLyp67lUHc9/l7Tq3EWu8tI+SeszTLdDNc=, api error AccessControlListNotSupported: The bucket does not allow ACLs
│ 
│   with module.s3_bucket_source.aws_s3_bucket_acl.default[0],
│   on .terraform/modules/s3_bucket_source/main.tf line 143, in resource "aws_s3_bucket_acl" "default":
│  143: resource "aws_s3_bucket_acl" "default" {
│ 
╵}TestExamplesComplete 2025-08-07T02:26:22Z retry.go:99: Returning due to fatal error: FatalError{Underlying: error while running command: exit status 1; ╷
│ Error: creating S3 Bucket (eg-ue2-test-glue-a2c1wp-destination) ACL: operation error S3: PutBucketAcl, https response error StatusCode: 400, RequestID: 702K8K1WTCNK6ETV, HostID: pDI8jq9TUAiJphlAxTAA5Qsx71t8DyJFOcYxQCbV/6fT/4NyDVjbY6pa6fPnxZljYoHvXoUcRTA=, api error AccessControlListNotSupported: The bucket does not allow ACLs
│ 
│   with module.s3_bucket_destination.aws_s3_bucket_acl.default[0],
│   on .terraform/modules/s3_bucket_destination/main.tf line 143, in resource "aws_s3_bucket_acl" "default":
│  143: resource "aws_s3_bucket_acl" "default" {
│ 
╵
╷
│ Error: creating S3 Bucket (eg-ue2-test-glue-a2c1wp-source) ACL: operation error S3: PutBucketAcl, https response error StatusCode: 400, RequestID: 702QEGYBCEXNDC6K, HostID: F+5VDGTV3+sEK4G7D/qkd7W4GhNXEmN6w+g2t1rqnxLyp67lUHc9/l7Tq3EWu8tI+SeszTLdDNc=, api error AccessControlListNotSupported: The bucket does not allow ACLs
│ 
│   with module.s3_bucket_source.aws_s3_bucket_acl.default[0],
│   on .terraform/modules/s3_bucket_source/main.tf line 143, in resource "aws_s3_bucket_acl" "default":
│  143: resource "aws_s3_bucket_acl" "default" {
│ 
╵}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request minor New features that do not break anything

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants