Skip to content

[xtable-utilities] Remove duplicate AWS SDK v2 modules from shaded jar - #923

Open
abdul-traya wants to merge 2 commits into
apache:mainfrom
abdul-traya:xtable-sync-shade-fix
Open

[xtable-utilities] Remove duplicate AWS SDK v2 modules from shaded jar#923
abdul-traya wants to merge 2 commits into
apache:mainfrom
abdul-traya:xtable-sync-shade-fix

Conversation

@abdul-traya

Copy link
Copy Markdown

software.amazon.awssdk:bundle already contains internally-consistent copies of regions/sdk-core/utils. Including them separately in the shade artifactSet caused the shade plugin to merge two different builds of these classes, so the bundle's STS classes (needed for IRSA/web-identity credentials on EKS) ended up paired with an IoUtils class missing a method they call -- NoSuchMethodError at JVM shutdown when closing the STS web-identity credentials provider.

Important Read

  • Please ensure the GitHub issue is mentioned at the beginning of the PR

What is the purpose of the pull request

(For example: This pull request implements the sync for delta format.)

Brief change log

(for example:)

  • Fixed JSON parsing error when persisting state
  • Added unit tests for schema evolution

Verify this pull request

(Please pick either of the following options)

This pull request is a trivial rework / code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end.
  • Added TestConversionController to verify the change.
  • Manually verified the change by running a job locally.

software.amazon.awssdk:bundle already contains internally-consistent
copies of regions/sdk-core/utils. Including them separately in the
shade artifactSet caused the shade plugin to merge two different
builds of these classes, so the bundle's STS classes (needed for
IRSA/web-identity credentials on EKS) ended up paired with an IoUtils
class missing a method they call -- NoSuchMethodError at JVM shutdown
when closing the STS web-identity credentials provider.
…bundle

The first pass only removed regions/sdk-core/utils, which fixed one
NoSuchMethodError but exposed another (auth's LazyAwsCredentialsProvider
paired with a mismatched IoUtils again) since auth/aws-core/glue/apache-client
/netty-nio-client/etc were still separately included alongside bundle.
bundle already contains internally-consistent copies of every individual
software.amazon.awssdk:* module, so keep only it -- verified via javap that
LazyAwsCredentialsProvider.close()'s IoUtils.closeIfCloseable call now
resolves against a matching overload in the rebuilt jar.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant