Skip to content

Conversation

@zhixuanjia
Copy link
Contributor

@zhixuanjia zhixuanjia commented Oct 1, 2025

Issue Summary
Currently in the request body of createGenericEntities API , the entity name is passed in as camel case like FooBar but the entity type stored in DB is lowercase such as urn:li:foobar:123. This results in empty list in the response.

Fix
Perform case-insensitive matching on the result map key.

@github-actions github-actions bot added devops PR or Issue related to DataHub backend & deployment community-contribution PR or Issue raised by member(s) of DataHub Community labels Oct 1, 2025
@datahub-cyborg datahub-cyborg bot added the needs-review Label for PRs that need review from a maintainer. label Oct 1, 2025
@codecov
Copy link

codecov bot commented Oct 1, 2025

Bundle Report

Bundle size has no change ✅

@codecov
Copy link

codecov bot commented Oct 1, 2025

Codecov Report

❌ Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...roject/openapi/v3/controller/EntityController.java 0.00% 2 Missing ⚠️

❌ Your patch check has failed because the patch coverage (0.00%) is below the target coverage (75.00%). You can increase the patch coverage or adjust the target coverage.

📢 Thoughts on this report? Let us know!

@datahub-cyborg datahub-cyborg bot added merge-pending-ci A PR that has passed review and should be merged once CI is green. and removed needs-review Label for PRs that need review from a maintainer. labels Oct 1, 2025
for (String entityName : entityTypes) {

// Case-insensitive matching.
List<IngestResult> entityResults = resultsByEntityType.getOrDefault(entityName, List.of());
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zhixuanjia we can always check for lower case here ?
cc: @david-leifker

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that something about this feels off. Above we're setting the entries in the map to be the direct entityType from the urn (which should not be lowercase, it would be cased in the way the entity is defined, usually camelCase). Entity types are technically case sensitive (practically not, because of some auto lowercasing in entity registry, but not by the urn itself). I would expect consistency between how the keys are in the map are being set up and this code here. Either we should be consistently lowercasing them in the map above or it shouldn't need this lowercase call here (calling both of the getOrDefaults feels even weirder).

Would like to see tests exemplifying this case as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the urn (which should not be lowercase, it would be cased in the way the entity is defined, usually camelCase).

@RyanHolstien to provide some context here, we are not using the standard Datahub entity types like dataset, dataJob etc. Our models are loaded as plugins and they are all lowercased. And we use Open API spec to create Rest clients to interact with this endpoint. In the Open API spec, the entity names are from entity registry and they are in camelCases but entity types are all lowercased.

There are a few options we could fix this:

  1. We could change all our entity types to camelCases to solve this issue but it's just too much work on our side. It will requires data migration.

  2. We could also change the open api spec generator to use lowercases. But that will change the Rest client / interface which our other services have dependencies on.

  3. Case-insensitive matching so that "FooBar" and "foobar" are considered the same entity type. This is the option used in this PR.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, wouldn't it make more sense for your entity types in the registry to be the same casing as it is for your actual entities? I.e. just change your plugin entity-registry.yml to match what the entities actually have. This kind of feels like putting customization specific code into the main OSS trunk the way you've described it 🤔

auto-merge was automatically disabled October 6, 2025 18:15

Head branch was pushed to by a user without write access

@zhixuanjia zhixuanjia force-pushed the fix-entity-controller branch from bc04288 to 0e93f1d Compare October 6, 2025 18:15
for (String entityName : entityTypes) {

// Case-insensitive matching.
List<IngestResult> entityResults = resultsByEntityType.getOrDefault(entityName, List.of());
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
List<IngestResult> entityResults = resultsByEntityType.getOrDefault(entityName, List.of());

Then remove this second check

@zhixuanjia zhixuanjia force-pushed the fix-entity-controller branch from 5cd26d4 to 6ba1bfb Compare October 7, 2025 03:12
@RyanHolstien RyanHolstien merged commit d87f46d into datahub-project:master Oct 7, 2025
29 checks passed
@yoonhyejin yoonhyejin added hacktoberfest-accepted Acceptance for hacktoberfest https://hacktoberfest.com/participation/ hacktoberfest Targeted Issues & PRs for Hacktoberfest labels Oct 9, 2025
alplatonov pushed a commit to alplatonov/datahub that referenced this pull request Oct 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-contribution PR or Issue raised by member(s) of DataHub Community devops PR or Issue related to DataHub backend & deployment hacktoberfest Targeted Issues & PRs for Hacktoberfest hacktoberfest-accepted Acceptance for hacktoberfest https://hacktoberfest.com/participation/ merge-pending-ci A PR that has passed review and should be merged once CI is green.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants