Higher Memory Requirements for SPDX Validation in v2.0.3

[augelu-tng]

With the release of the new v2.0.3 version it seems the tool trades performance for a greater memory usage.
For small files the performance improvements are great.
But for larger files the increased memory usage makes the tool less usable than before (v2.0.2).

tested file	file size	v2.0.2	v2.0.3
sbom-source.spdx.json	8 MB	2m 45s	0m 27s
sbom-build.spdx.json	150MB	1m53s	Out of memory

For example, I got these results when running
java -Xmx15G -Xms12G -jar "tools-java-${SPDX_TOOLS_VERSION}-jar-with-dependencies.jar" Verify "$spdx_document"
via the validate-spdx-documents.sh script in spdx-validation-example.zip locally on my machine. In each run the tool received a memory limit of up to 15GB. While in v2.0.2 this was enough to validate the document in under 2 minutes now with v2.0.3 the validation process is killed due to being out of memory.

[goneall]

From a quick look, this looks like an issue with the schema validation for the JSON-LD file.

This recently merged PR may help: #248

@augelu-tng - I just published a new release yesterday, if you haven't already, please try with the latest

[goneall]

I just checked and it looks like the updated schema somehow didn't make it into the release - I may need to turn a new build.
Stay tuned.

[goneall]

I just found the inconsistency in the schemas - one of the library dependencies has an outdated version of the schema, but it should not affect the validation, so the new version should help.

I'm running with the latest - the schema validation is taking a very long time - I'll wait and see if it runs out of memory.

[goneall]

I verified it is spending all of it's time and memory in the schema validation:

tools-java/src/main/java/org/spdx/tools/Verify.java

Line 184 in de4734d

List<Error> messages = schema.validate(root);

cc: @JPEWdev

It looks like it is spending time in the "anyOf" sub-schemas. I'll try to figure out which subschemas and if we can apply more "if"s to the schema logic to speed it up.

[goneall]

@augelu-tng I just realized from the title and comments you are using the latest.

As noted above, the changes are in the schema file and the performance problems are in the schema validator.

We're using ae-jsonschemafriend-core for validation unless we can find a more efficient JSON schema.

[goneall]

It looks like the newtonsoft schema validation also runs out of memory

[JPEWdev]

I'm curious why is takes more memory for "anyOf" vs. "oneOf"

[goneall]

I'm curious why is takes more memory for "anyOf" vs. "oneOf"

@JPEWdev - It looks like for this particular file the new schema is significantly slower than the schema version before we added the "if"s. Not really sure why. I wonder if there were any changes between what I tested from the schema file posted in slack vs the schema file generated from the spdx-spec repo CI. I'll do some more investigating tomorrow.

[goneall]

Did some more experimenting - there may be a problem other than the schema. If I use the schema from the 2.0.2 version with the 2.0.3 version, it still seems to take a long time. There's a new version of the schema validator library - I suspect this may be an issue, so @JPEWdev - don't worry about any analysis for now. I'll try going back to the older version of the library tomorrow.

[goneall]

Thanks @augelu-tng for reporting this so quickly after a release

I just merged in a PR with a fix - reverting back to a previous version of the library

I also submitted this issue in the upstream library: networknt/json-schema-validator#1216

I'll spin another release shortly.