Plugin inappropriately assumes Apache 2 license for mysql:mysql-connector-java

[zarebski-m]

I think the problem is caused by the fact that LICENSE file includes texts of licenses for 3rd party components, see e.g.:

[...]
Ant-Contrib

   The following software may be included in this product:
Ant-Contrib
Copyright (c) 2001-2003 Ant-Contrib project. All rights reserved.
Licensed under the Apache 1.1 License Agreement, a copy of which is reproduced b
elow.

The Apache Software License, Version 1.1

Copyright (c) 2001-2003 Ant-Contrib project.  All rights reserved.
[...]

Those should not be considered as licenses of mysql-connector-java, but apparently they are:

{
    "moduleName": "mysql:mysql-connector-java",
    "moduleVersion": "8.0.22",
    "moduleUrls": [
        "http://dev.mysql.com/doc/connector-j/en/"
    ],
    "moduleLicenses": [
        {
            "moduleLicense": "Apache License, Version 2.0",
            "moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
        },
        {
            "moduleLicense": "The GNU General Public License, v2 with FOSS exception",
            "moduleLicenseUrl": ""
        }
    ]
}

To be honest I have no idea how this can be fixed, but it definitely is not correct.

[firat-sivrikaya]

I have the same issue with itextpdf, version 5.5.13.2.

"Apache License, Version 2.0" is not a part of the licenses of this dependency, but it still appears in the list:

...
{
            "moduleName": "com.itextpdf:itextpdf",
            "moduleVersion": "5.5.13.2",
            "moduleUrls": [
                "http://itextpdf.com"
            ],
            "moduleLicenses": [
                {
                    "moduleLicense": "Apache License, Version 2.0",
                    "moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
                },
                {
                    "moduleLicense": "GNU Affero General Public License v3",
                    "moduleLicenseUrl": "http://www.fsf.org/licensing/licenses/agpl-3.0.html"
                }
            ]
        },
...

Per the allowed licenses definition in my project, checkLicense task should fail because of this dependency's AGPL license, but unfortunately it does not fail due to the presence of Apache License, Version 2.0 in the list.

[psibre]

FWIW, I was surprised to notice this issue; I'm currently trying to convince this plugin to fail the checkLicense task by disallowing the GPL license, specifically for the mysql:mysql-connector-java dependency in my project.

However, my $0.02 here have a different angle: I'm processing the generated BOM using the cyclonedx CLI tool to generate a CSV file...

$ cyclonedx convert --input-file build/reports/bom.xml --output-file bom.csv

...and it turns out that apparently the GPL license "wins" by overwriting the wrong (Apache) one as the CSV is being generated (or maybe the last one is chosen -- I didn't look at the CLI tool source code).

Excerpt:

Group	Name	Version	Scope	LicenseExpressions	LicenseNames
mysql	mysql-connector-java	8.0.29			The GNU General Public License_ v2 with FOSS exception

Go figure... =)

[psibre]

I'm currently trying to convince this plugin to fail the checkLicense task by disallowing the GPL license, specifically for the mysql:mysql-connector-java dependency in my project.

As it turns out, this issue did cause me some more grief. If I disallow the GPL in my license whitelist JSON file, the MySQL Connector dependency fails to trigger the expected license policy violation, because it also has the other license(s)...

With the plugin v2.0, I see this (as reported above):

        {
            "moduleName": "mysql:mysql-connector-java",
            "moduleVersion": "8.0.29",
            "moduleUrls": [
                "http://dev.mysql.com/doc/connector-j/en/"
            ],
            "moduleLicenses": [
                {
                    "moduleLicense": "Apache License, Version 2.0",
                    "moduleLicenseUrl": "https://www.apache.org/licenses/LICENSE-2.0"
                },
                {
                    "moduleLicense": "The GNU General Public License, v2 with FOSS exception",
                    "moduleLicenseUrl": ""
                }
            ]
        },

This means I need to disallow the GPL as well as the Apache License in my whitelist to trigger the violation with the MySQL connector. Obviously I don't want to disallow the Apache License, since that would flag many other compliant dependencies.

The plot thickens with the current plugin version (v2.1); now I get:

        {
            "moduleName": "mysql:mysql-connector-java",
            "moduleVersion": "8.0.29",
            "moduleUrls": [
                "http://dev.mysql.com/doc/connector-j/en/"
            ],
            "moduleLicenses": [
                {
                    "moduleLicense": "Eclipse Public License - v 2.0",
                    "moduleLicenseUrl": "https://www.eclipse.org/org/documents/epl-2.0/EPL-2.0.txt"
                },
                {
                    "moduleLicense": "The GNU General Public License, v2 with FOSS exception",
                    "moduleLicenseUrl": ""
                }
            ]
        },

Now I need to adapt my whitelist to block GPL and EPL if I want to cause it to flag the MySQL Connector. (Turns out this is less messy in my particular dependency tree, but still hacky.)

So apparently the processing behavior with multiple licenses has changed between v2.0 and v2.1 of this plugin. (But without tags or release notes, this is quite tedious to investigate; I've opened #250 accordingly.)

[nsoft]

This might be duplicate of #264, (fixed by #265)

[nsoft]

This might be duplicate of #264, (fixed by #265)

Or maybe not looks like mysql-connector-java doesn't have a parent pom. ... though it's possible that "child poms" is actually even broader than I expected in my fix. (

Gradle-License-Report/src/main/groovy/com/github/jk1/license/reader/PomReader.groovy

Line 185 in 027d572

private PomData createPomData(GPathResult rootPom, List<GPathResult> childPoms) {

)

[d2callpod]

FWIW, I was surprised to notice this issue; I'm currently trying to convince this plugin to fail the checkLicense task by disallowing the GPL license, specifically for the mysql:mysql-connector-java dependency in my project.

However, my $0.02 here have a different angle: I'm processing the generated BOM using the cyclonedx CLI tool to generate a CSV file...

$ cyclonedx convert --input-file build/reports/bom.xml --output-file bom.csv

...and it turns out that apparently the GPL license "wins" by overwriting the wrong (Apache) one as the CSV is being generated (or maybe the last one is chosen -- I didn't look at the CLI tool source code).

Excerpt:
Group Name Version Scope LicenseExpressions LicenseNames
mysql mysql-connector-java 8.0.29 The GNU General Public License_ v2 with FOSS exception

Go figure... =)

Which renderer are you using that cyclonedx can convert?

[psibre]

I'm quite happy using the CycloneDX Gradle plugin to generate our SBOMs.

[itchyny]

We're facing the same issue. I think the plugin should respect the manifest license if available. Following patch fixed the issue.

diff --git a/src/main/groovy/com/github/jk1/license/render/LicenseDataCollector.groovy b/src/main/groovy/com/github/jk1/license/render/LicenseDataCollector.groovy
index 966d3c3..774ba32 100644
--- a/src/main/groovy/com/github/jk1/license/render/LicenseDataCollector.groovy
+++ b/src/main/groovy/com/github/jk1/license/render/LicenseDataCollector.groovy
@@ -61,10 +61,12 @@ class LicenseDataCollector {
             }
         }
 
-        data.licenseFiles*.fileDetails.flatten().each { LicenseFileDetails details ->
-            if (!info.licenses.find { it.name == details.license }) {
-                if (details.license || details.licenseUrl) {
-                    info.licenses << new License(name: details.license, url: details.licenseUrl)
+        if (info.licenses.isEmpty()) {
+            data.licenseFiles*.fileDetails.flatten().each { LicenseFileDetails details ->
+                if (!info.licenses.find { it.name == details.license }) {
+                    if (details.license || details.licenseUrl) {
+                        info.licenses << new License(name: details.license, url: details.licenseUrl)
+                    }
                 }
             }
         }

[nsoft]

The original report is mentioning https://github.com/mysql/mysql-connector-j/blob/release/9.x/LICENSE which is an unfortunate way of listing third party dependencies, but it is important to realize that one should detect both Eclipse and Apache based on https://github.com/jetty/jetty.project/blob/jetty-10.0.22/LICENSE because the intent of that file is to offer the user a choice of licenses¹.

So I don't think there will be a code solution for this unless someone employs a (100% accurate) NLP processing to understand the text separating project licenses and 3rd party licenses so that the tool can stop processing at that point. One could add a "license stop regex" but that's going to be super error prone too.

Ultimately I think one has to view this tool as a great way to get a first pass at documenting license dependencies, and to detect changes in dependency licensing, but one simply can't assume it gets everything 100% correct (well one can, but I guess that's a risk I'm unwilling to take). In JesterJ I use this tool to drive a custom gradle process that enforces required elements of a directory structure I use to:

• document the license I have chosen to use/distribute the dependency under (where choice is available) and
• use to generate my own NOTICE file. That structure is filled out and hand validated for each license.

The result is that the build complains any time I add/change something and forces me to update that directory. I also have code that fails for orphaned license information, automatically applies my license priority list, and ensures that every link referenced in that directory structure is not a dead link (it at least returns 200 OK). Also note that I have carefully crafted both allowed-licenses.json and license-normalizer-bundle.json to emphasize URL and License name and only apply limited use of license file patterns since license files are the biggest and dirtiest source (a last resort).

You are of course free to use/mimic any of that under the JesterJ's ASL 2.0 license. Please bear in mind I am not a lawyer and this is only a statement what I have done, not legal advice that what I have done is good/safe legally. Judge that for yourself of course.

I truly wish that this stuff could be fully automated because managing 250+ license dependencies is the opposite of fun, but given the difficulty (all) people have in following any process, let alone a legal process I don't hold out hope. Someone somewhere will do something dodgy and if it's useful, someone else will use it and mask that... one simply needs to go look at each thing. This tool helps a lot with managing the list of things to look at.

Finally I think that we should file issues with the projects that mix 3rd party and 1st party licenses in a single file suggesting that they adopt a THIRD_PARTY_LICENSES or NOTICE file to separate the two licenses

Footnotes

1. From the jetty project home page: "These components are open source and are freely available for commercial use and distribution under both the EPL2 and Apache2 licenses." ↩

[itchyny]

The package mysql-connector-j declares their license as GPL-2.0 in their POM file. I understand the difficulty of extracting the license from the LICENSE file (in possibly unfortunate format). But why do you think we have to merge the untrusted parsing results of LICENSE file into the license list extracted from the manifest file? I'm suggesting if we find the license declaration in the package's manifest file, we don't merge the (possibly untrusted) licenses from the LICENSE file.

[nsoft]

What you want seems to be possible with carefully crafted license-normalizer-bundles? I worry about a hard coded choice. There will inevitably be a case where some project fails to specify all available licenses at the manifest level, but does express the desired license in the license file. The general shape of solution that seems best to me is some sort of configurable preference order, and/or exceptions/overrides for individual dependencies that are "known problems"

We have rules that usually work and then problem projects that do weird stuff that we need to handle too.

[chadlwilson]

There is always going to be a need to fixup broken ones and do 'best guess' at times, but it still seems better to prefer manifest to me, given LICENSE is even less standardized.

Projects that redistribute often need to include the LICENSE file contents so they have additional considerations which don't necessarily need to trouble normal users looking to validate usage-without-redistribution. So for me it seems better for 'rules' to be based off manifest so we can nudge projects towards SPDX identifiers and such, which IIRC can also express license choices and add-ons (such as the very important FOSS exception with the MySQL connector, critical for those redistributing)

[nsoft]

"Gather everything by default, and exclude special cases if so configured" seems better (to me at least) than "Make implicit choices that silently ignore available information and leave it to the user to figure out if some other license was available if they some how expect or suspect that to be the case"

Or reformatted as an over simplified quip: Start with a firehose & filter as needed.

[itchyny]

It's clear to a human reader that the mysql-connector-j's LICENSE file also lists licenses for third-party packages, so I can't blame the package owner. I don't think anyone would write a LICENSE file with the intention of it being read by a program. Programs should respect package metadata, and string matching implementation without a deep understanding of the LICENSE file contents can lead to serious issues like this. As has been pointed out many times in this issue, the checkLicense command is problematic when it terminates successfully allowing a package with a license that is disallowed by the configuration. Users cannot be aware of this issue and add a package violating their allowed license list someday, that's a big problem. Even if a package owner forgets to include one of the dual licenses in the metadata, the command simply fails due to the license user can choose, and user can notice and report the package owner to update the metadata. What I want is for the checkLicense command to work properly and adopt this plugin to my work project.