RAT-578: Support core.excludesFile from global git config by rmannibucau · Pull Request #746 · apache/creadur-rat

rmannibucau · 2026-10-08T16:44:08Z

Support the git configuration option core.excludesFile (including [include] path directives) when processing global gitignore files.

Changes

New GitConfig class in apache-rat-core that parses the git global configuration files ($XDG_CONFIG_HOME/git/config and ~/.gitconfig) with git semantics:
- core.excludesFile key (case-insensitive), last value wins
- quoted values with escape sequences
- comments (# / ;) and line continuations
- ~/ expansion, relative excludesFile paths resolved against the current working directory
- [include] path recursively, relative to the including file, with cycle detection (depth limit 10 like git)
- includeIf and /etc/gitconfig are not supported (out of scope)
GitIgnoreBuilder.globalGitIgnore() now consults GitConfig first, then falls back to the default XDG location
When core.excludesFile is configured but the file is missing, no fallback occurs (matching git behavior)
RAT_NO_GIT_GLOBAL_IGNORE still disables all global gitignore processing

Tests

New GitConfigTest (16 tests): parsing, case-insensitivity, quoting, comments, precedence XDG vs ~/.gitconfig, ~/ expansion, relative paths, includes (simple/relative/tilde/recursive/cycle), subsections, missing files, default fallback
New test_global_gitignore_from_config integration test in GitIgnoreBuilderTest
Full module build passes: 1147 tests, checkstyle clean

Verification

mvn -pl apache-rat-core test → BUILD SUCCESS (checkstyle included)

GitIgnoreBuilder now reads the git configuration files ($XDG_CONFIG_HOME/git/config and ~/.gitconfig) to resolve the core.excludesFile option, including [include] path directives, before falling back to the default XDG global gitignore location. New GitConfig class parses the git config syntax (sections, quoted values, comments, line continuations) and resolves paths with git semantics (~/ expansion, relative paths against the current working directory, include paths relative to the including file with cycle detection). Also added RAT-578 changelog entry and unit + integration tests.

ottlinger

Thanks for the wonderful PR and the Jira ticket.

I really like the approach and hope that many users find this addition useful. Apart from my little remarks and the failing tests under Windows I'm okay with the PR.

@Claudenw do you see problems with this PR in the different UIs and the new architecture under development?

- Replace inner Section class with a Java record - Use org.apache.rat.utils.FileUtils in GitConfigTest instead of the local write helper doing raw Files IO - Make testMissingFileReturnsConfiguredFile portable across platforms by using a relative path resolved against the simulated CWD - Keep backslashes for unknown escape sequences in quoted values (fixes Windows style paths like C:\Users\...) - Add test covering backslash preservation in quoted values

ottlinger

LGTM

Claudenw

Not really requesting changes, requesting clarification and manual test to ensure that your analysis of relative file name resolution is correct.

Claudenw · 2026-10-12T15:58:21Z

+    /** The maximum depth of nested includes, matching the git limit. */
+    private static final int MAX_INCLUDE_DEPTH = 10;
+    /** The home directory prefix of a path. */
+    private static final String HOME_PREFIX = "~/";


What is the HOME_PREFIX on a windows machine?

it is HOME too - ̀EnvVar, guess you meant to use USERPROFILE` or whatever var git is using?

Claudenw · 2026-10-12T16:50:20Z

            return Optional.empty();
        }

+        Optional<File> fromConfig = new GitConfig().coreExcludesFile();


Can you run a test for me please.

As per your notes, the GitConfig code resolves relative files against the current working directory (CWD).

Please configure a systems such that your GitConfig code will resolve the file against the CWD.

Create a directory tree something like this.

CWD

the relative file
+---> subdirectory
+ .gitignore file
+ same relative file name different contents

When the git ignore is processed in the CWD it shoujld decend and find the .gitignore in the subdirectory. Does the relative file name get reresolved since the CWD has changed?

See https://git-scm.com/docs/gitignore command line execution.

if i didnt mess up my impl it is resolved once - we speak of global gitconfig there

Patterns read from exclude sources that are outside the working tree, such as $GIT_COMMON_DIR/info/exclude and core.excludesFile, are treated as if they are specified at the root of the working tree."

from https://git-scm.com/docs/gitignore#_pattern_format

Sorry, this got lost in my email swamp.

My understanding is that the git ignore process is re-evaluated by git on every .gitignore. I would expect the relative file to be evaluated against the directory with the .gitignore. but I don't know of that is actually how git evaluates hence the test with the command line git ignore call to verify we get the same answer.

have to admit i never used excludesFile in a nested gitignore but my understanding of the doc - didnt take time to check git sources but can next week if it helps - is that it is not supported, do you get it differently?

…_GLOBAL

Add the RAT_GIT_GLOBAL_IGNORE environment variable so RAT's global gitignore handling no longer depends on the machine an audit runs on. It accepts 'off' to disable reading the machine global git ignore, 'on' (resolved from the git configuration, the RAT-578 default), or an explicit file path that is always used as the global ignore. The legacy RAT_NO_GIT_GLOBAL_IGNORE keeps precedence. RAT's own CI build now pins RAT_NO_GIT_GLOBAL_IGNORE so runner git config cannot perturb self-checks or integration test reports. Also drop a new spotbugs finding (IM_BAD_CHECK_FOR_ODD) introduced by GitConfig.parseFile by testing the trailing-backslash count with a bitwise check, and document the new option in EnvVar, StandardCollection GIT and changes.xml. New GitIgnoreBuilderTest cases cover 'off', an explicit path (including a missing file) and 'on'/unset resolution. Verification: apache-rat-core verify passes with 1155 tests and clean checkstyle; apache-rat-tools (38) and apache-rat-tasks (113) still pass.

…FILE Rename the EnvVar.USERPROFILE constant to USER_PROFILE for naming consistency with the other constants, while keeping the actual environment variable read as USERPROFILE (the canonical Windows name) via a getValue() override on the enum constant. Convert the RAT_GIT_GLOBAL_IGNORE 'on'/'off' keywords to a tri-state value documented explicitly as not a plain boolean: 'false' disables the global git ignore, 'true' (or unset) resolves it from the git configuration, and any other value is treated as an explicit global git ignore path. This matches the existing path-mode design while aligning the boolean keyword spelling with the rest of the RAT configuration options. Update the EnvVar, GitIgnoreBuilder and StandardCollection javadoc to use well-formed closed block HTML so it builds cleanly across JDK 17/21/25. changes.xml: mark the RAT_GIT_GLOBAL_IGNORE entry as type=add with due-to=Romain Manni-Bucau and reword it for the tri-state semantics. Switch the CI workflow from RAT_NO_GIT_GLOBAL_IGNORE to the new RAT_GIT_GLOBAL_IGNORE=false for reproducible audits. GitIgnoreBuilderTest: use 'false'/'true' (with case-insensitive checks) and keep the explicit-path coverage. Full apache-rat-core build passes: 1155 tests, checkstyle and javadoc clean.

ottlinger · 2026-10-24T23:19:54Z

@Claudenw anything open here from your side? Thanks

Claudenw

I have questions about the correctness of the algorithm as well as wanting to see standard libraries used so that we don't have to support custom parsing.

Claudenw · 2026-10-26T12:04:41Z

+    /**
+     * The Windows user profile directory.
+     * <p>
+     * The environment variable that is read is {@code USERPROFILE} (no underscore), which is the
+     * canonical name used on Windows. The enum constant is spelled {@code USER_PROFILE} to stay
+     * consistent with the naming of the other constants.</p>
+     */


I don't think we want to do this. We have tried not to introduce OS specific variable and options. when you need to access the HOME directory you can use the Java system property user.home

@ottlinger I think this should be removed. Do you agree?

issue is user.home is not authoritative for OS specific folders it is just a default (and windows one had some bugs years ago) so not sure it is always great in companies setup (with remote user folders)

Claudenw · 2026-10-26T12:06:26Z

I don't like the GitConfig class in the same package as the file processors. It is small and I would like to see it in the GitIgnoreBuilder class.

Claudenw · 2026-10-26T12:10:35Z

+    /** The maximum depth of nested includes, matching the git limit. */
+    private static final int MAX_INCLUDE_DEPTH = 10;
+    /** The home directory prefix of a path. */
+    private static final String HOME_PREFIX = "~/";


The HOME_PREFIX should be build using the FSInfo object.

FSInfo fsInfo = DocumentName.FSInfo.getDefault()' String home_prefix = "~" + fsInfo.dirSeparator()

you can also use fsInfo.normalize(text) to convert the text into a path for the specified file system.

Claudenw · 2026-10-26T12:10:48Z

+    /** The home directory prefix of a path. */
+    private static final String HOME_PREFIX = "~/";
+    /** The Windows style home directory prefix of a path. */
+    private static final String HOME_BACKSLASH_PREFIX = "~\\";


not necessary with FSInfo usage.

Claudenw · 2026-10-26T12:19:55Z

+     * @param includingFile the file that contains the include directive.
+     * @return the file to include or an empty Optional if it can not be resolved.
+     */
+    private Optional<File> resolveInclude(final String value, final File includingFile) {


resolveInclude and resolveExclude seem to be the same. Perhaps a single method to do the resolution is warranted.

Claudenw · 2026-10-26T12:31:52Z

+        File path = new File(value);
+        if (!path.isAbsolute()) {
+            // git resolves relative paths against the current working directory
+            path = new File(cwd, value);


I did some extensive testing and I could not find a configuration where relative paths were resolved within the current working directory. Git always seems to resolve the file relative to the INI file that it was read from.

you can use find . | git check-ignore -v -n --stdin --no-index to evaluate a configuration. I execute this in the resources/GitIgnoreBuilderTest/RAT-578-root directory with multiple configurations.

Claudenw · 2026-10-26T12:34:32Z

+     * @param line the line to check.
+     * @return the number of trailing backslashes.
+     */
+    private int trailingBackslashes(final String line) {


This method is not necessary with the use of the INIConfiguration.

Claudenw · 2026-10-26T12:35:41Z

+     * @param value the raw value.
+     * @return the unquoted value.
+     */
+    private String unquote(final String value) {


I believe that this also goes away with use of INIConfiguraiton.

Claudenw · 2026-10-26T12:52:30Z

+        Set<File> includeChain = new HashSet<>();
+        if (globalConfig != null) {
+            // when GIT_CONFIG_GLOBAL is set, neither ~/.gitconfig nor $XDG_CONFIG_HOME/git/config are read
+            parseFile(globalConfig, configured, includeChain, 0);
+        } else {
+            parseFile(xdgConfig, configured, includeChain, 0);
+            parseFile(userConfig, configured, includeChain, 0);
+        }
+        if (!configured.isEmpty()) {
+            String value = configured.get(configured.size() - 1);
+            return resolveExcludesFile(value);
+        }


This seem incorrect.

https://git-scm.com/book/en/v2/Customizing-Git-Git-Configuration

specifies that all the configuration files should be taken together and the specification closest to the project should take prescidence.

This code does not consider the system config, nor does it consider the .git/config file for the project.

I can not find documentation that says the use of GIT_CONFIG_GLOBAL means that the ~/.gitconfig and XDG based config are not read.

I can see that XDG based config is designed to override the ~/.gitconfig so only one of them would be expected.

I did find a reference that said that GIT_CONFIG_GLOBAL may contain multiple entries. Infact setting XDG_CONFIG_HOME and placing a .gitignore in that directory and then running git var -l lists both the ${XDG_CONFIG_HOME}/.gitignore and the ~/.gitignore as entries for the GIT_CONFIG_GLOBAL value

Explicitly setting GIT_CONFIG_GLOBAL does limit the git var -l results to the single value.

Explicitly setting:

set GIT_CONFIG_GLOBAL="/path/to/dir1 /path/to/dir2"

will cause git var -l to list both files.

I tested the GIT_CONFIG_GLOBAL behavior against real git and it is a single file path, not a newline-separated list. Setting it to a value containing a newline makes git try to open that literal filename and yields no config:

$ GIT_CONFIG_GLOBAL="/tmp/dir/a.gitconfig\n/tmp/dir/b.gitconfig" git config --get core.excludesFile # exit 1, empty $ GIT_CONFIG_GLOBAL=/tmp/dir/b.gitconfig git config --get core.excludesFile # exit 0, prints value

The "some can contain multiple values separated by newlines" text in git-var(1) describes that command's output formatting, not the input variable. In the git source (config.c::git_global_config_paths) GIT_CONFIG_GLOBAL is read as a single char * pathname which replaces ~/.gitconfig and the XDG file, exactly as this code does.

On the path resolution question I also verified with git check-ignore -v -n:

a relative core.excludesFile value is resolved against the current working directory and its patterns are anchored at the root of the working tree (matches the same file both at the root and in a subdirectory);

a relative [include] path value is resolved against the directory of the including config file.

Both match this implementation. The system config (/etc/gitconfig), includeIf and the repository .git/config remain intentionally out of scope for RAT's audit-side read.

I have updated the code accordingly (GitConfig moved into GitIgnoreBuilder as a protected static nested type, home resolved from user.home, and GIT_CONFIG_GLOBAL kept as a single path).

Claudenw · 2026-10-26T13:09:49Z

+     * @param includeChain the canonical files of the current include chain to detect cycles.
+     * @param depth the current include depth.
+     */
+    private void parseFile(final File file, final List<String> excludesFileValues, final Set<File> includeChain, final int depth) {


This is parsing what is effectively an INI file. Please use Apache commons configuration2 INICoanfiguration.

Since the documentation for the configurations specifies the order in which the options are accepted (which overrides which) you can use the CombinedConfiguration to get the proper layering to return the values.

In addition, you can parse the input into multiple lines by using BufferedReader.readLine() constructed on the result from fetching configuration the value.

FTR git config files are not ini files (subsections for ex are diff)

Nest GitConfig into GitIgnoreBuilder as a protected static final inner class so the git-config reading logic stays co-located with its only consumer while remaining independently unit-testable from the same-package test class. Remove the USERPROFILE/USER_PROFILE environment variable handling and resolve the home directory from the user.home system property only. On modern JVMs (JDK 8+) user.home is derived from the OS home/profile location on every platform, so no platform specific environment variable lookup is needed. Deduplicate include and excludesFile resolution into a single resolve(value, base) helper: [include] path values are resolved against the directory of the including file and core.excludesFile values against the current working directory, matching git. Derive the home directory prefix from FSInfo.getDefault().dirSeparator() instead of a hardcoded "~/", while still accepting the alternate separator so portable git configuration files (which may use either / or \) resolve on any host. Keep GIT_CONFIG_GLOBAL as a single file path, matching git: empirical testing shows git treats the value as one pathname and a newline-separated value fails to load, contrary to git-var(1) output formatting. Full apache-rat-core build passes: 1155 tests, checkstyle and javadoc clean.

ottlinger requested a review from Claudenw October 8, 2026 17:17

ottlinger requested changes Oct 8, 2026

View reviewed changes

Comment thread apache-rat-core/src/main/java/org/apache/rat/config/exclusion/fileprocessors/GitConfig.java Outdated

Comment thread apache-rat-core/src/test/java/org/apache/rat/config/exclusion/fileprocessors/GitConfigTest.java Outdated

ottlinger approved these changes Oct 8, 2026

View reviewed changes

Merge branch 'master' into RAT-578

5293cc7

Claudenw requested changes Oct 12, 2026

View reviewed changes

rmannibucau added 3 commits October 12, 2026 23:47

RAT-578: Resolve global gitignore from core.excludesFile / GIT_CONFIG…

bcbb583

…_GLOBAL

[RAT-578] missing files due to actual gitignore

e076329

ottlinger requested changes Oct 13, 2026

View reviewed changes

Comment thread apache-rat-core/src/main/java/org/apache/rat/api/EnvVar.java Outdated

Comment thread apache-rat-core/src/main/java/org/apache/rat/api/EnvVar.java Outdated

Comment thread ...-rat-core/src/main/java/org/apache/rat/config/exclusion/fileprocessors/GitIgnoreBuilder.java Outdated

[doc] fix javadoc

5850ac8

ottlinger requested changes Oct 13, 2026

View reviewed changes

Comment thread src/changes/changes.xml Outdated

ottlinger requested review from Claudenw and ottlinger October 24, 2026 23:16

Merge branch 'master' into RAT-578

934ba5e

ottlinger approved these changes Oct 24, 2026

View reviewed changes

Claudenw requested changes Oct 26, 2026

View reviewed changes

Conversation

rmannibucau commented Oct 8, 2026

Changes

Tests

Verification

Uh oh!

ottlinger left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ottlinger left a comment

Choose a reason for hiding this comment

Uh oh!

Claudenw left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rmannibucau Oct 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ottlinger commented Oct 24, 2026

Uh oh!

Claudenw left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rmannibucau Oct 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rmannibucau Oct 12, 2026 •

edited

Loading

rmannibucau Oct 26, 2026 •

edited

Loading