Separation of completion directories

[akinomyoga]

We currently instruct upstream packages to place completion files in the directory $prefix/share/bash-completion/completions/, while we also provide completions in the same directory.

When an upstream project decides to host its completion setting in the upstream project, we have renamed our completion setting from completions/<cmd> to completions/_<cmd> to mark it as a fallback. However, this only works when the bash-completion package of the system has newer versions. Otherwise, it would end up with a situation where two packages (of bash-completion and the upstream project) in distribution would want to install files in a conflicting path.

Technically, the package repository maintainer could choose the proper combination of the versions of bash-completion and the upstream project, but bash-completion may have another restriction. For example, macOS bash-completion is a very old one that needs to work with Bash 3.2. Also, this requirement on the version of the bash-completion packages seems inconsistent with the arguments for the new API. In the discussion of the new API introduced in 2.12 (such as in #1328), we expect a new version of bash-completion to be available everywhere only after about a decade.

I'm going to start threads for each subtopic.

[akinomyoga]

Separation of external/bundlded completion directories

I think it would be cleaner and easier to separate the directories for the external completions and the completions that we provide by default.

Since the external completions are already installed to $prefix/share/bash-completion/completions/, we may move our bundlded completions to another directory, such as $prefix/share/bash-completion/completions/default/ and $prefix/share/bash-completion/completions-dist/.

[yedayak]

I think this is a good idea, will be better than prefixing everything with _, although it might hurt discoverability of where completions are coming from a bit. Maybe adding a README file that points to the default completions will be nice.
Do you have a proposal for what to do with the already deprecated completions starting with _? We can remove the prefix when moving them, but maybe a distinction between 3rd party completion fallbacks (like for cobra) and stuff that are implemented here should stay? This does introduce another check for file, but it seems we already do a lot of those.
I guess #1329 (comment) is also related

[akinomyoga]

Directory for "eagerly loaded completions"

There are some completion settings that people want to load at the shell startup rather than when completion of the corresponding command is attempted. Such a setting includes the one that wants to replace/modify the behavior of the default completion complete -D.

Our README calls such a setting "eagerly loaded completions" and recommends the user to put it in /etc/bash_completion.d. However, at the same time, /etc/bash_completion.d is kind of a deprecated mechanism for bash-completion v1 (which covered Bash <= 4.0 lacking complete -D). I raised a discussion about this subtlety in #1220 (comment). A partial answer has been given at #1220 (comment), but I think this subtlety itself hasn't been discussed.

I personally think the directory for the "eagerly loaded completions" and "v1 completions (in /etc/bash_completion.d)" can also be separated to make the intention clear to the upstream package maintainers. While the latter with API v1 (as defined in #1328) is deprecated, the former is still useful with API v2/v3 loading. The directory for eagerly loaded completions can be, for example, completions/startup/. This was essentially proposed by @kislyuk at #1220 (comment) originally. Here, I quote my reply at that time, which summarized the situation:

and if not, I'm looking for opinions from @scop on whether this functionality could be added.

@scop I think the current situation of BASH_COMPLETION_COMPAT_DIR is unclear. Although its name looks like it will be deprecated and should not be used, we added 000_bash_completion_compat.bash which I guess we need to maintain perpetually. The description in README is also not so clear about whether its use is encouraged or discouraged: "Q. How can I override a completion shipped by bash-completion?" seems to suggest using compatdir while "Q. I author/maintain package X and would like to maintain ..." (which is a question for the completion for a specific command) seems to deprecate compatdir. I've been thinking we would continue to offer compatdir for eagerly loaded settings while deprecating it for the completion settings for specific commands, but it is not so obvious. Maybe another option would be, as suggested, to prepare another directory where the eagerly loaded files can be put, though it is functionally equivalent to the current compatdir.

[akinomyoga]

Recommending filename extension .bash

This may be a separate discussion, but another thing is that the filename extension of a completion setting. Currently, we accept both completions/<cmd> and completions/<cmd>.bash, but this is ambiguous when there are two commands task and task.bash.

From a packaging point of view, this may also introduce conflicts between packages when the package for task wants to install its completion to <cmd>/task.bash (with the filename extension) while the package for task.bash wants to install its completion to <cmd>/task.bash (without the filename extension). The executable names shared by people do not usually have the filename extension .bash, so this wouldn't become a big problem, but still, personal scripts installed to a home directory (~/.local) may have the filename extension, where a conflict may happen on the filename path ~/.local/share/bash-completion/completions/task.bash.

Even without such a conflict, a problem arises in loading the completions. When a user wants to complete the arguments to task.bash [TAB], the completion loader first attempts reading task.bash, but it might be a completion setting for task. Or, when a user wants to complete the arguments to task [TAB], the completion loader first checks task and, if not found, task.bash. However, task.bash might not be the completion setting for task.bash.

I feel we should prefer completions/<cmd>.bash to completion/<cmd> because it confuses text editors less (see #1323). We have -*- shell-script -*- and ex: filetype=sh in every completion file, but not all the text editors respect this. For example, if a completion setting for a python script mytool.py would install the completion setting at completions/mytool.py, text editors may easily mis-detects the language to be Python even though the completion setting is written in Bash.

[akinomyoga]

xfunc compatibility

This is even another topic but somewhat related. In #1321 (comment), it was suggested to move some of nslookup completions to the upstream bind9 project so that they include all the related functions in a single completion file and reuse the functions in completions/nslookup from their completion.

An alternative solution would be to keep those functions in completions/nslookup (but rename them to _comp_xfunc_nslookup_*) and call them through _comp_xfunc. However, this is non-trivial when an older version of completions/nslookup is loaded, where the xfunc interface is not defined.

[akinomyoga]

Deprecated completions in completions/

I think #894 is also related, where moving "the deprecated completions" to a separate place was suggested. However, the details haven't yet been discussed.

For example, I think these could be moved to completions/deprecated/.

[akinomyoga]

I probably don't understand your question, but all versions of Bash support loading the file directly by source completions/arbitrarydirname/commandname. If you mean automatic loading of the file (without explicitly calling source ...), no versions of Bash support loading completions from anywhere. What implements the automatic loading is bash-completion but not bash. If you mean automatic loading by bash-completion, the answer is no. We need to implement it after this discussion.

[akinomyoga]

Thankyou.

If you mean automatic loading of the file (without explicitly calling source ...), no versions of Bash support loading completions from anywhere.

This is something what I wanted to know.

We need to implement it after this discussion.

I prefer merging sub-directries (or filter out) at ./configure and make install instead of implementing something to bash-completion. I think user is not interested about deprecated or not.

Could you reply in the thread (rather creating a new thread)?

[akinomyoga]

I prefer merging sub-directries (or filter out) at ./configure and make install instead of implementing something to bash-completion. I think user is not interested about deprecated or not.

I don't understand. We discuss the separation of the directory structure of the installation target but not the one in the source tree. The reason that I suggest introducing a separate directory is to avoid conflicts happening in the installation target. In that sense, the directory structures within the source tree don't need to be modified, but the directory tree in the target tree needs to be modified to solve the problem.

[oech3]

User does not need both of completions: for fallbacking and deprecated. So they does not conflict at file level. No?

[akinomyoga]

Right. Fallback and deprecated completions wouldn't conflict with each other because both are maintained by us, but the original discussion should have been the conflict between the deprecated/fallback completions vs external completions. If files completions/<subdir>/* are merged (or filtered out) into the parent directory completions, the conflicts appear.

If you talk about merging the completion files into a single sub-directory completions/<single-subdir>, the conflict problem doesn't appear, but bash-completion doesn't support it currently, so we still need to implement it after discussion.

[oech3]

Thankyou.

If you mean automatic loading of the file (without explicitly calling source ...), no versions of Bash support loading completions from anywhere.

This is something what I wanted to know.

We need to implement it after this discussion.

I prefer merging sub-directries (or filter out) at ./configure and make install instead of implementing something to bash-completion. I think user is not interested about deprecated or not.