Question: is there a way to source completion files always and not just on-demand

[calestyo]

Hey there.

I was looking at Debian bug #1013356 which is about fzf’s completion file, which Debian installs at /usr/share/bash-completion/completions/fzf not being loaded (unless one does fzf <TAB>) which however breaks the functionality of that (it's not just completion for fzf but like a hook that allows completion of arbitrary other things (e.g. SSH hostnames) usingfzf.

So I wondered whether bash-completion provides means to have such things sourced in general. Couldn't find anything in the FAQ or previous issues. And I think helpers aren’t what I'm looking for either.

Thanks,
Chris.

[calestyo]

Oh and I should add:

I was specifically looking for some place in /usr where such script would be sourced.

I mean there's:

bash-completion/bash_completion

Lines 3206 to 3223 in f260fa8

	# source compat completion directory definitions
	_comp__init_compat_dirs=()
	if [[ ${BASH_COMPLETION_COMPAT_DIR-} ]]; then
	_comp__init_compat_dirs+=("$BASH_COMPLETION_COMPAT_DIR")
	else
	_comp__init_compat_dirs+=(/etc/bash_completion.d)
	# Similarly as for the "completions" dir, look up from relative to
	# bash_completion, primarily for run-in-place-from-git-clone setups.
	# Notably we do it after the system location here, in order to
	# prefer in-tree variables and functions.
	if [[ $BASH_SOURCE == */* ]]; then
	_comp__init_compat_dir="${BASH_SOURCE%/*}/bash_completion.d"
	[[ ${_comp__init_compat_dirs[0]} == "$_comp__init_compat_dir" ]] ||
	_comp__init_compat_dirs+=("$_comp__init_compat_dir")
	else
	_comp__init_compat_dirs+=(./bash_completion.d)
	fi
	fi

but /etc/ should typically rather be used by admins and not packages.

[calestyo]

Oh and last but not least:

There might be more potential users for such functionality. E.g. again in Debian there's /etc/bash_completion.d/git-prompt which merely seems to exist in order to unconditionally load __git_ps1() and friends.

For something in /usr to make sense, there would also need to be a way for users to mask such files, e.g that they can selectively disable loading the fzf and/or git stuff.

[akinomyoga]

One option is to put it in a compat directory. The compat directory can be configured by BASH_COMPLETION_COMPAT_DIR. The default location of the compat directory for the system-wide installation is typically /etc/bash_completion.d. Or you can check it by the following command:

$ pkg-config bash-completion --variable compatdir

If you clone the bash-completion directory in your home directory, it would be $repo/bash_completion.d. If you install bash-completion under a specific installation prefix, it would be $prefix/etc/bash_completion.d.

---

Another option is to put it in profile.d. If you would like to put it in a system-wide location, if the system profile supports the profile directory /etc/profile.d (or /etc/profile/profile.d), you can put it in $profile_dir/001_fzf.sh, where $profile_dir is the path to the profile directory, and 001 should be some number to control the loading order of scripts inside the profile directory. If you would like to configure it for a specific user, you may put it in ~/.profile.d/ or some user profile directories supported by the default ~/.bashrc of the distribution. If the system default ~/.bashrc does not support such a directory, you can manually add the code in ~/.bashrc:

# ~/.bashrc

# something like this (Please examine it. I haven't tested it)
for _bashrc_profile_script in ~/.profile.d/*.{sh,bash}; do
  if [[ -f $_bashrc_profile_script ]]; then
    source "$_bashrc_profile_script"
  fi
done
unset -v _bashrc_profile_script

---

Or one of the easiest solutions is to source the file directly from ~/.bashrc.

# ~/.bashrc

source /path/to/your/fzf.sh

[akinomyoga]

There might be more potential users for such functionality. E.g. again in Debian there's /etc/bash_completion.d/git-prompt which merely seems to exist in order to unconditionally load __git_ps1() and friends.

This is a general problem that also happens with /etc/profile.d/*.sh (which is unrelated to bash-completion). If a distribution profile script has a solution, maybe we can take the same approach as the distribution profile script. For the time being, one way is to override the builtin . and source to filter the calls by the first argument:

# ~/.bashrc

function source {
  if [[ $1 == ?(*/)your-exclude-pattern ]]; then
    return 1
  fi
  builtin source "$@"
}
function . { source "$@"; }

This approach can be used for any sourced scripts without being limited to bash-completion auto-loaded files. edit: A drawback is that it can unexpectedly localize global variables declared by declare when the sourced Bash script assumes that it is sourced from the top-level context instead of the inside of functions. Since Bash 4.2 introduced declare -g, it should probably be fine for the recent scripts.

[calestyo]

Let's see.

Your first solution is the one that I also found (i.e. /etc/bash_completion.d). Works of course, the only issue with that would have been, that it's perhaps not the most well suited location for packages to install stuff (at least not if one follows the schema done by projects like systemd, where package configuration goes into some /usr location and the one in /etc is merely for admins to override the former on a system-wide basis.

Your second proposal (some profile.d) is IMO less suited than the first one, as such files would be sourced, even if e.g. bash-completion is not even installed, but then fzf’s scripts would be useless, too (I guess).

It works of course, just like your 3rd proposal, which is however also not really suited for some out-of-the-box-installation by a package.

Your idea with overriding source is nice, though it's probably not so well suited for end-users - at least not in comparison to e.g. the schema used by systemd, where a config file in /usr/something is disabled e.g. by symlinking a same-named file in /etc/something/ to /dev/null.

For /etc/profile.d/ I'd say the problem is somewhat non-existing. E.g. in Debian, if a file is placed there by a package, it's typically a conffile, and when that is removed it shouldn’t get re-installed on upgrades, thus providing a way for the user to "disable" it.

I guess there wouldn't be any interest on the bash-completion side, to implement a schema like systemd does?!
E.g. where one had something like:

/usr/share/bash-completion/always.d/

for scripts that are always loaded, and a counterpart file in

/etc/bash-completion/always.d/ or so would override the one in /usr when it has the same name.

If there is (interest), than we could leave this issue open to track that, otherwise, I think my question is answered and the best approach to me seems /etc/bash_completion.d and we can close it.

Thanks :-)

Chris.

[akinomyoga]

bash-completion is not even installed, but then fzf’s scripts would be useless, too (I guess).

This is not true. Fzf's completion.bash is unrelated to bash-completion works without bash-completion. The only requirement is that if one wants to use fzf's completion.bash and bash-completion at the same time, completion.bash needs to be loaded after bash-completion so that completion.bash can turn on the integration.

In this sense, trying to put fzf's completion script under the control of bash-completion is strange. There is also another Bash configuration provided by fzf: key-bindings.bash. Fzf's completion.bash is usually installed together with fzf's key-bindings.bash under the tree of /usr/share/fzf or /usr/share/doc/fzf so that users can find it by themselves and source it from their ~/.bashrc. The actual location depends on the distribution and the package. The typical paths are $prefix/share/fzf/shell/completion.bash, $prefix/share/fzf/completion.bash, $prefix/share/doc/fzf/examples/completion.bash, etc.

I guess there wouldn't be any interest on the bash-completion side, to implement a schema like systemd does?!
E.g. where one had something like:

Isn't it already possible? You can create a symbolic link in /etc/bash_completion.d/ pointing to /usr/share/fzf/shell/completion.bash (or the path where the user's distribution places completion.bash). The problem might be that in this way the list of available options is not visible, but in the first place, fzf's completion script is not a submodule of bash-completion. It is an independent configuration unrelated to bash-completion, though it contains a piece of code to make it work with bash-completion when both are loaded.

You don't seem to be happy with directly adding source /path/to/fzf/completion.bash in your ~/.bashrc because it's not an out-of-the-box installation, but it is subtle whether the systemd-style scheme can be considered out-of-the-box. The user needs to turn it on by creating a symbolic link (directly or indirectly by using another command). This is not so different from adding one line source /path/to/fzf/completion.bash. One difference is that to turn it off, the symbolic link can be removed programmatically in a safe way while it is difficult to safely remove the source line by programmatically editing ~/.bashrc in the latter case. However, I doubt there is sufficient demand to enable programmatically turning fzf's completion on/off in practical cases.

[calestyo]

This is not true. Fzf's completion.bash is unrelated to bash-completion works without bash-completion.

Ah, I see. I hadn't realised that it can be used without bash-completion… but I'm a bit confused now :-D

completion.bash does contain:
https://github.com/junegunn/fzf/blob/76364ea767cca7ca8f6570a049fbb8d3fff751a9/shell/completion.bash#L324-L329
and:
https://github.com/junegunn/fzf/blob/76364ea767cca7ca8f6570a049fbb8d3fff751a9/shell/completion.bash#L65-L133

Which, AFAIU, is bash-completion for the fzf program itself, right? So that should in fact go into /usr/share/bash-completion/completions/, to be on-demand loaded for fzf only.

Whereas everything else(?) in completion.bash is for the "integration", right?

So would the best thing to do be to split up completion.bash with the completion for fzf itself going into /usr/share/bash-completion/completions/ and the rest going into some other /usr/ location from where either users source it manually or the package installs it into e.g. /etc/profile.d/, somehow making sure it's loaded after bash-completion?

[calestyo]

Isn't it already possible?…

plus:

You don't seem to be happy with directly adding

Well my main intention is to get the Debian bug fixed.

For that - assuming some out-of-the-box functionality is desired - any modification of files in ~/ is simply not possible.

The systemd schema, in man cases at least, is in so far out of the box, that it would load them from e.g. /lib/systemd/system without anything needed to be present in /etc´ to do so. (Though, yes, it does happen in some cases, that symlinks are created in /etc/systemd/system`, but AFAIU, that's not generally necessary.

I personally can in principle also live with putting the file (or better a symlink to it) in either /etc/profile.d/ or /etc/bash_completion.d/ (perhaps the former would be cleaner - if fzf’s completion.bash would be split up (if my above assumptions are true).

It's just that I had the feeling (though I may be wrong here), that recently it was considered better style in Debian if such configs are not placed in /etc

[akinomyoga]

Which, AFAIU, is bash-completion for the fzf program itself, right?

No, fzf's completion.bash is a Bash completion but not bash-completion. Bash completion settings and bash-completion are different. The latter is merely one case of the former. When we say bash-completion, it means this particular project, scop/bash-completion. (In the past there seem to have been multiple projects called "bash-completion", but they were merged into this bash-completion). The lines you mentioned use the complete builtin and COMPREPLY, but those are not provided by bash-completion but by Bash itself. Bash completion configurations can be created independently of bash-completion.

So that should in fact go into /usr/share/bash-completion/completions/, to be on-demand loaded for fzf only.

/usr/share/bash-completion is the data directory of this project and is not a directory shared by arbitrary Bash completion configurations. /etc/bash_completion.d is also a directory defined by this project, and is not a directory defined by Bash. In fact, fzf's completion works without bash-completion.

Whereas everything else(?) in completion.bash is for the "integration", right?

No. The integration with bash-completion (not general Bash completions) is only these lines.

So would the best thing to do be to split up completion.bash with the completion for fzf itself going into /usr/share/bash-completion/completions/

There is no need to put it in /usr/share/bash-completion/completions as far as fzf's completion does not rely on bash-completion. The directory /usr/share/bash-completion/completions are the directory for the completion setting that wants to use the loading mechanism of bash-completion, but it is NOT the directory for all the Bash completion configurations defined by Bash. It is always allowed to put your completion script in /usr/share/bash-completion/completions even when it doesn't rely on bash-completion, but it's not mandatory at all. The author of fzf should finally decide it.

and the rest going into some other /usr/ location from where either users source it manually

If the rest settings are placed in another place (such as /usr/share/fzf/shell/completion.bash) and sourced, you can put the completion setting of the fzf command in that file. From the point of view of the upstream Bash project, there is no difference between putting it in /usr/share/bash-completion/completions/fzf or in /usr/share/fzf/shell/completion.bash. Both fzf and bash-completion are equally third-party projects. Then I don't see any reason to split completion.bash.

For that - assuming some out-of-the-box functionality is desired - any modification of files in ~/ is simply not possible.

What is the problem in modifying files in ~/? Probably modifying ~/.bashrc cannot be automated, but putting a file in ~/.profile.d or something should not be so difficult.

The systemd schema, in man cases at least, is in so far out of the box,

What do you mean by OOTB in this context? Shouldn't a configuration be enabled by e.g. calling systemctl enable? It internally creates symbolic links such as in /lib/systemd/system/multi-user.target.wants. It's not OOTB in the sense that e.g. just defining a service as a unit file in /lib/systemd/system doesn't immediately start the service or enable starting on the system boot. systemctl start and/or systemctl enable should be called somewhere (manually or automatically by an installer). It is true that it is non-trivial for installers to automatically add source /path/to/fzf/completion.bash in ~/.bashrc, but it should be easy to automatically put a symbolic link in ~/.profile.d without any conflicts just as done in systemd.

For example, a distribution can define a system-wide directory (such as /usr/share/profile) that provides a set of available shell settings, and to enable a setting, a user can create a symbolic link to /usr/share/profile/some-setting.bash in ~/.profile.d. If you don't like to manually create a symbolic, you could instead create a shell script profilectl so that a user can call profilectl enable some-setting.

---

Also, as a developer, I would like to request packagers not to independently invent a random scheme that is only available in a specific distribution. If you would like to invent a new scheme, I would like to ask packagers to put some effort into discussing it with packagers of all major distributions and introduce the scheme after some agreement. Otherwise, every distribution requires a specific way of configuration, which is a mess from the perspective of the developers of the projects that rely on the setting. In particular, for fzf's shell configurations, the location of the script file is already diverse and troublesome to automatically detect. Here are the related code to detect the location of fzf's shell settings in blesh-contrib:

• https://github.com/akinomyoga/blesh-contrib/blob/13e0682649a6cb40cbc5b82081f64cd458534b90/integration/fzf-initialize.bash#L29-L47
• https://github.com/akinomyoga/blesh-contrib/blob/13e0682649a6cb40cbc5b82081f64cd458534b90/integration/fzf-completion.bash#L8-L16

Before adding even another location that possibly contains fzf's shell settings, you should normalize those locations and form an agreement among the packagers of fzf in all the major distributions.

[calestyo]

No, fzf's completion.bash is a Bash completion but not bash-completion.

Okay, what I meant was, that completion.bash provides two kinds of completion:

1. the one for fzf itself, comparable to what bash-completion ships for many programs and what other programs ship themselves but using functions form bash-completion, despite that fzf doesn't use bash-completion specific functions but only the completion functions provided by bash
2. the more general completion functionality, that fzf provides for other cases (like the hostname completion).

I "only" got in wrong, in that I assumed completion.bash would also use bash-completion functionality for (1).

The directory /usr/share/bash-completion/completions are the directory for the completion setting that wants to use the loading mechanism of bash-completion, but it is NOT the directory for all the Bash completion configurations defined by Bash. It is always allowed to put your completion script in /usr/share/bash-completion/completions even when it doesn't rely on bash-completion, but it's not mandatory at all.

I think that's just what I'm trying to find out:

The (1) part of fzf’s completion.bash would in principle be well suited fort hat on-demand loading provided by bash-completion, even if it doesn't really need bash-completion.

The (2) part of fzf needs always to be loaded (in order to take effect). And despite completion.bash not using bash-completion and /etc/bash_completion.d being strictly speaking a directory of that, it would still be a logical choice for a distribution packager to put the (2) part in - or alternatively into /etc/profile.d/.

At least nobody will introduce a /etc/bash/non-bash-completion-completions.d/ or so.

The author of fzf should finally decide it.

Of course. But he may not even have thought about it yet. So that's why I tried to find out what would be the cleanest way, in order to present that to him.

Then I don't see any reason to split completion.bash.

Well my only point would have been to keep the shell env as clean as possible and load the functions needed for fzf’s own completion only on-demand (as nicely provided by bash-completion).

---

What is the problem in modifying files in ~/? Probably modifying ~/.bashrc cannot be automated, but putting a file in ~/.profile.d or something should not be so difficult.

In principle it would of course work, but I don't think this is currently done, at least not in Debian, by any packages.

Actually, I think, Debian’s default .profile do not even source any ~/.profile.d or so.

Also, I rather think packages should generally never write to /home.

It's not OOTB in the sense that e.g. just defining a service as a unit file in /lib/systemd/system doesn't immediately start the service or enable starting on the system boot. systemctl start and/or systemctl enable should be called somewhere (manually or automatically by an installer).

Uhm, I thought any such files (that were just in /lib/systemd/system) should be loaded. Whether the unit then gets activated is another thing of course, but even for that it should not generally be necessary to have symlinks in some /etc/systemd/system/foo.type.wants.

It's enough if any unit that gets activated has a Wants=, Requires= or so on the unit in question.

Weren't the /etc/systemd/system/foo.type.wants/* anyway just created for reverse dependencies like WantedBy= and friends?

For example, a distribution can define a system-wide directory (such as /usr/share/profile) …

Sounds definitely reasonable. Actually I'd love to see that, cause I, personally, don't like that one silently gets stuff enabled by packages putting stuff into e.g. /etc/profile.d/.

But it's unlikely that I'll be able to convince Debian to set up such structure.

Also, as a developer, I would like to request packagers not to independently invent a random scheme that is only available in a specific distribution.

Same here… I agree, but I just try to fix that one bug in a clean manner.

I think I might have some chance (and even that is all but sure) to convince fzf usptream to perhaps split completion.bash, if that would help packagers in the end with their job.

But convincing all distros about some general structure how they load various types of completions some using bash-completion, some not, as well as other arbitrary shell configs... or even just convincing all fzf maintainers of doing it he same way,... that sounds pretty much impossible :-D