hckrnws
I didn't like the idea. I prefer the alternative approach: _I_ decide the order of dirs in the PATH env. If I introduce an executable with a name, that overrides a system one - I probably do that intentionally.
If I introduce an alias (like `grep='grep --binary-files=without-match --ignore-case --color=auto`) that matches the name of a system binary - I probably do that intentionally.
And if I EVER need to call grep without my alias - I just prefix it with a backslash: \grep will search with case sensitivity and no color and will scan binaries.
Looked so backwards to me, too. However, I decided to give it a go, anyway. Now, I have some scripts and small commands which start with a comma, and it looks neat and time saving.
Yes, I can do path ordering to override usual commands. However, having a set of odd-job scripts which start with a comma gives a nice namespacing capability alongside a well narrowed-down tab-completion experience.
While it's not the neatest thing around, it works surprisingly well.
Another idea which looks useless until you start using is text expanders (i.e.: Espanso and TextExpander).
I never knew that what I've known as 'hotstrings' (since the AutoHotKey days) other sometimes also call 'text expanders'.
Love Alfred Snippets for this same text expander need.
The irony in the number of extra commas you've used in this comment...
As a non-native English speaker and writer/typer I'm not well versed in usage of commas unfortunately.
Feel free to add the required ones while reading this comment.
Sorry for the inconvenience this might create.
As a native speaker the original comment seemed completely fine, ignore them. Also, I never would never guessed that you weren't also a native English speaker.
Agreed. The commas before the sentence-ending 'too' and 'anyway' were perhaps slightly unusual, but not enough so that I even noticed them, and I don't think either is incorrect. All the rest were perfectly normal.
This is an example of the difficulty in expressing tone through text. Meant it as a passing lighthearted observational joke.
No inconvenience at all.
Either adding your script directory in front of the PATH, or creating `alias` that provide a full path to your script where a conflict exists, makes a whole lot more sense to me.
I've never had this collision problem yet, despite appending my script directory to the end, but I'll use either of the above solutions if that ever becomes a problem.
From my own aliases:
alias curl='/opt/homebrew/opt/curl/bin/curl '
alias rsync-copy='/opt/homebrew/bin/rsync -avz --progress -h '
alias rsync-move='/opt/homebrew/bin/rsync -avz --progress -h --remove-source-files '
alias rsync-synchronize='/opt/homebrew/bin/rsync -avzu --delete --progress -h '
alias rsync-update='/opt/homebrew/bin/rsync -avzu --progress -h '
alias vi='/opt/homebrew/bin/vim -S ~/.vimrc'
alias vim='/opt/homebrew/bin/vim -S ~/.vimrc'
alias wget='/opt/homebrew/bin/wget -c '
Slightly confused as to why - surely homebrew adds itself to the PATH ahead of the system utilities?
Also - surely vim auto-reads your vimrc?
I learned to deliberately declare paths pretty early on in my adventures at the CLI. I don't leave room for accidental alternative execution. It might be overkill, but it gives me a sense of security and that's why it's there. Don't worry, I probably made a terrible mistake somewhere else that completely negates my attempts at a correct shell environment.
One rarely actually needs to shadow binaries. Some cases could indeed be covered by introducing an alias that binds the binary's name to call a different copy of that binary.
You use shadowing to fix issues where you install some software that expects you to have a sane and ~recent version of some tool like git, but you don't as your system provides that binary and unfortunately it is either not sane (not GENERALLY sane [while it could be sane for system scripts]) or not recent enough. In that case the program's function would simply fail if it would call the system's binary and you shadow the binary with your version to fix that.
> adding your script directory in front of the PATH
That's a poor advice for the scripts you call relatively frequently. Instead, (as a general approach, we aren't discussing some particular script) don't use shadowing for scripts: just pick a non-conflicting script name and append the script's dir to $PATH.
> That's a poor advice for the scripts you call relatively frequently.
Why? It protects you from someone else (cough updated packages introducing new commands cough) picking a name you already use.
Because it's useless extra typing. People try to narrow commands down to two fucking chars and you suggest to type the whole goddamn path!
Nobody suggested typing whole paths.
They did. Just not explicitly.
The only situation where you'd need to type a full path is when you have a name collision. But the suggestion to not worry about collisions with unknown system binaries also said to put the script folder first. In that situation running the script doesn't need a full path, and you won't be running the unknown system binary. So you won't be typing full paths.
Please explain the non-explicit suggestion you see, because I don't see it.
It's clear that "adding your script directory in front of the PATH" means a one time edit to .bashrc or equivalent, right? In normal use you're not typing it.
I've asked you why prepending paths to PATH instead of appending was a bad idea. How is having typing full paths has anything to do with it?
Comment was deleted :(
TIL: Backslash overrides alias - wow!
Thanks, mathfailure - this genuinely improves my life!
‘command grep’ also works in several shells. A little longer but looks good in scripts etc.
> I just prefix it with a backslash: \grep
Word of warning, I use `\grep` quite frequently. The usage is when you are piping after grep or saving to a variable.
Illustrative example:
$ TO_DL=$(curl "https://foo.com/releases/" \
| grep -e "latest" \
| head -n1 \
)
$ curl $TO_DL
curl: (3) bad range in URL position XX
https://foo.com/releases/latest.tar.gz
^^^^^^
Annoying footgun, but something to be aware of in case you go down this extremely confusing rabbit hole like me. I couldn't figure it out until I decided to hexdump the string.
[Side note]: My first thought reading the article was also about how `\` basically solves the problem but they do have one advantage in that they can do `,<tab>` and get a list of all their custom commands. Personally not worth it for me but I can definitely see this being useful for some. Especially on shared systems. But getting everyone to prefix a script name seems just as unlikely as getting everyone to place programs in the right location.
The problem with this is that you don't know what future conflicts will be. You spend years training yourself to use your own "jq" alias and then you find yourself needing to use the "jq" program and you have to remember to prefix it with backslash every time you use it (including when you copy-and-paste a command line from a webpage), or rename your own alias and re-train yourself to use its new name.
When “I” means me then this usually works for me. But when “I” becomes “we”, sometimes this goes off the rails because someone introduces a bin with breaking changes that silently fucks up projects that dev doesn’t really know about, or forgot about.
Call it the Chesterton’s Fence of ‘which’.
The premise of the article is the desire to avoid accidental collisions, especially from newly installed system binaries.
In such cases you might get errors like sl being both a version control system and the steam locomotive
I do this, and routinely shadow commands with my own wrappers to do things like set environment variables.
And then there’s Claude. It deletes whatever it finds at ~/.local/bin/claude, so I have to use a shell function instead to invoke the full path to my wrapper.
You can use an alias, which takes priority over $PATH. e.g. I have this in .zhsrc to override the "claude" executable to run it in the OS sandbox:
alias claude="sandbox-exec -f ~/agents-jail.sb ~/.local/bin/claude --dangerously-skip-permissions"How does your sandbox ruleset look? I've been using containers on Linux but I don't have a solution for macOS.
Here's my ruleset https://gist.github.com/eugene1g/ad3ff9783396e2cf35354689cc6...
My goal is to prevent Claude from blowing up my computer by erasing things it shouldn't touch. So the philosophy of my sanboxing is "You get write access to $allowlist, and read access to everything except for $blocklist".
I'm not concerned about data exfiltration, as implementing it well in a dev tool is too difficult, so my rules are limited to blocking highly sensitive folders by name.
That's neat. I'm going to base my ruleset off of yours. I've been messing around with claude more and more lately and I need to do something.
I would recommend against overriding standard system binaries, you could break compatibility on your system with scripts that depend on those binaries. I just use an abbreviation like rg=“grep -RE”
Why are those scripts running in interactive login shells? If they are influenced by the configuration of profile, then the scripts are bad.
That’s true, but I would still call overloading system binaries bad practice. Your making yourself foot gun.
No, in fact he is correct: system scripts won't pick up your overrides configured via your shell's rc scripts.
I said he was right, but in your case you are wrong. Your shell loads thr profile script for login shells, but will load your bashrc regardless of login or not
Excluding this argument, overloading system commands is still bad practice lol. Have you ever logged into an embedded device and had a busy box environment? the behaviour of some utilities can be different from what you expect for utilities of the same name.
Any severe side effects so far? Have you set PATH up somehow so it is effect only on interactive prompt, and not in the launched processes?
Because I cannot imagine much 3rd party scripts working with random flags added to core tools
I also do this.
Random flags added to core tools are done with aliases, which do not affect the launched processes, not by shadowing them in ~/bin. Shadowing in ~/bin are for cases where a newer (compared to the system-wide version) or custom version of a tool is needed.
Not really, since if one usually does that - they probably understand the possible consequences and don't shadow whatever they like, but do it carefully.
On MacOS I shadow that way just curl and git binaries to the versions installed from homebrew and nothing has broken (yet). I know that tar on MacOS is also a weirdo that I'd rather shadow with the homebrew's gtar, but their args are different and I of course understand that there's a high probability of something in system to be bound to mac's version of tar, so here I better remember to use 'sane' tar as gtar or use an alias (instead of shadowing the binary) for tar to use gtar (because aliases are for users, not for system scripts/processes).
And on my home desktop's Debian - I don't even use shadowing of binaries at all (never needed it).
Also, I just realized: I change PATH env via my shell's rc script (~/.zshrc), so I probably could worry even less about shadowing system binaries (like tar on MacOS) possibly breaking things.
I do the same thing, but I also have a command that shows me what functions or scripts might be shadowing other scripts
Care to share?
the sibling answer but with `-a` before command name, will display all path hits for a command.
which <commandname>Seemed like it was more than that, but the comment is ambiguous. I took it to mean "show me all the commands which are shadowed" not "is this command shadowed"...
nope, that just gets you the first hit, not all of them
type -a commandname
quite simple
type -a <commandname>
this. which(1) and whereis(1) are not bash, only an approximation or coincidence at best:
$ type -a which
which is /usr/bin/which
$ type -a type
type is a shell builtin
type is /usr/bin/type
$ help type
type: type [-afptP] name [name ...]
Display information about command type.
For each NAME, indicate how it would be interpreted if used as a
command name.
Options:
-a display all locations containing an executable named NAME;
includes aliases, builtins, and functions, if and only if
the `-p' option is not also used
-f suppress shell function lookup
-P force a PATH search for each NAME, even if it is an alias,
builtin, or function, and returns the name of the disk file
that would be executed
-p returns either the name of the disk file that would be executed,
or nothing if `type -t NAME' would not return `file'
-t output a single word which is one of `alias', `keyword',
`function', `builtin', `file' or `', if NAME is an alias,
shell reserved word, shell function, shell builtin, disk file,
or not found, respectively
Arguments:
NAME Command name to be interpreted.
Exit Status:
Returns success if all of the NAMEs are found; fails if any are not found.
$ $SHELL --version
GNU bash, version 5.3.9(1)-releaseJust on your first suggestion, this also means that if a person or process can drop a file (unknown to you) into your ~/bin/ then they can wreak havoc. Eg they can override `sudo` to capture your password, or override `rm` to send your files somewhere interesting, and so on.
Btw on the second suggestion, I think there's a command named `command` that can help with that sort of thing, avoids recursive pitfalls.
That would require someone to already want to sabotage me in particular, learn my private workflows, and also have write access to my home folder. At that point, All is Lost.
Don't tell people to sacrifice agency for apocalypse insurance that doesn't work, lol
If someone can drop a file in your ~/bin, they can also edit your shell’s startup files to add their malicious command.
I think it's already game over if they have access to your home directory. They can also edit your path at that point.
The issue of rootless malicious command overrides is solved by typing the whole path, such as "/bin/sudo".
No, don't do that as a precaution. As others have already answered correctly - it's too late to worry about such things if a malicious agent has write access to your ${HOME} dir.
While true, what you describe is very unlikely to happen and most definitely won’t happens on systems where i’m the only users.
> If I introduce an executable with a name, that overrides a system one
... and breaks existing scripts that reference the system one, right?
Not if it is an alias.
But yes if it’s another executable.
Comment was deleted :(
curious if you're customizing anyway, why not use eg ripgrep?
Others have already given valid answers: grep is not ripgrep [their params don't match], so it's a bad idea to alias 'grep' to use ripgrep. But it's okay to alias 'ripgrep' (or 'rg' or whatever) to use ripgrep with some args.
Comment was deleted :(
repgrep's CLI options and general behavior are different from grep. I tend to use both for different things.
Not OP, but I use ripgrep and customize it with an alias as well, so it applies equally there
Tangentially related. Don't ever put "." in your PATH. I used to do this to avoid typing the "./" to execute something in my current directory. BAD IDEA. It can turn a typo into a fork bomb. I took down a production server trying to save typing two characters.
It used to be very common to "own" a unix system by adding a `ls` binary in some folder and waiting for an administrator to run it.
Why would this own a server? ls lists itself, but listing itself shouldn't cause it to run again? Where's the infinite loop that brings the server down?
I think parent comment means "cp badthing ls" and leave it latent for someone to run. Maybe $PATH has CWD first for convenience?
They're not talking about the same scenario. Owning isn't denial of service. And they didn't say the `ls` lists things (though it probably will do that at the end).
Related: My grad school had a shared bin/ directory with common local tools. Of course, that directory was after /bin and /usr/bin in the PATH so that no one could override “ls” or “more”. So… one grad student added scripts with names that matched common typos: “sl” and “mroe”!
Sure enough, they got run. The scripts didn’t take over your account. They ran “ls” and “more”. They may have also logged your username in a file so he could lord it over you.
I like to follow my own convention where I name files with shell scripts with an extension: .sh for POSIX-compatible scripts, .bash for scripts with bashisms or .zsh for scripts with zshisms.
If I ever wanted to achieve what you initially wanted to achieve - I could use something like
alias -s sh=sh
alias -s bash=bash
alias -s zsh=zsh
Just like I do bind .txt and .conf to 'less', .pdf to 'qpdf', .json to 'ijq', video formats to 'mpv' and so on.
Might I ask exactly what the typo was?
Elaborate?? "." has been at the end of my PATH for like 20 years.
I could drop a shell script that does something like
echo "lanyard2 ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/lanyard2 ; ls
if you ran ls in my dir, you would give me sudoers access
Just to save the trouble of writing './'?
Yes??? :)
I'm not the crazy one here! That's a whole two characters on the same side of the keyboard...
How often do you find yourself running executables from the current directory? Is this a daily thing?
For my workflow, yes
I don’t think it’s a severe security vulnerability. The same thing can happen with $home/bin.
I think it's substantially riskier. At the very least, it means you are trusting any directory you cd into, rather than just trusting your $home/bin.
Stuff that would not typically raise eyebrows has been made risky. You might cd into less privileged user's $home, or some web service's data directory, and suddenly you've given whoever had access to those users, access to your user.
Maybe you could argue "well, I just won't cd outside of my $home", but the sheer unexpectedness of the behavior seems deeply undesirable to me.
Why does this go wrong and in what situation?
Somebody mentioned it elsewhere, but it is a security risk: if you end up in a directory that's not under your control, and you do a "ls", it might execute "./ls" instead of /usr/bin/ls, and that can be doing anything, including piping your ~/.ssh/id_* to a remote server.
This can also happen by downloading something off the internet (git clone, or tar xz foo.tar.gz), or on a multi-user system (eg. someone can put any of these common commands into /tmp/ and wait for you to drop into it and try a "ls" there) — if you have any untrusted content anywhere, you are exposed.
> if you end up in a directory that's not under your control, and you do a "ls", it might execute "./ls" instead of /usr/bin/ls,
Not if if you APPEND the dot path to the PATH env: the system traverses the dirs specified in the PATH env from left to right and stops at first match. Your system's ls binary is in the dir that's to the left of your '.' dir.
Yeah, there are ways to reduce the impact (as a sibling comment mentioned, typos or commands missing on a system could still be used to trick the operator), but I was mostly explaining the attack vector in case it is present in the PATH.
Then it's a little bit of a stretch but they could put a script with the name of a common typo similar to commonly run commands there. Maybe "ls-l" without the space in case they miss the space. Yeah, that's a stretch. I went looking for better sources.
> The current directory ( . ) is not in PATH by default, for security reasons. This prevents accidentally running unintended programs in your current directory.
-- POSIX Shell scripting from scratch, By Sultan Zavrak (states it in general terms. They also use ls as an example though, which shouldn't be affected if you have "." at the end.)
Practical UNIX and Internet Security has an example of "." (or having a null entry in the PATH, which also indicates the current directory; I didn't know that![0]) at the beginning, which is obviously a bad idea, but he (Simson Garfinkel) makes a good point:
> More generally, you should never have a path that is writable by other users.
Ah yes, finally, he covers a situation where you have a directory at the end of your path, that is writable by others ("." would count) and having a trojan named "mroe" (for "more") waiting patiently for the superuser to mess up.
He even goes so far to say that root should run commands with full paths, such as /sbin/chown and not just chown. I've never gone that far, except I can see the benefit of doing that in scripts.
So anyway, besides the typo example, there's also a kind of shadowing: let's say you expect a command to fail because the program is not installed. Or maybe you try to run a command you think is installed but it's not. You might even have a command or way of working that tries various commands until one works. If you have a path that someone can write to (including ".") then instead of failing, it will run something unintended, if they have shadowed that command in that directory.
[0] to quote the bash man page: A zero-length (null) directory name in the value of PATH indicates the current directory. A null directory name may appear as two adjacent colons, or as an initial or trailing colon.
A trip down the recursion hole. Also, scripts will inherit the relative path so they will have different absolute paths from each other. Seems easier to just type ./ so it's kinda funny in a "UNIX haters handbook" kind of way, but it's not even a fault in linux's command interface in that case. We've all been there.
Oh, that's without even going into the security risks and loss of portability.
Presumably a script that aliases a common thing or something and then it uses the same. E.g. someone adds ./sed that has some default params and calls sed. You’re intended to call it with ~/not-in-path/defaulted/sed and it is supposed to then call sed but instead calls itself if it’s earlier in the path hierarchy.
Might even be as simple as “detect if I’m running gnu sed or bsd sed and use the appropriate one”. Obviously you can not have this problem by being smart about other things but defense in depth right?
Not if if you APPEND the dot path to the PATH env: the system traverses the dirs specified in the PATH env from left to right and stops at first match. Your system's sed binary is in the dir that's to the left of your '.' dir.
Appending is much better than prefixing, but having "." in the path, anywhere, can still open you up to running mistyped commands (arguably a much less common possibility, but still a possibility).
I.e., you have "." as the very last item in your path. You are in /tmp/ (so a directory other uses can write files to). You mean to type "ls -l something" to look for "something" files. But instead, you just miss the space, and type "ls-l something*", and some other nefarious user has left a /tmp/ls-l binary behind just waiting to be run. It could package up your ~/.ssh folder and ship it off to "nefarious" user, and then do a proper "ls -l" so that you may not even notice the typo.
And, if you happen to be root when you are in /tmp and mistype ls-l, and if the ls-l binary checks to see if it is being run as root, it could then do even worse. For example, it could leave behind an suid to root bash or sh executable in 'nefarious user's' home dir, so that 'nefarious' can now become root at some point later and proceed to actually 'own' the system.
Right, that's one way to be half-smart about it. But you have to make sure that's the final thing you append to the path. An easy mistake to make is temporal. You add `.` to the path, and time passes, someone appends `/opt/bin` to the path, and time passes, someone writes `~/not-in-path/defaulted/busybox` that references `/opt/bin/busybox` as just `busybox` and tests it by running `~/not-in-path/defaulted/busybox` while being in `~` and it works so they leave it alone, then you go `cd ~/not-in-path/defaulted/` and run it and die.
"I don't understand. I very specifically appended `.` at the end!"
Of course you can stick a comment "#the following should always be at the end of the file" or whatever or say "we should always make sure to reference binaries by their full path, so always write out `/opt/bin/busybox` rather than just `busybox`" and stuff like that. With enough system you can make this unlikely.
lol. What a beautiful footgun — for such a tiny optimization.
Most of my aliases contain `--` for the same reason, `git--progress`, `grep--rIn`, `nvidia--kill`, `ollama--restart`, `rsync--cp`, `pdf--nup`...
Easy autocomplete, I know there won't be any collision, and which command is mine.
Kinda makes no sense to me: so you don't use '--' as a prefix, you use it in the middle of an alias, so you first have to autocomplete, say, 'gi' not to 'git' but to 'git--progress'. What does that alias do? Doesn't it call git with some args? If so - why not just alias it to git?
Great hack!
Glad it worked for OP, but I've never once in 30+ years of this had a conflict that did something I didn't want. ~/bin/ is early in my PATH, and for a good reason. Things I put in there I want to take precedence, so I use this to purposely override provided bins. (Though I can only think of one time I wanted to do that, too.)
Prefixing commands solves the namespace problem and discoverability (at least partly). I use a slightly more sophisticated method, which helps me remember which custom utilities are available and how to use them: sd [1], a light wrapper written for zsh that, in addition to namespaces, provides autocompletion, custom help texts + some other QoL enhancements. Can definitely recommend if you're looking for something a bit more fancy.
Every tool and shell that lay in arm's reach treated the comma as a perfectly normal and unobjectionable character in a filename.
It's not a completely non special character: for instance in bash it's special inside braces in the syntax where "/{,usr/}bin" expands to "/bin /usr/bin". But the need to start that syntax with the open brace will remind you about the need to escape a literal comma there if you ever want one.
You may enjoy learning about the [ binary.
You never used CVS/RCS with its “,v” files?
Until someone forces you to use a file system that cannot tolerate commas...
Which file system would that be?
Many early file systems like the original FAT, RSX-11, VMS ODS-2 ... Probably not a concern for anything in the past 30 years.
What about using the filename in arrays in bash/sh?
But Bash arrays don’t use comma, what’s the problem?
Oh, that might be true, I do remember encountering some escaping issues when creating a more complex POSIX (or bash) script that involved lists and iterating through stuff.
I see Bash only uses commas in Brace expansions:
file{1,2,3}.txt # file1.txt file2.txt file3.txt
I guess it would only be a problem if you want to expand
file,.txt
file,,.txt
file,,,.txtImagine seeing this code:
echo file{",",",,",",,,"}.txtAh, I see you're a man of culture as well!
Have you met Bash? It’s a shrine to space-delimited everything lol
I reworded my comment for clarity now.
Worth pointing out that with Nix/NixOS this problem doesn't exist.
The problem in other distros is that if you prefix PATH so that it contains your executable "foo", and then run a program that invokes "foo" from PATH and expects it to do something else, the program breaks.
With Nix, this problem does not exist because all installed programs invoke all other programs not via PATH but via full absolute paths starting with /nix/store/HASH...
The "solution" of only ever using full absolute paths works on any unix system, doesn't it?
Yes with a but:
NixOS simultaneously smooths the path to using absolute paths while putting some (admittedly minor) speed-bumps in the way when avoiding them. If you package something up that uses relative paths it will probably break for someone else relatively quickly.
What that means is that you end up with a system in which absolute paths are used almost everywhere.
This is why the killer feature of NixOS isn't that you can configure things from a central place; RedHat had a tool to do that at least 25 years ago; it's that since most of /etc/ is read-only, you must configure everything from a central place, which has two important effects:
1. The tool for configuring things in a central place can be much simplified since it doesn't have to worry about people changing things out from under it
2. Any time someone runs into something that is painful with the tool for configuring things in a central place, they have to improve the tool (or abandon NixOS).
So if I want to use grep in a small script, do I have to write:
/nix/store/grep-hash -flags files | /nix/store/head-hash
instead of: "grep -flags files | head"?
If it's a one off, you just use something like "nix shell" to add it to your path for running the script.
For non one-off sorts of things, you would substitute in the nix expression "${gnugrep}/bin/grep" the "${gnugrep}" will expand to "/nix/store/grep-hash" and also make a dependency on the gnugrep package, so that the grep install won't get garbage-collected as long as your package is still around.
Here's an example[1] from a package expression for e-mail client I use, which will shell out to base64 and file. Upstream relies on these two programs being in $PATH, but this replaces the string used for shelling out with the absolute path in the nix store.
For shell scripts, I'll just do something like this near the top:
GREP="${GNU_GREP:-$(command -v grep)}"
1: https://github.com/NixOS/nixpkgs/blob/master/pkgs/by-name/no...
[user@nixos:~]$ which grep
/run/current-system/sw/bin/grep
[user@nixos:~]$ ls -l /run/current-system/sw/bin/grep
lrwxrwxrwx 1 root root 65 Jan 1 1970 /run/current-system/sw/bin/grep -> /nix/store/737jwbhw8ji13x9s88z3wpp8pxaqla92-gnugrep-3.12/bin/grep
Yes, the original comment that this problem doesn't exist in Nix is wrong for a typical user environment.
It does contain the issue a bit though:
I'm running isync in a systemd service, yet the program "mbsync" is not in my path. I have several services installed, yet their programs aren't in my path. My e-mail client shells out to "file" for mime-type verification, yet "file" is not in my path.
Run "compgen -c |wc -l" to get a list of commands; its over 7000 on my Ubuntu system and right around 2000 on my NixOS system.
As an aside, the packages that put the most executables in my path are probably going to be in the path for most NixOS installs (231 just for coreutils+util-linux):
27 /nix/store/csxa6mi2mpjl9vqxbv2j0bha6sz6nbjw-cups-2.4.14
31 /nix/store/334afxah19b3hr6ll93pfxlcyhhh2vws-pipewire-1.4.9
31 /nix/store/h2jsb5i4yfblr2f3ac2c7zpmlmj7zjym-perl-5.40.0
33 /nix/store/914x32c982bs3i1998yxvkg9svm3ycr5-shadow-4.18.0
33 /nix/store/a6s3hzj3b2z6rsyfkjyxwn265iyfl2gn-mtools-4.0.49
33 /nix/store/jky7jszaci5n7g426wf6nsg5dmik9nfw-kbd-2.9.0
37 /nix/store/2v1l6mqz0d7mfpp4ksw2048v3g0a1a19-hplip-3.25.2
45 /nix/store/90wlc37ljr6rpy2lan46bp0gq19vbgl5-iptables-1.8.11
48 /nix/store/1byhxs7b28grh8s15jc2dvs2zg36swjb-lvm2-2.03.35-bin
61 /nix/store/9xwxjkrwxjsvc5gs1l0syr4wbfvvvvcn-bluez-5.84
64 /nix/store/zf8qy81dsw1vqwgh9p9n2h40s1k0g2l1-systemd-258.2
72 /nix/store/1igrj9w84w7s3r80l3nkxcqwd84sw9mz-qemu-10.1.2
106 /nix/store/v4q3154vdc83fxsal9syg9yppshdljyk-coreutils-full-9.8
125 /nix/store/3c6r8gh8zrqw8xmncmlj9vivz9rz6r30-util-linux-2.41.2-binTrue enough, but in my experience it's not really much of a problem because if I'm not doing Nix, then I'm doing containers which are widely available.
What can be a problem is muscle memory, when you expect it to autocomplete one way and it doesn't because something you want now shares first two or three letters with something else in your path. That's where FIGNORE comes in.
Using commas in filenames feels kind of weird to me, but I do use a comma as the initiator for my Bash key sequences. For example: ,, expands to $ ,h expands to --help ,v expands to --version ,s prefixes sudo
You put keyseqs in ~/.inputc, set a keyseq-timeout, and it just works.
You could also do this sort of thing with XCompose, yes?
would an alias just work in this use-case?
Global aliases are a zsh feature and not avaliable in bash. So if you want:
openssl ,v openssl --versionThen again most of the examples OP gave are usually available as short options, and aliasing ,s to sudo is certainly possible. So the only one which makes sense to me is ,,=$. But it's probably not worth the trouble to my muscle memory.
> most of the examples OP gave are usually available as short options
Yes, but a lot of commands behave differently for -h and --help.
> aliasing ,s to sudo is certainly possible
Sure, but my ,s key sequence doesn’t just expand to sudo. It actually moves the cursor to the start of the current line, prefixes the command with sudo, and then moves the cursor to the end of the line. The idea is when you type a command which requires root privileges but forget to use sudo, you can just hit ctrl+p ,s to fetch the previous command and prefix it with sudo. This is what it looks like in ~/.inputrc: ",s":"^Asudo ^E"
also. did you mean .inputrc ?
Yes, I meant ~/.inputrc .
This is one of those ideas that is so simple and elegant that it makes you think “why did I never think of doing this?!”
Neat trick! I don’t think I’ll namespace everything this way, because there’s some aliases and commands I run so often that the comma would get annoying, but for other less frequently used helper scripts then this will be perfect!
I do something similar with build trees, naming them +build, +cross-arm etc.
This convention was suggested by the GNU Arch version control system years ago (maybe 20??), but it's really useful for the same tab completion reason and I have kept it for almost two decades, even when I switched to git.
It was suggested by Tom Lord (RIP), who used it heavily long before he wrote GNU Arch.
File names or directories starting with a comma where considered “junk”, and ones with a plus sign I think where considered “precious”.
Maybe then try ending your commands with a comma so that you don’t break first-char autocomplete !
But that's the killer feature for me! I always forget the little commands I've written over the years, whereas a leading comma will easily let me list them.
This has been a popular topic nearly every time the post makes the HN front page.
* https://news.ycombinator.com/item?id=40769362 (2024, 169 comments)
* https://news.ycombinator.com/item?id=31846902 (2022, 123 comments)
* https://news.ycombinator.com/item?id=22778988 (2020, 90 comments)
hn is at it's best when discussing bash shenanigans
There’s this program on nix that lets you type a comma, then any application name that exists anywhere in the Nix repos. It then downloads that app and runs it once, without “installing” it. Sometimes I find myself running something dozens of times this way before I realize it should probably be in my config.
I'm not sure I'll ever understand why they replaced their working ~50 line shell script with a Rust program that just shells out to the same nix-* commands. I appreciate that there are some safety benefits, but that program is just not complex enough to benefit.
Because it's "a proper language" [1]. Not to mention webscale!
It does seem to do some more complex stuff now that would've been annoying, but not impossible, to write as a shell script.
Shell is already memory safe, so there's not even "we replaced C" to lean on.
Maybe it was fun?
I use a different prefix character, e.g. "[", but I have been doing this for years
I started using a prefix because I like very short script names that are easy to type
I prefer giving scripts numbers instead of names
Something like "[number"
I use prefixes and suffixes to group related scripts together, e.g., scripts that run other scripts
I have an executable directory like ~/bin but it's not called bin. It contains 100s of short scripts
I have zero problems with the terseness of the k language, the names of the source files nor the source code they contain
I find brevity easier to work with. I wish all software was like that
I like the shell (ash not bash). I like assembly language
I have to "adjust" to verbosity, and sometimes I honestly can't, it's paralyzing to the brain, terseness feels more natural
Why not name scripts in natural language like an LLM prompt perhaps (I don't use LLMs so pardon the ignorance), with spaces and punctuation
Bash allows it
echo echo hello > "dear computer, please output the word \"hello\". thank you"
chmod +x "dear computer, please output the word \"hello\". thank you"
"dear computer, please output the word \"hello\". thank you"
But neither of those things is true. The scripts are for communicating with a computer and are intended to be used only by me
UNIX allows anyone to rename any file to whatever they want. The UNIX user is free to pursue their own preferences in naming, whatever those may be
> I prefer giving scripts numbers instead of names
> Something like "[number"
> It contains 100s of short scripts
So you call scripts like [1 [2 [3 [4 ... and remember what each one of them does? If yes - that's nuts, I'd visit a doctor.
Thank you
"..., I'd visit a doctor"
But I'm not you
do you publish dotfiles and scripts anywhere? I'm interested to see them
This is utterly unhinged. I freaking love it. It reminds me of the old joke about prisoners and numbers for jokes (Stanislaw Lem has a similar concept in a book):
A prisoner, new to a particular cell block, was surprised to discover that his fellow inmates passed much of their day by calling out numbers, after which they would laugh heartily for a few moments. Every few minutes an inmate would call out a number and everyone would laugh, and then, after a few moments of silence, someone else would call out a number, and once again laughter. The inmate asked one of the other inmates whom he'd come to know to explain this strange behavior to him.
"It's simple", came the reply. "We know all of our jokes by heart, and there's really no reason to tell them at lenght. Instead, we simply call them out by number."
Though this was strange to him, the new inmate thought he'd join in on the fun. After a few weeks listening to the jokes, he took some initiative and called out "number 27!". But nobody laughed. This seemed very strange to him, since he'd heard others call out that same number, with everyone laughing afterwards. After waiting and waiting, with still no laughter, he finally asked: "why is it that when others call out that joke you laugh, and when I called it, nobody laughed?".
The reply promptly came: "You told it wrong".
I know another version, where 27 was a joke about the guards and then they come and beat him
Why so many people use ~/bin/? What’s wrong with ~/.local/bin?
Random things are installed in ~/.local/bin. In ~/bin I have only what I put there.
Python and rust (for example) package managers install user wide tools there.
People tend to want some separation between what's theirs and what's others. Other programs/scripts quite often put something into ~/.local/bin, so it's not yours actually, it's theirs.
I personally use both, each for different purposes.
I snapshot my entire home directory every hour (using btrfs+snapper), but I exclude ~/.local/ from the snapshots. So I use ~/.local/bin/ for third-party binaries, since there's no reason to back those up; and ~/bin/ for scripts that I wrote myself, since I definitely want to back those up.
This is a pretty idiosyncratic use though, so I'd be surprised if many other people treated both directories this way.
I use ~/.local/bin for installed programs, and ~/bin for my own scripts.
I prefer ~/bin/ for my scripts, links to specific commands, etc.
~/.local/bin is tedious to write, when I want to see directory content and - most important - I treat whole ~/.local/ as managed automatically by other services and volatile.
The latter is XDG.
~/bin predates it.
And of course you can use both.
Personally I use ~/opt//bin where ~/opt is a ‘one stop shop’ containing various things, including a symlink to ~/local and directories or symlinks for things that don't play well with others (e.g. cargo, go), and an ~/opt/prefer/bin that goes at the start of PATH containing symlinks to resolve naming conflicts.
(Anything that modifies standard behaviour is not in PATH, but instead a shell function present only in interactive shells, so as not to break scripts.)
Unix lore: Early unix had two-letter names for most common names to make them easy to type on crappy terminals, but no one* letter command names because the easier were reserved for personal use.
What's the difference between opt and local?
I thought was for mixin externally provided systems like Homebrew, local is for machine or org-level customizations, and ~ is for user-level customizations.
/opt showed up as a place for packaged software, where each package (directory) has its own bin/, lib/, man/, and so on, to keep it self-contained rather than installing its files in the main hierarchy. ~/opt is just a per-user equivalent, analogous to /usr/local vs ~/.local.
The advantage of /opt is that multi-file software stays together. The disadvantage is that PATHs get long.
~/bin/ preceeds the XDG Base Directory Specification.
~/.local was only invented around 2003 and gained widespread usage maybe 15 years or so ago...
People used ~/bin already in the 90s ;-)
Why would you want to store your binaries in a hidden directory?
It kind of goes against the idea why dotfiles are dot-prefixed.
Nothing. I also use `~/.local/bin/`
I actually have the below clashes function in my bashrc to detect overloaded names, and a few compgen aliases for sorted formatted command lists. Clashes takes a few seconds to chew through the 3000ish commands on my Steam Deck.
I also discovered the comma thing myself, and use it for `xdg-open <web-search-url>` aliases, i.e. `,aw xdg` searches archwiki for "xdg", since the SteamOS on this thing doesn't have manpages.
alias words='iftty xargs' # join by spaces
alias c='compgen -c | sort -u | words' # eg: c | grep ^k | chop
# ... a for aliases, f for functions etc ...
alias man='g !archman' # https://man.archlinux.org
alias ,aw='g !aw' # https://wiki.archlinux.org
alias ,mdn='g !mdn' # https://developer.mozilla.org
alias ,nix='g !nixpkgs' # https://search.nixos.org
# ... a dozen others ...
g() { local IFS=+; xdg-open "https://ddg.gg/?q=$*"; }
iftty() { if [ -t 1 ]; then "$@"; else cat; fi; }
chop() { # [TABS] [STARTCOL] [CUTOPTS..] # eg. ls ~/.co* | chop
set -- "${1:-2}" "${2:-1}" "${@:3}"
cut -c "$2"-$(($2+8*$1-2)) "${@:3}" | column #| less
}
clashes() { # [CMDS..] # print overloads eg: clashes $(c)
(($#)) || set -- $(compgen -A alias -A function)
local ty cmd; for cmd; do ty=($(type -at "$cmd"))
((${#ty[@]}>1)) && echo -n "$cmd "; done; echo
}Clever hack! <3 I also do namespacing yet in a different way.
I create a home directory "x" for executables that I want to manage as files, and don't want on PATH or as alias.
To run foo: ~/x/foo
For example I have GNU date as ~/x/date so it's independent of the system BSD date.
The comma has an unusual but super handy usage in PowerShell. When it precedes a variable of any array (list) type, the code it's the input to handles it as an array object, defending against the usual tendency to "unpack" the array and read in the elements that compose it.
It’s clever, but is not aesthetic. A comma feels unnatural in the fs.
So did the dot in dotfiles originally. You’ll get used to it if you want to.
It doesn't have to be a literal file, it can be an alias.
That doesn't make it "feel" less "unnatural".
Nice although I think the ASCII comma feels wrong as part of a filename even if for purely aesthetic reasons.
If we want to stay within (lowercase) alphabetic Latin characters I think prefixing with the least common letters or bigrams that start a word (x, q, y, z, j) is best.
`y' for instance only autocompletes to `yes' and `ypdomainname' on my path.
Choosing a unique bigram is actually quite easy and a fun exercise.
And we can always use uppercase Latin letters since commands very rarely use never mind start with those.
Its some what natural to german spkrs who use a special set of double quotes to start a quote in print.
I do something similar but with a project prefix. I maintain a lot of packaging scripts and settled on prefixing them by project - gps-build, gps-deploy, gps-sync etc. Tab completion does the rest.
The real issue isn't collisions though, it's discoverability. Six months later you forget half of what you wrote. The prefix at least lets you type gps-<tab> and see everything at a glance.
Finally a post that is relevant to what I have been looking for quite some time.
Also, kudos to keeping it so concise and to the point, thats some prime writing.
,Start all of your commands with a comma
Should be titled Prefix your script names with a comma. Current title is a little clickbait-y through its ambiguity.
Agree.
I thought the title meant I should type ,ls instead of ls.
,sudo make me a sammich
Like so?
Love this thread reminding me that most experts in computers are unhinged lunatics doing convoluted and tortured things on a daily basis that they consider perfectly normal and even convenient.
As a non-native English speaker I just name them in my native language or using British English spelling.
I have a command named "decolour", which strips (most) ANSI escape codes. Clear as day what it does, almost nobody uses this spelling when naming commands that later land as part of a distribution.
instead of using ~/bin I use ~/installed/bin, sometimes I need build a command from source then install it, which might have share/ man/ etc so I can avoid installing them under the home dir.
~/.local/ could work for this, and as another user mentioned easier to separate the back up.
This is one of those "obvious in hindsight" tricks. The comma prefix gives you a namespace that's guaranteed to never collide with system binaries, shell builtins, or anything from a package manager.
I do something similar with my personal scripts — prefix them with a short namespace. The real win isn't just avoiding collisions though, it's tab completion. Type the prefix and tab, and you immediately see all your custom stuff without wading through hundreds of system commands.
The 2009 date on this is wild. Some of these simple unix conventions age better than most frameworks.
Properly manage PATH for the context you're in and this is a non-issue. This is the solution used by most programming environments these days, you don't carry around the entire npm or PyPI ecosystem all the time, only when you activate it.
Then again, I don't really believe in performing complex operations manually and directly from a shell, so I don't really understand the use-case for having many small utilities in PATH to begin with.
Comment was deleted :(
One could set an env var to their local bin dir which is otherwise not in the path, like L=/home/ahepp/.local/bin, and then do $L/mycommand. Doesn't meet the OP's requirement of no shift key.
Or prefix files in the local bin dir with a couple letters from your username, like /home/ahepp/.local/bin/ah-mycommand
A lot of vim users use comma as the leader key so this is probably natural for a lot of folks.
Also seems convenient to be able to type ,<tab> to autocomplete over your custom commands, in case you forgot the name you assigned them.
I prefer all my custom commands as 1 letter.
On my most frequently used machine/dev env this means -
e for vim
m for mise
n for pnpm
c for Claude
x for codex
r for uv run
j for just
I use fish abbreviations for this, as they expand to the full command in the shell history.
d for deploy to production
f for friday
Interesting, though I never had enough custom scripts to justify this, I prefer oh-my-zsh plugin style short aliases instead, i.e. https://github.com/ohmyzsh/ohmyzsh/tree/master/plugins/git
I tried a variant or this idea so many years ago after I leaned git and rearranged some of my personal tools as subcommands (like git) of a single executable named "dude,"
It went weird pretty quickly...
dude, whois my.car.. examples?
I don’t think this is a terrible idea, though stylistically it bothers me. I suppose you could simply have a prefix command router that would essentially do the same thing. I also started using “task” recently and it’s been a game changer for my CLI life.
What is task?
It is like make but designed specifically for the way non-C(++) users - people like me for example adding scripts like "make run" and "make build" to my node/python/PHP/etc repos - use it. It is great! I still don't use it literally just because make is already installed on any *nix system I encounter day to day.
Interesting, I have never compared make with task but I suppose there’s some overlap. My favorite feature is that it’s cross-platform. I do use it for performing complex builds (like chaining several environment setup and docker compose commands, etc.). Of course you could do this with shell scripts, but this adds a layer of abstraction.
I used task previously and now use mise for it since I have a mise version file usually anyway.
I never did this and never once experienced a name conflict.
This is a really good practical step if you worry about name collisions
quick, easy and consistent. entirely voluntary.
Bravo
In many contexts in which I am trying to deconflict namespaces, I use my initials. I hadn't thought about it in this particular context, though now that I do, it seems fortunate that I am ced rather than sed.
Alternative, you can check whether there are duplicates under $PATH resolutions - binaries that can come from more than one place in $PATH.
#!/usr/bin/env -S cargo eval --
// cargo-deps: derive-error="0", is_executable="0"
// vi: ft=rust
use derive_error::Error;
use is_executable::IsExecutable;
use std::collections::{BTreeMap, HashSet, btree_map};
use std::os::unix::fs::MetadataExt;
use std::path::{Path, PathBuf};
#[derive(Debug, Error)]
pub enum Error {
VarEnv(std::env::VarError),
Io(std::io::Error),
}
fn main() -> Result<(), Error> {
let mut paths_seen = HashSet::new();
let mut path_fileid_seen = HashSet::new();
let mut exes_seen = BTreeMap::new();
for path in std::env::var("PATH")?.split(":") {
let path = PathBuf::from(path);
if !path.is_dir() {
continue;
}
if !path_fileid_seen.insert(path.metadata()?.ino()) {
continue;
}
if paths_seen.insert(path.clone()) {
match std::fs::read_dir(&path) {
Err(_) => {
eprintln!("error listing {:?}", path);
}
Ok(readdir) => {
for entry in readdir {
let entry = entry?;
if entry.path().is_executable() {
let mut item = match exes_seen.entry(
String::from(entry.path().file_name()
.unwrap().to_string_lossy().as_ref()))
{
btree_map::Entry::Vacant(v) => v.insert(vec![]),
btree_map::Entry::Occupied(o) => o.into_mut(),
};
item.push(path.clone());
}
}
}
}
}
}
for (exe, paths) in exes_seen.into_iter() {
if paths.len() == 2 {
let mut strs = vec![];
for path in paths.iter() {
strs.push(path.to_str().unwrap());
}
if strs.as_slice() == ["/usr/bin", "/usr/sbin"] {
continue;
}
}
if paths.len() > 1 {
println!("{}: ", exe);
for path in paths {
println!(" {}", path.to_str().unwrap());
}
println!("");
}
}
Ok(())
}I think its a fairly good idea - but for myself, i had already mapped csh’s default history character (!) to a comma (,) for the same reason - no shift key to invoke.
csh? I used the enhanced tcsh on Irix in the 90s.
I do this, and it's a huge quality of life improvement. No so much because of shadowing existing binaries, but for better command auto-complete. For example: I have a bunch of tmux utilities and all start with `,t` which is not a polluted command-name prefix compared to just `t`.
But I'm now facing the problem that LLM agents don't like this, and when I instruct them to run certain tools, they remove the leading comma. It's normally fixed with one extra sentence in the prompt, but still inconvenient.
I would have never thought of that. Funny that a comma can be used like that.
Off-topic: What the hell is that font on this website? And why does the "a" look like that?
Gentium is a very beautiful font: https://software.sil.org/gentium/
Why not just start every alias or script with ë or something?
Comma is easier to type and less visually distracting
makes sense, article mentions not wanting composite keys. There goes my sarcasm.
However, I'd advocate for an ë key then.
can someone explain security consideration of placing scripts into $HOME? Some time ago I moved all my scripts to /usr/local/bin, because I feel that this is better from security perspective.
There are no security implications, on the contrary.
It is objectively cleaner to keep your user scripts in your home, that way they are only in _your_ PATH, whereas putting them in /usr/[local/]bin implicitly adds them to every [service] user on the machine, which I can see creating obscure undesired effets.
Not even mentioning the potential issues with packages that could override your scripts at install, unexpected shadowing of service binaries, setuid security implications, etc.
Someone with access to your home dir can also set your $PATH and aliases to anything they want, so I don’t see any extra security considerations here.
(2009)
Previous discussions:
2024: https://news.ycombinator.com/item?id=40769362
similarly, I start all my underscorends with an underscore
in vim i am very fond of two mapping prefixes: <space> and ,
they're both so easy to reach for
> Like many Unix users, I long ago created a ~/bin/ directory in my home directory
`.local/bin` seems to be much more common in my experience for this use case. And for good reason.
~/bin is actually created per default on OpenSUSE (though it's removal has been discussed several times).
Unclutter your $HOME!
yes this is totally cursed but I kinda love the idea
This is just brilliant. Thanks.
I appreciate the idea, but the comma just looks horrible to me as part of a filename. I can imagine someone unfamiliar with the naming scheme to get confused.
I'd prefer to use underscore (when writing BASH scripts, I name all my local variables starting with underscore), but a simple two or three letter prefix would also work. I don't like the idea of a punctuation prefix as punctuation usually has a specific meaning somewhere and including it as the first character in a filename looks wrong. (e.g. Comma is typically used as a list separator and it's a bit of cognitive dissonance to see it not used in that context)
I use my_ as a prefix.
That’s a more meaningful prefix than “,” at the expense of a couple more key strokes. I consider that to still be a win in the book of tab completions.
I would replace underscore with “-“ or “.”
I'd warn against creating files starting with "-" as that can lead to unexpected results with tools if you forget to use "--" to end options. Nothing wrong with using "my-" as a prefix though.
I used to use "do" as a prefix e.g. "doBackup"
Nowadays, I tend to skip using a personal prefix and just try to name commands with a suitable verb in front (e.g. "backupMySQL") and ensure that there's no name collisions.
Whenever I see "my" as a prefix, it feels like such a childish "my first Sony" thing. I hate official sites using that.
This was actually the same feeling I had when I tried to learn perl. I just had a visceral dislike for "my" as the keyword to declare a local variable.
Underscore requires pressing Shift, however.
> I don't like the idea of a punctuation prefix as punctuation usually has a specific meaning somewhere and including it as the first character in a filename looks wrong.
So you don’t use dotfiles? ;)
On non-English keyboards (Serbian/Croatian/Slovenian, but as they are based on QWERTZ, I imagine German and possibly others too), both "+" and "-" might not require pressing Shift either, and are much better characters than comma.
These are inconvenient for doing anything with the script files except invoking them, because these characters introduce command-line options.
Which was the point here, wasn't it? Script files that you will be commonly running and only editing rarely, I'd optimize for how easy they are to run, not operate other commands on them from within a shell.
Naming a file with a "-" as the first character means you have to be careful to use "--" with commands to signify the end of options as otherwise the filename will be interpreted as being additional options with unexpected results.
e.g. ls -l -- *
Even when you're not deliberately operating on the commands, it's too easy to get caught out by it with wildcards etc.
Sure, but this would be contained to when your CWD is ~/bin or ~/.local/bin (or wherever you put your "-" scripts in).
Yes, but it doesn't make it a good idea to leave potential traps lying around even if they're in a different room.
Well dotfiles demonstrate that punctuation can have a special meaning in filenames.
I'm not convinced by "quicker to type" arguments as that's rarely the bottleneck, so I'm perfectly happy with using underscores in filenames and variables. I wouldn't use underscore as the beginning character of a filename unless it had a specific meaning to me (e.g. temporary files), so I'd be more inclined to use a two or three character prefix instead.
For me it’s not about quickness, but about strain. Like in RSI.
I read this blog a few years ago, and implemented it soon after with a refresh of my rc files and shortcuts. Gamechanger - has helped me every single day since. It’s easy to remember, autocompletes easily, and adds a little flair of personalization.
Comment was deleted :(
Comment was deleted :(
[dead]
Crafted by Rajat
Source Code