I'm having problems passing properly in bash parameter values that include spaces and/or newlines from wsl to a program on the win10 host.
Well. it's a generic question, but it comes out of a concrete use-case.
The concrete use-case is using the host-level git binary, namely git.exe from within the wsl.
I know there are rather popular solutions like wslgit, that lets you call from windows the git binary in the wsl. However, this does not only implies that your wsl would lunch for every git command if it was not already running, but it incurs a gotcha-full setup and a questionably optimizable performance footprint.
My wsl is not always on - many of my tasks do not require it, and I don't want to keep it running for git - so my options are the opposite: use from the wsl the git.exe, which is always present.
Anyway, I believe this represents a more generic case and can be relevant to any host-level program called from wsl.
So, in the concrete example of git - I needed a shim that will pass the call to the git.exe whenever the git command is called from within any of the shells of the WSL.
This can be done by replacing /usr/bin/git with a shim that will wire the call to git.exe.
So. The 1st naive implementation was:
#!/usr/bin/bash
/c/Program\ Files/Git/cmd/git.exe $@
The naive implementation worked almost well - up to the point where argument contained whitespaces and/or quotes (e.g. git commit -m "a human comment with \"important\" hints"). Then it went bananas.
So the next step was using "$@" instead of just $@, and nothing worked because git.exe hears "commit" instead of commit, and refuses to cooperate.
So I tried to skip wrapping the 1st arg. Now git stash list did not know what "list" means, because it expects list.
So I went around all the other forms for no avail.
I even tried going in a loop and wrap with quote marks only arguments that contain spaces (and escape any quote marks the user provided intentionally as part of her value) - so, yea - a rabbit hole - and could not get a setup that is working.
After not cracking it for days I admitted defeat - I just don't understand bash well enough to crack this one (or wsl-win-bash relations), so I fell back to the following solution - which works for me, but is less than ideal:
#!/usr/bin/env node
require('child_process').spawn(
`git.exe`,
process.argv.slice(2),
{ stdio: [0, 1, 2 ] },
);
The problem is that this solution is dependent on implementation details of MY workspace: nodejs. :P
I need your help to take it to the next level.
It would be better if I knew of a generic solution that relays on sh/bash, or whatever is available out-of-the-box on any linux available for WSL.
CodePudding user response：
That's the answer:
first, just for backup:
sudo mv /usr/bin/git /usr/bin/git_linux
then, create /usr/bin/git instead as:
#!/bin/bash
cmd.exe /c @ git.exe "$@"
Make sure it's runnable - you might need to chmod x it, I don't remember if I had to...
All hail Bender the Greatest, who pointed out in comments the following link:
CodePudding user response：
I'm also going to post this as an answer. I'm unable to reproduce this issue, calling a Windows-installed git.exe binary from WSL with both spaces and escaped quotes from bash within WSL seems to work fine, provided you escape the requires special characters in the binary path either with a backslash \ or double-quote ":
cd /mnt/c/some/working/gitrepo
# With backslashes in the binary path
/mnt/c/Program\ Files/Git/bin/git.exe commit -m "This commit has some spaces \"and quotes\""
# Quoting the binary path due to spaces
"/mnt/c/Program Files/Git/bin/git.exe" commit -m "This commit has some spaces \"and quotes\""
Adding /mnt/c/Program Files/Git/bin/ to the PATH variable in bash should also work so you don't have to utilize a fully-qualified name to the binary.
I did not need to create a shim or do anything else special other than make sure the path to the git.exe binary was escaped properly.
Note: This is a guess and not a claim (I will remove this if disproven) but my suspicion here is that OP is using a Git binary for Windows that just isn't as compatible with direct invocation as the version I'm using. It may not even be specific to a Git version, but resultant of how it was built for a Windows target in the first place. Read on for how I arrived at this hypothesis.
Some background: This issue does not affect all exe files run from WSL. According to this Microsoft/WSL issue, this isn't so much a "bug" as it is due to the way WSL/bash ships arguments to Windows executables, coupled with how certain programs actually process command line input. This solution won't work for all binaries, and OP's answer may not work for all binaries either. This sort of inconsistency is unfortunate, but not uncommon, when working with two runtimes in tandem.
The linked issue seems to call for documentation to be added/updated so developers and operators can better understand how WSL actually passes arguments to native Windows' executables.
CodePudding user response：
Alternative that works for me instead of a shim:
sudo mv /usr/bin/git /usr/bin/git_linux
sudo ln -s $(command -v git.exe) /usr/bin/git
This assumes that git.exe is in the path when creating the link (which it should be, given defaults). If it's not, just replace the $(command -v git.exe) with the fully qualified path to git.exe.
Passing in strings with escaped quotes and spaces works this way as well.
CodePudding user response：
On Unix, the system call that starts a program requires, and passes, what humans understand as the command line options and arguments as individual strings: if a unix command line is git commit -m "Initial commit", the argument strings the shell passes to the Unix execve system call, and received by the started program, will be git, commit, -m and Initial commit.
On Windows, the system call that starts a program requires, and passes, a single string argument: the undigested command line. Microsoft's runtime supplies a default utility often linked in to built programs that scans that command line to break it out into separate arguments, which are then passed in to whatever equivalent of main the programming language has, and that's why command lines on Windows often follow those quoting and escape handling conventions.
So: when you use a Unix shell to start a program, Windows or not, the shell will scan the command line into the program name and its arguments, after processing any program-specific variable assignments and I/O rewiring; it will then hunt down the program file that runs that program and issue the execve system call to run it with the scanned arguments and environment.
But if the invoked program's loader is for a Windows program, it knows the target is not set up to receive parsed argument strings: it's expecting an undigested command line. So the windows program loader in the unix-to-windows compatibility layer has to reconstruct a command line the windows program will parse into the provided arguments.
And that's where you're getting bit: you (eventually) did
/c/Program\ Files/Git/cmd/git.exe "$@"
and bash passed the program path followed by the arguments you gave as the strings commit, -m and a human comment with "important" hints. Since that .exe is a windows program, the Windows program loader fires it up, and has to reconstruct a command line that the receiving program will scan to identify those same arguments . . . without any way to know how it will do that.
It seems the shim does rely on programs using the double-quoted string convention, but does not rely on programs processing internal escapes, so it doesn't add them to the quotes around "important" when reconstructing the command line.
You have to do it. Since you're using bash, that means getting bash to do it for you.
You can probably use
/c/Program\ Files/Git/cmd/git.exe commit -m 'A human comment with \"important\" hints'
but you'll have to try it to be sure. That command line is parsed by the shell into /c/Program Files/Git/cmd/git.exe, commit, -m and A human comment with \"important\" hints after all quote processing; if the windows program loader shim Microsoft supplies just pastes them back together slapping double quotes around anything with embedded white space, that'll work.
If the shim doesn't even do that much, use
/c/Program\ Files/Git/cmd/git.exe commit -m '"A human comment with \"important\" hints"'
instead.
Then, of course, you have to somehow convince bash to apply the proper transformations to any arguments you want to pass along. For the default scanner, which I believe git.exe uses, you could
/c/Program\ Files/Git/cmd/git.exe "${@//\"/\"\"}"
to get the double quotes escaped in a way it'll recognize.