bash support

[dsyme]

Powershell support is great, but millions of people know bash and other unix shells and will expect support for it to be "in the box". Currently you can simulate it via this:

#!pwsh
bash -c "ls ~/.nuget/packages/libtorch-cpu/0.3.52118/runtimes/linux-x64/native"

I'd like this to just be

#!bash
ls ~/.nuget

If you can point me in the right direction I can attempt to add an extension that enables this, at least for Linux. We could also add support for bash on WSL, perhaps separately.

[dsyme]

Note that Python notebooks allow a one line execution using unix shell, e.g.

!ls ~/.nuget

Just to note people will complain about the two-line ceremony needed to get to the shell. We can deal with that question as orthogonal to the basics of getting bash scripting in place

[brettfo]

I'm not sure we can easily support #!bash at the top level, per se; PowerShell support always works because it's included in the tool and works cross platform, and it might not be reasonable to include a full copy of bash that works on any platform (then again, given how friendly MS has been to Linux recently, maybe we can do this).

I am interested, however, in adding some kind of 'native shell' experience that might default to bash on Linux/MacOS (or the user's default shell?) and CMD on Windows. I'm not sure what the language specifier should be; perhaps #!shell? The downside to this is that scripts written this way won't work on other operating systems. E.g.,

#!shell
ls !/.nuget

would only work on Linux/MacOS while

#!shell
dir %HOMEPATH%\.nuget

would only work on Windows.

[jonsequitur]

I think the problem with a "default shell" language is that it won't work cross-platform. Shell-specific magic commands like #!bash or #!zsh would be more stable, but will require pulling in an extension at runtime which I expect would use an already-installed version of the shell rather than installing the shell.

We haven't got a working example yet of adding a new subkernel via a NuGet extension but it should work in theory. I'll get back with an example.

[dsyme]

If you could make it that #!xyz searches for a nuget extension that would be great (maybe you do that already?)

I believe a very short one line syntax is simply necessary to be cognitively competitive with Jupyter+python. People do an awful lot with ! in python data science notebooks - downloading files, munging them, moving them around, etc. Perhaps

#!bash -c echo "hello" 

would be enough but I'm sure you'll get people asking for just ! too. Maybe that can happen somehow.

[dsyme]

As part of this you should likely reserve lines beginning with ! - however I'm not sure if that would caus any problems for any languages, I expect not.

Speaking of which, how do you know what the default language for the cells of a notebook is?

[jonsequitur]

If you could make it that #!xyz searches for a nuget extension that would be great (maybe you do that already?)

Are you suggesting reserving these magic commands and acquiring the shell-specific extensions on the fly via NuGet? Definitely doable.

[jonsequitur]

! would be difficult as a reserved prefix for the same reason that we moved away from %, which is that it introduces ambiguities with the embedded languages. # and #! were already effectively comments in C#, F#, PowerShell, and several other languages.

That said, within .NET Interactive, #!-prefixed magic commands are user-extensible, including the custom command line syntax, so #!bash -c echo "hello" would be straightforward.

The default language for a notebook is mapped to the kernel name, which is why in Jupyter we install three different "kernels". The difference between them is just the command line arguments to dotnet-interactive in the kernelspec, in effect:

> dotnet interactive jupyter --default-kernel fsharp /path/to/connectionfile

[TylerLeonhardt]

I can sympathize with wanting a magic command for running a command in the shell that's available on the system.

@dsyme I'm sure your example is a bit of a simplification of something much larger... but ls is ls in PowerShell.

That is to say

#!pwsh
ls ~/.nuget/packages/libtorch-cpu/0.3.52118/runtimes/linux-x64/native

uses the actual ls executable. Any linux executable on the PATH is available in PowerShell.

But yeah a way to invoke one-off commands in whatever the default shell is would be nice.

The only thing we need to be careful is the feature creep because PowerShell support is more than just a shell/one off commands - you get plotting, and get/set variables from other languages, and other rich features of .NET Interactive.

[SteveL-MSFT]

To expand on what @TylerLeonhardt is saying, pwsh is also just a shell. So if you want to run any native commands (like ls), it should just work. You only need bash if you want to write bash scripting.

[dsyme]

Are you suggesting reserving these magic commands and acquiring the shell-specific extensions on the fly via NuGet? Definitely doable.

Yes, that's the suggestion

That said, within .NET Interactive, #!-prefixed magic commands are user-extensible, including the custom command line syntax, so #!bash -c echo "hello" would be straightforward.

That would be perfect

[tmds]

I am for the first time playing with dotnet interactive notebooks, and I was trying to figure out how I can use bash which lead me here.

I guess when you use #!pwsh that pwsh refers to something known to .NET interactive.

Now consider a unix script:

#!/bin/bash

echo "hello world"

Linux kernel looks at this file and sees the #! at the start. Then it knows to take the rest of the line, and use that to invoke the program and pass the script filename as an additional argument.

.NET interactive could do something similar. When the identifier after #! is not known, it can look if finds an executable that matches the filename, and if so, write out the file and execute it.

@jonsequitur @dsyme @TylerLeonhardt @SteveL-MSFT @brettfo wdyt?

[jonsequitur]

A number of the suggestions in this issue will work, but what's the best approach to making them also work on Windows?

Something we've done for SQL was to add a #!sql breadcrumb:

Could something like this work for #!bash and then on Windows, we could direct people to enable WSL and install a distro if needed?

[tmds]

@jonsequitur part of what I'm suggesting is not to limit this approach to bash identifier, but use the same mechanism like Linux kernel, to also allow other shells to be used.

Could something like this work for #!bash and then on Windows, we could direct people to enable WSL and install a distro if needed?

I'm not familiar with how this stuff works on Windows. afaik bash can be made available in other ways than WSL, like as part of the git installer.

Anyway, I think it makes sense that if some well known programs like bash are missing, the user gets an actionable suggestion.

[tmds]

Maybe there is a desire to make environment variables flow from one bash code section to the next?
That would be technically challenging.