First-time user observations and error

[paulmelis]

First up, great that someone's working on making Julia use on HPC systems easier!

As I'm now trying JUHPC for the first time here's some things I noticed/ran into. Background: viz guy at Dutch HPC site, so consider me an "internal user" of our system :)

First, there's a lot of output coming not just from JUHPC, but also some of the programs it calls itself. This is a bit confusing, as it's not immediately clear what to pay attention to and what can be ignored. Example below, apparently stuff gets installed in a tempdir, which resolves to /dev/shm on our system, which is fairly non-standard, but okay. The thing I wasn't clear on was if "You can uninstall at any time with juliaup self uninstall and these changes will be reverted" was something I needed to remember or not. It's only at the end when it reads "cleaning up temporary juliaup installation in /dev/shm/paulm/juliaup" that I figured all the /dev/shm stuff was not relevant for the end result. Perhaps putting all that output in a log file would be better, as it hides unnecessary details that aren't of interest to the average user.

...
info: downloading installer
Welcome to Julia!

This will download and install the official Julia Language distribution
and its version manager Juliaup.

Juliaup will be installed into the Juliaup home directory, located at:

  /dev/shm/paulm/juliaup

The julia, juliaup and other commands will be added to
Juliaup's bin directory, located at:

  /dev/shm/paulm/juliaup/bin

You can uninstall at any time with juliaup self uninstall and these
changes will be reverted.
...
[ JUHPC: cleaning up temporary juliaup installation in /dev/shm/paulm/juliaup.
[ JUHPC: ... The installation of the HPC setup for juliaup, julia and HPC key packages is complete.

For the lines below a similar thing applies, as I assume the "re-start Julia" instruction really is just a detail that isn't relevant for JUHPC installation.

Precompiling project...
  75 dependencies successfully precompiled in 322 seconds. 23 already precompiled.
[ Info: Configure the active project to use CUDA 12.6 from the local system; please re-start Julia for this to take effect.

After then trying to activate the installed JUHPC environment it seems the first steps are okay:

snellius paulm@tcn965 14:30 ~/c/JUHPC$ module list

Currently Loaded Modules:
  1) 2024                           5) GCC/13.3.0                      9) libpciaccess/0.18.1-GCCcore-13.3.0  13) UCX/1.16.0-GCCcore-13.3.0        17) UCC/1.3.0-GCCcore-13.3.0
  2) GCCcore/13.3.0                 6) numactl/2.0.18-GCCcore-13.3.0  10) hwloc/2.10.0-GCCcore-13.3.0         14) libfabric/1.21.0-GCCcore-13.3.0  18) OpenMPI/5.0.3-GCC-13.3.0
  3) zlib/1.3.1-GCCcore-13.3.0      7) XZ/5.4.5-GCCcore-13.3.0        11) OpenSSL/3                           15) PMIx/5.0.2-GCCcore-13.3.0        19) gompi/2024a
  4) binutils/2.42-GCCcore-13.3.0   8) libxml2/2.12.7-GCCcore-13.3.0  12) libevent/2.1.12-GCCcore-13.3.0      16) PRRTE/3.0.5-GCCcore-13.3.0       20) CUDA/12.6.0

snellius paulm@tcn965 14:30 ~/c/JUHPC$ source /scratch-shared/paulm/juhpc_setup_genoa/activate 
[ JUHPC: Activating HPC setup for juliaup, julia and HPC key packages requiring system libraries...
[ JUHPC: PATH: /scratch-shared/paulm/juhpc_setup_genoa/juliaup_wrapper:/scratch-shared/paulm/juhpc_setup_genoa/bin:/sw/arch/RHEL9/EB_production/2024/software/CUDA/12.6.0/nvvm/bin:/sw/arch/RHEL9/EB_production/2024/software/CUDA/12.6.0/bin:/sw/arch/RHEL9/EB_production/2024/software/OpenMPI/5.0.3-GCC-13.3.0/bin:/sw/arch/RHEL9/EB_production/2024/software/UCC/1.3.0-GCCcore-13.3.0/bin:/sw/arch/RHEL9/EB_production/2024/software/PRRTE/3.0.5-GCCcore-13.3.0/bin:/sw/arch/RHEL9/EB_production/2024/software/PMIx/5.0.2-GCCcore-13.3.0/bin:/sw/arch/RHEL9/EB_production/2024/software/libfabric/1.21.0-GCCcore-13.3.0/bin:/sw/arch/RHEL9/EB_production/2024/software/UCX/1.16.0-GCCcore-13.3.0/bin:/sw/arch/RHEL9/EB_production/2024/software/libevent/2.1.12-GCCcore-13.3.0/bin:/sw/arch/RHEL9/EB_production/2024/software/OpenSSL/3/bin:/sw/arch/RHEL9/EB_production/2024/software/hwloc/2.10.0-GCCcore-13.3.0/sbin:/sw/arch/RHEL9/EB_production/2024/software/hwloc/2.10.0-GCCcore-13.3.0/bin:/sw/arch/RHEL9/EB_production/2024/software/libxml2/2.12.7-GCCcore-13.3.0/bin:/sw/arch/RHEL9/EB_production/2024/software/XZ/5.4.5-GCCcore-13.3.0/bin:/sw/arch/RHEL9/EB_production/2024/software/numactl/2.0.18-GCCcore-13.3.0/bin:/sw/arch/RHEL9/EB_production/2024/software/binutils/2.42-GCCcore-13.3.0/bin:/sw/arch/RHEL9/EB_production/2024/software/GCCcore/13.3.0/bin:/scratch-shared/paulm/juhpc_setup_genoa/juliaup_wrapper:/scratch-shared/paulm/juhpc_setup_genoa/bin:/gpfs/admin/hpc/usertools:/bin:/home/paulm/.local/bin:/home/paulm/software/bob:/home/paulm/bin:/gpfs/admin/hpc/sw/hpc/bin:/gpfs/admin/hpc/sw/hpc/sbin:/opt/neovim/current/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin
[ JUHPC: JULIAUP_DEPOT_PATH: /scratch-shared/paulm/juhpc_setup_genoa/depot
[ JUHPC: JULIA_DEPOT_PATH: /scratch-shared/paulm/juhpc_setup_genoa/depot
[ JUHPC: JULIA_LOAD_PATH: :/scratch-shared/paulm/juhpc_setup_genoa/julia_preferences
[ JUHPC: CUDA_HOME: /sw/arch/RHEL9/EB_production/2024/software/CUDA/12.6.0
[ JUHPC: JULIA_CUDA_MEMORY_POOL: none

But not having a julia available is somewhat surprising:

snellius paulm@tcn965 14:31 ~/c/JUHPC$ which julia
/usr/bin/which: no julia in (/scratch-shared/paulm/juhpc_setup_genoa/juliaup_wrapper:/scratch-shared/paulm/juhpc_setup_genoa/bin:/sw/arch/RHEL9/EB_production/2024/software/CUDA/12.6.0/nvvm/bin:/sw/arch/RHEL9/EB_production/2024/software/CUDA/12.6.0/bin:/sw/arch/RHEL9/EB_production/2024/software/OpenMPI/5.0.3-GCC-13.3.0/bin:/sw/arch/RHEL9/EB_production/2024/software/UCC/1.3.0-GCCcore-13.3.0/bin:/sw/arch/RHEL9/EB_production/2024/software/PRRTE/3.0.5-GCCcore-13.3.0/bin:/sw/arch/RHEL9/EB_production/2024/software/PMIx/5.0.2-GCCcore-13.3.0/bin:/sw/arch/RHEL9/EB_production/2024/software/libfabric/1.21.0-GCCcore-13.3.0/bin:/sw/arch/RHEL9/EB_production/2024/software/UCX/1.16.0-GCCcore-13.3.0/bin:/sw/arch/RHEL9/EB_production/2024/software/libevent/2.1.12-GCCcore-13.3.0/bin:/sw/arch/RHEL9/EB_production/2024/software/OpenSSL/3/bin:/sw/arch/RHEL9/EB_production/2024/software/hwloc/2.10.0-GCCcore-13.3.0/sbin:/sw/arch/RHEL9/EB_production/2024/software/hwloc/2.10.0-GCCcore-13.3.0/bin:/sw/arch/RHEL9/EB_production/2024/software/libxml2/2.12.7-GCCcore-13.3.0/bin:/sw/arch/RHEL9/EB_production/2024/software/XZ/5.4.5-GCCcore-13.3.0/bin:/sw/arch/RHEL9/EB_production/2024/software/numactl/2.0.18-GCCcore-13.3.0/bin:/sw/arch/RHEL9/EB_production/2024/software/binutils/2.42-GCCcore-13.3.0/bin:/sw/arch/RHEL9/EB_production/2024/software/GCCcore/13.3.0/bin:/scratch-shared/paulm/juhpc_setup_genoa/juliaup_wrapper:/scratch-shared/paulm/juhpc_setup_genoa/bin:/gpfs/admin/hpc/usertools:/bin:/home/paulm/.local/bin:/home/paulm/software/bob:/home/paulm/bin:/gpfs/admin/hpc/sw/hpc/bin:/gpfs/admin/hpc/sw/hpc/sbin:/opt/neovim/current/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin)

And juliaup is where I would expect it, but running that gives me an error:

snellius paulm@tcn965 14:31 ~/c/JUHPC$ which juliaup
/scratch-shared/paulm/juhpc_setup_genoa/juliaup_wrapper/juliaup

snellius paulm@tcn965 14:31 ~/c/JUHPC$ juliaup


     ???  ???   ???  ???  ??????????  ???????
     ???  ???   ???  ???  ???????????????????
     ???  ???   ???  ???????????????????     
???  ???  ???   ???  ??????????????? ???     
????????  ?????????  ???  ??????     ????????
 ??????    ???????   ???  ??????      ???????


[ JUHPC: This is the first call to juliaup: installing juliaup and the latest julia...

info: downloading installer
Welcome to Julia!

This will download and install the official Julia Language distribution
and its version manager Juliaup.

Juliaup will be installed into the Juliaup home directory, located at:

  /scratch-shared/paulm/juhpc_setup_genoa

The julia, juliaup and other commands will be added to
Juliaup's bin directory, located at:

  /scratch-shared/paulm/juhpc_setup_genoa/bin

You can uninstall at any time with juliaup self uninstall and these
changes will be reverted.

You are trying to install Juliaup into the folder
`/scratch-shared/paulm/juhpc_setup_genoa`,
but that folder already exists. Please remove that folder
and then start the setup process again.
[ JUHPC: ERROR: juliaup installation failed.

The error doesn't make much sense to me, as I assumed juliaup was already installed earlier? Since the installation took quite a few minutes and processed a lot of Julia packages. So perhaps something is going wrong on my end?

So with respect to actually using (as an end-user) the installed JUHPC environment I'm unsure what to expect. For example, one of the example scripts in the repo here shows the line . $JUHPC_SETUP_INSTALLDIR/activate, but I assume that I don't need to keep $JUHPC_SETUP_INSTALLDIR set, as that would defeat the purpose of having separate install directories?

[omlins]

Thanks for sharing this! One could indeed write a lot of information into log file instead of showing it.
All the output that is from JUHPC start with [ JUHPC:

You are trying to install Juliaup into the folder
/scratch-shared/paulm/juhpc_setup_genoa,
but that folder already exists. Please remove that folder
and then start the setup process again.

It looks like you gave the same folder for the setup installation and the juliaup installation. If you change that then it should work.

[omlins]

You can try with VERSION="refs/heads/initial" which will use the file from the pull request. There it will write the output into a log file instead. The output looks then as follows:

[todi][omlins@todi-ln002 ~]$ bash -l /tmp/juhpc $JUHPC_SETUP_INSTALLDIR $JULIAUP_INSTALLDIR


     ██╗  ██╗   ██╗  ██╗  ██╗██████╗  ██████╗
     ██║  ██║   ██║  ██║  ██║██╔══██╗██╔════╝
     ██║  ██║   ██║  ███████║██████╔╝██║     
██╗  ██║  ██║   ██║  ██╔══██║██╔═══╝ ██║     
╚█████╔╝  ╚██████╔╝  ██║  ██║██║     ╚██████╗
 ╚════╝    ╚═════╝   ╚═╝  ╚═╝╚═╝      ╚═════╝


[ JUHPC: Starting installation of HPC setup for juliaup, julia and HPC key packages requiring system libraries...
[ JUHPC: Installing juliaup and julia temporarily in /dev/shm/omlins/juliaup...

info: downloading installer
Installing Julia 1.11.0+0.aarch64.linux.gnu
Configured the default Julia version to be 'release'.
[ JUHPC: ... done: temporary installation completed.
[ JUHPC: Creating preferences for HPC key packages...
[ JUHPC: ... done: preferences created.
[ JUHPC: Creating wrapper for juliaup...
[ JUHPC: ... done.
[ JUHPC: Creating activate script...
[ JUHPC: ... done.
[ JUHPC: cleaning up temporary juliaup installation in /dev/shm/omlins/juliaup.
[ JUHPC: ... The installation of the HPC setup for juliaup, julia and HPC key packages is complete.

To avoid that people get worried because of the time nothing is happening a progress bar should be added...

[omlins]

Note that at this point only the setup is installed. Juliaup and julia will be installed at first call of juliaup. The user who installs the set up does not have to be the same user asked the one who uses it finally... It looks then as follows when using it as in the examples:

omlins@Omlin-Lenovo:~/tmpwdir/juliadev/JUHPC$ ssh todi.cscs.ch

=========================================================================
             IMPORTANT NOTICE FOR USERS of Alps (TODI)

  Documentation: Alps User Guide - https://confluence.cscs.ch/x/_gD0E
  Request support: Service Desk at https://support.cscs.ch
=========================================================================

[todi][omlins@todi-ln001 ~]$ # Variable set in craype_config
[todi][omlins@todi-ln001 ~]$ JUHPC_SETUP_INSTALLDIR=$SCRATCH/${HOSTNAME%%-*}/juhpc_setup  # HPC setup installation environment variables must be expanded during installation.
[todi][omlins@todi-ln001 ~]$ 
[todi][omlins@todi-ln001 ~]$ # Load required modules (including correct CPU and GPU target modules)
[todi][omlins@todi-ln001 ~]$ module load cray

[todi][omlins@todi-ln001 ~]$ module switch PrgEnv-cray PrgEnv-gnu

Lmod is automatically replacing "cce/17.0.0" with "gcc-native/12.3".


Due to MODULEPATH changes, the following have been reloaded:
  1) cray-libsci/23.12.5     2) cray-mpich/8.1.28

[todi][omlins@todi-ln001 ~]$ module load cudatoolkit craype-accel-nvidia90
[todi][omlins@todi-ln001 ~]$ module load cray-hdf5-parallel
[todi][omlins@todi-ln001 ~]$ module list

Currently Loaded Modules:
  1) craype-arm-grace     4) xpmem/2.9.6-1.1_20240511212657__g087dc11fc19d   7) cray/23.12       10) cray-mpich/8.1.28    13) cudatoolkit/24.3_12.3
  2) libfabric/1.15.2.0   5) perftools-base/23.12.0                          8) gcc-native/12.3  11) cray-libsci/23.12.5  14) craype-accel-nvidia90
  3) craype-network-ofi   6) cpe/23.12                                       9) craype/2.7.30    12) PrgEnv-gnu/8.5.0     15) cray-hdf5-parallel/1.12.2.9

 

[todi][omlins@todi-ln001 ~]$ 
[todi][omlins@todi-ln001 ~]$ # Activate the HPC setup environment variables
[todi][omlins@todi-ln001 ~]$ . $JUHPC_SETUP_INSTALLDIR/activate
[ JUHPC: Activating HPC setup for juliaup, julia and HPC key packages requiring system libraries...
[ JUHPC: PATH: /capstor/scratch/cscs/omlins/todi/juhpc_setup/juliaup_wrapper:/capstor/scratch/cscs/omlins/todi/juliaup/bin:/opt/cray/pe/hdf5-parallel/1.12.2.9/bin:/opt/cray/pe/hdf5/1.12.2.9/bin:/opt/nvidia/hpc_sdk/Linux_aarch64/24.3/cuda/12.3/compute-sanitizer:/opt/nvidia/hpc_sdk/Linux_aarch64/24.3/cuda/12.3/bin:/opt/nvidia/hpc_sdk/Linux_aarch64/24.3/cuda/12.3/libnvvp:/opt/nvidia/hpc_sdk/Linux_aarch64/24.3/profilers/Nsight_Compute:/opt/nvidia/hpc_sdk/Linux_aarch64/24.3/profilers/Nsight_Systems/bin:/opt/cray/pe/mpich/8.1.28/ofi/gnu/12.3/bin:/opt/cray/pe/mpich/8.1.28/bin:/opt/cray/pe/craype/2.7.30/bin:/opt/cray/pe/perftools/23.12.0/bin:/opt/cray/pe/papi/7.0.1.2/bin:/opt/cray/libfabric/1.15.2.0/bin:/capstor/scratch/cscs/omlins/julia_local/julia_latest/bin:/users/omlins/stackinator/bin:/usr/local/bin:/usr/bin:/bin:/usr/lpp/mmfs/bin:/usr/lib/mit/bin:/opt/cray/pe/bin
[ JUHPC: JULIAUP_DEPOT_PATH: /capstor/scratch/cscs/omlins/todi/juliaup/depot
[ JUHPC: JULIA_DEPOT_PATH: /capstor/scratch/cscs/omlins/todi/juliaup/depot
[ JUHPC: JULIA_LOAD_PATH: :/capstor/scratch/cscs/omlins/todi/juhpc_setup/julia_preferences
[ JUHPC: CUDA_HOME: /opt/nvidia/hpc_sdk/Linux_aarch64/24.3/cuda/12.3
[ JUHPC: JULIA_CUDA_MEMORY_POOL: none
[todi][omlins@todi-ln001 ~]$ 
[todi][omlins@todi-ln001 ~]$ # Call juliaup to install juliaup and latest julia on scratch
[todi][omlins@todi-ln001 ~]$ juliaup


     ██╗  ██╗   ██╗  ██╗  ██╗██████╗  ██████╗
     ██║  ██║   ██║  ██║  ██║██╔══██╗██╔════╝
     ██║  ██║   ██║  ███████║██████╔╝██║     
██╗  ██║  ██║   ██║  ██╔══██║██╔═══╝ ██║     
╚█████╔╝  ╚██████╔╝  ██║  ██║██║     ╚██████╗
 ╚════╝    ╚═════╝   ╚═╝  ╚═╝╚═╝      ╚═════╝


[ JUHPC: This is the first call to juliaup: installing juliaup and the latest julia...

info: downloading installer
Welcome to Julia!

This will download and install the official Julia Language distribution
and its version manager Juliaup.

Juliaup will be installed into the Juliaup home directory, located at:

  /capstor/scratch/cscs/omlins/todi/juliaup

The julia, juliaup and other commands will be added to
Juliaup's bin directory, located at:

  /capstor/scratch/cscs/omlins/todi/juliaup/bin

You can uninstall at any time with juliaup self uninstall and these
changes will be reverted.

Now installing Juliaup
Installing Julia 1.11.0+0.aarch64.linux.gnu
Configured the default Julia version to be 'release'.
Julia was successfully installed on your system.
[ JUHPC: ... done: installation completed (location: /capstor/scratch/cscs/omlins/todi/juliaup). You can now use juliaup and julia normally.

[paulmelis]

You can try with VERSION="refs/heads/initial" which will use the file from the pull request. There it will write the output into a log file instead.

Okay, that indeed is a lot cleaner! Suggestion to move the log file to $JUHPC_SETUP_INSTALLDIR after juhpc is done, to have something to reference later, or in case of bugs.

It looks like you gave the same folder for the setup installation and the juliaup installation. If you change that then it should work.

Ah, indeed I messed that up. I also wasn't noticing that installation to $JUHPC_SETUP_INSTALLDIR is more or less a one-time global action, while things going to $JULIAUP_INSTALLDIR are assumed to be per-user? Because if those two dirs are related (and meant to be cleanly used together only), then why aren't they part of the same parent directory? But maybe I'm misunderstanding that part, i.e. if juhpc can be called multiple times with the same $JUHPC_SETUP_INSTALLDIR while varying $JULIAUP_INSTALLDIR.

[omlins]

@paulmelis : I think i have addressed all the issues you raised. Could you please try again using the branch from the PR setting version as before? I also added a key word argument for verbosity. All the changes are documented here: https://github.com/JuliaParallel/JUHPC/tree/initial?tab=readme-ov-file#2-call-juhpc

[paulmelis]

Yep, seems all okay now!

snellius paulm@tcn1174 19:51 ~/c/JUHPC$ m list 
No modules loaded
snellius paulm@tcn1174 19:51 ~/c/JUHPC$ cat snellius-genoa.sh
#!/bin/sh
source ./snellius-genoa.modules

export JUHPC_CUDA_HOME=$EBROOTCUDA
export JUHPC_CUDA_RUNTIME_VERSION=12.6
export JUHPC_MPI_HOME=$EBROOTOPENMPI
export JUHPC_MPI_EXEC=srun
#export HDF5

export JUHPC_SETUP_INSTALLDIR=/scratch-shared/paulm/juhpc_setup_genoa
# XXX need to escape any $ in this value
export JULIAUP_INSTALLDIR=/scratch-shared/paulm/juliaup_genoa

bash -l ./juhpc $JUHPC_SETUP_INSTALLDIR $JULIAUP_INSTALLDIR

snellius paulm@tcn1174 19:51 ~/c/JUHPC$ cat snellius-genoa.modules 
module load 2024
module load gompi/2024a
module load CUDA/12.6.0

snellius paulm@tcn1174 19:51 ~/c/JUHPC$ ./snellius-genoa.sh


     ██╗  ██╗   ██╗  ██╗  ██╗██████╗  ██████╗
     ██║  ██║   ██║  ██║  ██║██╔══██╗██╔════╝
     ██║  ██║   ██║  ███████║██████╔╝██║     
██╗  ██║  ██║   ██║  ██╔══██║██╔═══╝ ██║     
╚█████╔╝  ╚██████╔╝  ██║  ██║██║     ╚██████╗
 ╚════╝    ╚═════╝   ╚═╝  ╚═╝╚═╝      ╚═════╝


[ JUHPC: Starting installation of HPC setup for juliaup, julia and HPC key packages requiring system libraries...
[ JUHPC: Installing juliaup and julia temporarily in /dev/shm/paulm/juliaup...

info: downloading installer
Installing Julia 1.11.1+0.x64.linux.gnu
Configured the default Julia version to be 'release'.
[ JUHPC: ... done: temporary installation completed.
[ JUHPC: Creating preferences for HPC key packages...
|██████████████████████████████████████████████████| 100.00%[ JUHPC: ... done: preferences created.
[ JUHPC: Creating wrapper for juliaup...
[ JUHPC: ... done: wrapper created.
[ JUHPC: Creating activate script...
[ JUHPC: ... done: activate script created.
[ JUHPC: cleaning up temporary juliaup installation in /dev/shm/paulm/juliaup.
[ JUHPC: ... The installation of the HPC setup for juliaup, julia and HPC key packages is complete.

snellius paulm@tcn1174 19:55 ~/c/JUHPC$ source /scratch-shared/paulm/juhpc_setup_genoa/activate 
[ JUHPC: Activating HPC setup for juliaup, julia and HPC key packages requiring system libraries...
[ JUHPC: PATH: /scratch-shared/paulm/juhpc_setup_genoa/juliaup_wrapper:/scratch-shared/paulm/juliaup_genoa/bin:/gpfs/admin/hpc/usertools:/bin:/home/paulm/.local/bin:/home/paulm/software/bob:/gpfs/admin/hpc/sw/hpc/bin:/gpfs/admin/hpc/sw/hpc/sbin:/opt/neovim/current/bin:/gpfs/admin/hpc/usertools:/bin:/home/paulm/.local/bin:/home/paulm/software/bob:/home/paulm/.local/bin:/home/paulm/bin:/gpfs/admin/hpc/sw/hpc/bin:/gpfs/admin/hpc/sw/hpc/sbin:/opt/neovim/current/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin
[ JUHPC: JULIAUP_DEPOT_PATH: /scratch-shared/paulm/juliaup_genoa/depot
[ JUHPC: JULIA_DEPOT_PATH: /scratch-shared/paulm/juliaup_genoa/depot
[ JUHPC: JULIA_LOAD_PATH: :/scratch-shared/paulm/juhpc_setup_genoa/julia_preferences
[ JUHPC: CUDA_HOME: /sw/arch/RHEL9/EB_production/2024/software/CUDA/12.6.0
[ JUHPC: JULIA_CUDA_MEMORY_POOL: none

snellius paulm@tcn1174 19:55 ~/c/JUHPC$ which julia
/usr/bin/which: no julia in (/scratch-shared/paulm/juhpc_setup_genoa/juliaup_wrapper:/scratch-shared/paulm/juliaup_genoa/bin:/gpfs/admin/hpc/usertools:/bin:/home/paulm/.local/bin:/home/paulm/software/bob:/gpfs/admin/hpc/sw/hpc/bin:/gpfs/admin/hpc/sw/hpc/sbin:/opt/neovim/current/bin:/gpfs/admin/hpc/usertools:/bin:/home/paulm/.local/bin:/home/paulm/software/bob:/home/paulm/.local/bin:/home/paulm/bin:/gpfs/admin/hpc/sw/hpc/bin:/gpfs/admin/hpc/sw/hpc/sbin:/opt/neovim/current/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin)
snellius paulm@tcn1174 19:55 ~/c/JUHPC$ which juliaup
/scratch-shared/paulm/juhpc_setup_genoa/juliaup_wrapper/juliaup
snellius paulm@tcn1174 19:55 ~/c/JUHPC$ juliaup


     ██╗  ██╗   ██╗  ██╗  ██╗██████╗  ██████╗
     ██║  ██║   ██║  ██║  ██║██╔══██╗██╔════╝
     ██║  ██║   ██║  ███████║██████╔╝██║     
██╗  ██║  ██║   ██║  ██╔══██║██╔═══╝ ██║     
╚█████╔╝  ╚██████╔╝  ██║  ██║██║     ╚██████╗
 ╚════╝    ╚═════╝   ╚═╝  ╚═╝╚═╝      ╚═════╝


[ JUHPC: This is the first call to juliaup: installing juliaup and the latest julia...

info: downloading installer
Welcome to Julia!

This will download and install the official Julia Language distribution
and its version manager Juliaup.

Juliaup will be installed into the Juliaup home directory, located at:

  /scratch-shared/paulm/juliaup_genoa

The julia, juliaup and other commands will be added to
Juliaup's bin directory, located at:

  /scratch-shared/paulm/juliaup_genoa/bin

You can uninstall at any time with juliaup self uninstall and these
changes will be reverted.

Now installing Juliaup
Installing Julia 1.11.1+0.x64.linux.gnu
Configured the default Julia version to be 'release'.
Julia was successfully installed on your system.
[ JUHPC: ... done: installation completed (location: /scratch-shared/paulm/juliaup_genoa). You can now use juliaup and julia normally.
snellius paulm@tcn1174 19:56 ~/c/JUHPC$ julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.11.1 (2024-10-16)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> 
snellius paulm@tcn1174 19:56 ~/c/JUHPC$ which julia
/scratch-shared/paulm/juliaup_genoa/bin/julia
snellius paulm@tcn1174 19:56 ~/c/JUHPC$ which juliaup
/scratch-shared/paulm/juhpc_setup_genoa/juliaup_wrapper/juliaup
snellius paulm@tcn1174 19:56 ~/c/JUHPC$ juliaup status
 Default  Channel  Version                 Update 
--------------------------------------------------
       *  release  1.11.1+0.x64.linux.gnu         

[omlins]

Great! Could you do a PR to include your configuration and testing in the examples:
https://github.com/JuliaParallel/JUHPC/tree/main/examples
?

[paulmelis]

Sure, I can look into that. One thing I'm still not clear on is this point:

I also wasn't noticing that installation to $JUHPC_SETUP_INSTALLDIR is more or less a one-time global action, while things going to $JULIAUP_INSTALLDIR are assumed to be per-user? Because if those two dirs are related (and meant to be cleanly used together only), then why aren't they part of the same parent directory? But maybe I'm misunderstanding that part, i.e. if juhpc can be called multiple times with the same $JUHPC_SETUP_INSTALLDIR while varying $JULIAUP_INSTALLDIR?

[omlins]

JUHPC_SETUP_INSTALLDIR is meant conceptually for all users or for a group of users (of course in practice it can also be for a single end user)
JULIAUP_INSTALLDIR is is always meant per single user

[omlins]

juhpc can be called multiple times with the same $JUHPC_SETUP_INSTALLDIR while varying $JULIAUP_INSTALLDIR

No it will complain if any of the directories exist already

[omlins]

Sure, I can look into that.

Great!

[paulmelis]

These statements seem to conflict?

JUHPC_SETUP_INSTALLDIR is meant conceptually for all users or for a group of users (of course in practice it can also be for a single end user)
JULIAUP_INSTALLDIR is is always meant per single user

versus

juhpc can be called multiple times with the same $JUHPC_SETUP_INSTALLDIR while varying $JULIAUP_INSTALLDIR

No it will complain if any of the directories exist already

Can or can't JUHPC_SETUP_INSTALLDIR be used multiple times for different JULIAUP_INSTALLDIR's?

[omlins]

These statements seem to conflict?

JUHPC_SETUP_INSTALLDIR is meant conceptually for all users or for a group of users (of course in practice it can also be for a single end user)
JULIAUP_INSTALLDIR is is always meant per single user

versus

juhpc can be called multiple times with the same $JUHPC_SETUP_INSTALLDIR while varying $JULIAUP_INSTALLDIR

No it will complain if any of the directories exist already

Can or can't JUHPC_SETUP_INSTALLDIR be used multiple times for different JULIAUP_INSTALLDIR's?

No, the statements don't conflict. Only one HPC setup can be installed into a JUHPC_SETUP_INSTALLDIR, however, multiple users can use it and it should contain a JULIAUP_INSTALLDIR that expands differently for each user as explained here: https://github.com/JuliaParallel/JUHPC?tab=readme-ov-file#2-call-juhpc

[paulmelis]

No, the statements don't conflict. Only one HPC setup can be installed into a JUHPC_SETUP_INSTALLDIR, however, multiple users can use it and it should contain a JULIAUP_INSTALLDIR that expands differently for each user as explained here: https://github.com/JuliaParallel/JUHPC?tab=readme-ov-file#2-call-juhpc

Then I must be missing information, as it seems juhpc always needs both directories as arguments to run. And since it checks that they don't exist, how can you reuse an existing JUHPC_SETUP_INSTALLDIR? What would the set of juhpc calls be to create different per-user JULIAUP_INSTALLDIR's, while using the same JUHPC_SETUP_INSTALLDIR?