Ability to pass any compiler flags directly to the compiler

[connoraird]

Description

It would be very useful to pass compiler flags from the fpm.toml file directly to the compilation commands. It seems that currently it is possible to pass very specific flags such as -ffree-form but something more generic would be more flexible. For example

name = "my-fortran-project"

[fortran]
source-form = "free"
user-defined-flags = "-ffree-line-length-none"

The new user-defined-flags would then be passed straight through to the compiler and the compiler allowed to catch any errors with the flags.

Possible Solution

get_feature_flags could handle a new feature user_defined_flags

function get_feature_flags(compiler, features) result(flags)
    type(compiler_t), intent(in) :: compiler
    type(fortran_features_t), intent(in) :: features
    character(:), allocatable :: flags

    flags = ""
    if (features%implicit_typing) then
        flags = flags // compiler%get_feature_flag("implicit-typing")
    else
        flags = flags // compiler%get_feature_flag("no-implicit-typing")
    end if

    if (features%implicit_external) then
        flags = flags // compiler%get_feature_flag("implicit-external")
    else
        flags = flags // compiler%get_feature_flag("no-implicit-external")
    end if

    if (allocated(features%source_form)) then
        flags = flags // compiler%get_feature_flag(features%source_form//"-form")
    end if

    if (allocated(features%user_defined_flags)) then
        flags = flags // " " // features%user_defined_flags
    end if
end function get_feature_flags

This would of course require further changes to add user-defined-flags. Could this be added everywhere source-form also is currently?

Additional Information

No response

[perazz]

My understanding of the fortran features structure is to hide compiler details behind the flags, so I'm not 100% sure adding compiler-specific flags is the best approach here.

Regarding max line length, not all compilers support varying it but GNU definitely offers this good option.
So why don't we add it directly to the "get-feature-flags" for the free form? I think that having a generic line length is part of what a free-form source file means for most developers.

Instead of adding one more structure (remember that custom flags are passed via CLI rather than manifest), could we instead just change this:

fpm/src/fpm_compiler.F90

Line 173 in 1cfcaf8

flag_gnu_free_form = " -ffree-form", &

to this?

flag_gnu_free_form = " -ffree-form -ffree-line-length-none -ffixed-line-length-none", & 

[connoraird]

Thank you for your speedy comments!

My understanding of the fortran features structure is to hide compiler details behind the flags, so I'm not 100% sure adding compiler-specific flags is the best approach here.

This is intended to not be specific to any compiler and the responsibility then is passed to the "user". But I understand that hiding compiler specifics is a design choice and therefore shouldn't be changed lightly. However, I can still see the benefit of having a way to specify any compiler flags via the manifest, rather than requiring --flag to be passed in every command.

Regarding max line length, not all compilers support varying it but GNU definitely offers this good option. So why don't we add it directly to the "get-feature-flags" for the free form? I think that having a generic line length is part of what a free-form source file means for most developers.

I think this sounds good and would solve issues with that particular flag. I shall raise another issue/PR to update the free-form but I am still interested in this mechanism of passing any compiler flag through the manifest. Happy to be proved wrong though.

[perazz]

I agree, would probably make the most sense to have them under the [build] table? I.e. Cargo has rustflags, we may have something like this?

[build]
link = ["blas","lapack"]
flags = ["-some-other-flag"]
c-flags = ["-c-only-flag"]
cxx-flags = ["-cpp-only-flag"]

[JorgeG94]

I agree, would probably make the most sense to have them under the [build] table? I.e. Cargo has rustflags, we may have something like this?

[build]
link = ["blas","lapack"]
flags = ["-some-other-flag"]
c-flags = ["-c-only-flag"]
cxx-flags = ["-cpp-only-flag"]

I like this idea, it would make it easier to pass flags for -fopenmp and -fopenacc.

[Carltoffel]

Would it be possible to define different sets of flags for each supported compiler? I guess there are projects, that are only working with one specific compiler, or maybe don't work with one specific compiler. It would be nice to define this in the toml. This would be an opportunity for defining compiler specific flags.

E.g.

[build]
default_flags = ["-O3", "-g"]

[build.compilers.ifort]
flags = ["-O2", "-fast"]
min_version = "19.1.0"

[build.compilers.gfortran]
supported = false

[jalvesz]

One would need to cross this with the OS. Intel uses slightly different nomenclature for the same flags depending on the OS.

[perazz]

Exactly, that's why flags for shared Fortran features are standardized via the [fortran] table and internally resolved:

fpm/src/fpm_compiler.F90

Lines 160 to 174 in 1cfcaf8

	character(*), parameter :: &
	flag_gnu_coarray = " -fcoarray=single", &
	flag_gnu_backtrace = " -fbacktrace", &
	flag_gnu_opt = " -O3 -funroll-loops", &
	flag_gnu_debug = " -g", &
	flag_gnu_pic = " -fPIC", &
	flag_gnu_warn = " -Wall -Wextra", &
	flag_gnu_check = " -fcheck=bounds -fcheck=array-temps", &
	flag_gnu_limit = " -fmax-errors=1", &
	flag_gnu_external = " -Wimplicit-interface", &
	flag_gnu_openmp = " -fopenmp", &
	flag_gnu_no_implicit_typing = " -fimplicit-none", &
	flag_gnu_no_implicit_external = " -Werror=implicit-interface", &
	flag_gnu_free_form = " -ffree-form", &
	flag_gnu_fixed_form = " -ffixed-form"

so I think this structure seems preferable. For example for unlimited line length, I think it should be the default setting for the "Free" source form on all compilers (-ffree-line-length-none, -132 for Intel, etc.).

Then, one may still want compiler-based flags, I think it could make sense to have just the flag name so the initial symbol is resolved internally (e.g. because Intel wants - on unix and / on Windows):

[build]
link = ["blas","lapack"]
flags = ["some-other-flag"] # note: no dash
flags.intel = ["intel-only-flag"]
c-flags = ["c-only-flag"]
c-flags.gnu = ["gnu-only-flag"]
cxx-flags = ["cpp-only-flag"]

it would make it easier to pass flags for -fopenmp and -fopenacc.

note OpenMP is a metapackage: dependencies.openmp = "*"

I believe openacc could be implemented the same way, especially as it may carry over additional linking flags

[jalvesz]

e.g. because Intel wants - on unix and / on Windows

it would be - (linux) and /Q (windows), but there are some exception, Ex: -fpp vs /fpp for activating preprocessing with ifort, there might be others.

[lockstockandbarrel]

This seems to be a subset of allowing for profile definitions in the fpm.toml manifest file. Extensive work was being done on that at one time, and there is a related pull request but no action since that I see of. Allowing compiler options in the fpm.toml file seems like it should take into account how the package is built when it is a dependency, and which compiler and which OS as discussed;
as well as different compilations (different MPI versions, different external libraries, ...) and that project was intended to resolve those issues.
#498
Not sure why that went cold, as it was intended as a feature from the beginning; but user-defined profiles (including the default one for a package) and conditional select of which profiles seems like a good generic solution.

[emmabastas]

Would it be possible to define different sets of flags for each supported compiler? I guess there are projects, that are only working with one specific compiler, or maybe don't work with one specific compiler. It would be nice to define this in the toml. This would be an opportunity for defining compiler specific flags.

E.g.

[build]
default_flags = ["-O3", "-g"]

[build.compilers.ifort]
flags = ["-O2", "-fast"]
min_version = "19.1.0"

[build.compilers.gfortran]
supported = false

I can imagine a scenario where this contributes to fragmentation in the package ecosystem:

Let's imagine for the sake of argument that we implement the above solution. The good thing is that we now have an escape-hatch if we want to manipulate our compilers behaviors in ways not implemented by fpm. Let's also imagine (this is contrived but that's the example I came up with) that fortran.source-form = "free"was never implemented. No problem, since we have the escape-hatch a package author can simply do

[build.compilers.ifort]
flags = ["-free"]

[build.compilers.gfortran]
flags = ["-ffree-line-length-none"]

...

for all the relevant compilers. This is a little annoying, sure it would be nice if a fortran.source-form option existed to save you some time and typing, but.. we get the job done and we have more pressing issues than this little quality-of-life improvement.

Imagine down the road a package author writes a package and they use gfortran, they are kind enough to publish it for all of us to use, however, they don't bother looking up the flags equivalent to -ffree-line-length-none for other compilers to add to their manifest, maybe they forgot about it in all their haste, we end up with this manifest:

name = "A"
[build.compilers.gfortran]
flags = ["-ffree-line-length-none"]

and now if someone wants to use package A with the ifort compiler they'll run into issues.

This example is contrived, but for more complex cases, and in a scenario where many packages end up using the escape hatch I think it risks fragmenting the package ecosystem in ways that can be difficult to reverse.

[Carltoffel]

Imagine down the road a package author writes a package and they use gfortran, they are kind enough to publish it for all of us to use, however, they don't bother looking up the flags equivalent to -ffree-line-length-none for other compilers to add to their manifest, maybe they forgot about it in all their haste, we end up with this manifest:

name = "A"
[build.compilers.gfortran]
flags = ["-ffree-line-length-none"]

and now if someone wants to use package A with the ifort compiler they'll run into issues.

This example is contrived, but for more complex cases, and in a scenario where many packages end up using the escape hatch I think it risks fragmenting the package ecosystem in ways that can be difficult to reverse.

I don't think that will be a problem. Compiler profiles would probably not be used on a daily basis. And those who are not looking for it because it is necessary for them may not even know that this feature exists. If you want to use it for a project that should work everywhere, you have to take care of it accordingly. If you don't do this, there is always the option of creating a pull request or a fork.

However, compiler profiles would enable projects that would otherwise be impractical or impossible with fpm. E.g. CUDA Fortran projects.

[jalvesz]

Compiler profiles would probably not be used on a daily basis

I wouldn't say this

However, compiler profiles would enable projects that would otherwise be impractical or impossible with fpm. E.g. CUDA Fortran projects.

One does not need to go that far to see the need for profiles. Most of the time one is looking for a way of achieving the same behavior across different compilers (or close enough). Take file inlinement optimization: gfortran -flto, ifort -ipo(linux) /Qipo (windows), nvfortran -Minline. You want to compile for avx512 architecture? the flags are also different.

So, the key problem from my point of view is not achieving different behaviors for each particular compiler/platform. It is being cross-platform/compiler compatible.