Split goose into separate modules (same repository)

[mfridman]

More of a thought at the moment, but wanted to jot this down.

Currently, the goose package and the CLI are tracked as a single Go module and versioned together (think single go.mod in the entire repository). However, there are some limitations to this approach:

Limitations

• The package and CLI share the same minimum Go version. This means existing and new SQL drivers (as direct dependencies) must be compatible with the minimum supported Go version in goose. For example, in Add Turso  #658 (comment), we'd need to bump the minimum version, and even though the package doesn't use the driver, it is still bumping the "global" go.mod file.
• In the CLI, we want to use the latest Go version and the most up-to-date drivers when building the CLI binary. This is at odds with the goose package, which aims to be backwards compatible with the last few Go releases and support the lowest possible version.

So, what I'm proposing is to add go.{mod|sum} files into ./cmd/goose and track the CLI and its dependencies as a separate module.

module github.com/pressly/goose/cmd/goose

For local development can use

1. go work init (initialize a go.work file)
2. go work use -r . (recursively add all modules, and sub-modules)

There are a few downsides to this approach through,

Cons

• Need to figure out versioning between the CLI and the package
• Users would no longer be able to install the binary like so:

# before
go install github.com/pressly/goose/v3/cmd/goose@latest
# after
go install github.com/pressly/goose/cmd/goose/v3@latest

• Cannot use ./internal packages in github.com/pressly/goose/v3 module because it is now a separate module.

Pros

• Ability to pin the goose CLI module to the latest Go version, while keeping the goose package go.mod a few Go versions back
• Flexibility to use any driver, no limitation on minimum Go version
• Versioning may be considered a pro, since consumers don't have a dependency bump for unrelated code changes.

[FelipeLema]

My take is that, given how I've seen other code being organized, is that this tool supports "latest stable" Go version and previous versions are manually supported by backporting code and/or deactivating features.

my bet is that dependencies should follow this scheme, which sounds more likely to happen than them holding on to previous Go releases. At least, to me.

In this scheme, keeping separate branches sounds like a way to go. Something like having these binaries per goose release

• "goose-amd64-linux-Go-1.88"
• "goose-amd64-linux-Go-1.1 (no clickhouse, no turso, see NOTES)"

[kannman]

Currently, goose ships with drivers for multiple databases (postgres, mysql, sqlite, etc.) all bundled together, but in my case, I only need postgres support.
This results in unnecessary bloat in my binaries and docker images.
When you're only using one database type, pulling in all the other drivers adds dependencies and increases build size for no benefit.

[mfridman]

@kannman, yes, it's a general-purpose tool that by default supports many database technologies.

If size really matters, then you could build a lite version with tags, a bit more here:

https://pressly.github.io/goose/installation/#lite-version

go build \
    -tags='no_clickhouse no_libsql no_mssql no_mysql no_sqlite3 no_vertica no_ydb' \
    -o goose ./cmd/goose

This should only include the postgres driver, and the resulting binary should be ~3.5x smaller.

[mfridman]

I was curious what the numbers where:

Driver	Build tag	Size when excluded	Marginal cost
ydb	no_ydb	39.01 MB	17.01 MB
clickhouse	no_clickhouse	46.70 MB	9.32 MB
postgres	no_postgres	50.30 MB	5.73 MB
sqlite3	no_sqlite3	50.30 MB	5.73 MB
mssql	no_mssql	54.39 MB	1.63 MB
turso/libsql	no_libsql	54.51 MB	1.51 MB
mysql	no_mysql	55.27 MB	0.75 MB
vertica	no_vertica	55.39 MB	0.64 MB