Allow sharing the Go cache directory for crossbuilds

[simonpasquier]

No description provided.

[sylr]

@simonpasquier I am massively rewriting cmd/crossbuild.go in #177

[simonpasquier]

@sylr yes I know but I'm not sure it's the right direction...

[simonpasquier]

I've managed to make it work (though a few things need to be cleaned up). The build job with a prefilled buildcache goes down from 13m40s to 1m28s.
One concern is that the build cache's size for this repository (which isn't too big) is approximately 390MB while Circle CI recommends keeping it under 500MB.

[sylr]

When does the build cache prove effective ? if "our" go sources are modified from one commit to another will it still improve things ?

[roidelapluie]

can't we use 1804?

[simonpasquier]

it's not listed in the docs.

[sylr]

@simonpasquier As I rewrote crossbuild entirely to not use docker anymore in #177 (I created a crossbuild-docker command) would you mind to only commit the tweak you made in the .circleci/conf.yml to save the GOCACHE dir ?

[sylr]

Why the v01-gocache- key ?

[simonpasquier]

that's for testing

[simonpasquier]

@sylr

When does the build cache prove effective ? if "our" go sources are modified from one commit to another will it still improve things ?

Obviously it would need to be tested on Circle CI but this is what I get with my local machine when crossbuilding different versions of github.com/prometheus/prometheus.

# crossbuild the release-2.15 branch without cache.

real    23m53.257s
user    0m3.456s
sys     0m7.881s

# crossbuild the master branch with the cache directory from the previous step.

real    14m23.807s
user    0m6.726s
sys     0m13.227s

# crossbuild the master branch with the cache directory from the previous step.

real    11m38.822s
user    0m6.661s
sys     0m13.647s

So the difference between a fresh cache and an outdated cache isn't huge. Note that it assumes that go.mod hasn't changed between master and release-2.15 which isn't true. We can also have a per-project context variable keeping the cache's version number and bump it to force a refresh of the cached data.

[sylr]

OK, thank you for this.

I also did some tests on my side and running crossbuild on prometheus on my mac generated 6GB of data in the cache dir, do you roughtly have the same number ?

[simonpasquier]

I've reached 8.5GB of build cache after the runs I've described.

[SuperQ]

How's this going? It's approved, but there's a lot of un-resolved threads.

[SuperQ]

Ping, should we finish this up?

[SuperQ]

Ping @simonpasquier, can you rebase this?

[SuperQ]

This needs a rebase

[kakkoyun]

I think we can revive and merge this @simonpasquier.

LGTM.

[kakkoyun]

Suggested change

	image: ubuntu-1604:201903-01
	image: ubuntu-2204:2024.04.4