Staged layer creation #378

Luap99 · 2025-10-08T10:49:30Z

No description provided.

podmanbot · 2025-10-08T10:50:41Z

✅ A new PR has been created in buildah to vendor these changes: containers/buildah#6414

Luap99 · 2025-10-08T11:41:13Z

Podman PR containers/podman#27251 and the buildah test PR containers/buildah#6414 from the bot both look good so that means we can remove the special case from ApplyDiff() in overlay I think, ref containers/podman#25862 (comment)

I still need to work on the actual feature here though to extract while the store in unlocked.

mtrmac

ACK, simplifying ApplyDiff this way does look correct. (I didn’t carefully look at the tempdir addition yet.)

storage/store.go

mtrmac

I’m mostly looking because I was curious — feel free to disregard.

The tar-split comment might explain some of the “unexpected EOF” test failures.

storage/internal/tempdir/tempdir.go

storage/layers.go

storage/drivers/overlay/overlay_test.go

storage/drivers/overlay/overlay.go

storage/layers.go

Luap99 · 2025-10-15T16:43:26Z

@mtrmac FYI I have not really addressed most of your comments yet, I am just trying to push things to see how much things break. Still seeing plenty of test failures.

Issue 1 I see is that I just use the 700 permission from the tmpdir due the rename instead of the proper diff dir creation permissions that are in the driver.create() code

	diff := path.Join(dir, "diff")
	if err := idtools.MkdirAs(diff, forcedSt.Mode, forcedSt.IDs.UID, forcedSt.IDs.GID); err != nil {
		return err
	}

	if d.options.forceMask != nil {
		st.Mode |= os.ModeDir
		if err := idtools.SetContainersOverrideXattr(diff, st); err != nil {
			return err
		}
	}

Not sure if I should expose that into the tmpdir creation logic, I guess that makes the most sense since only the dirver should now the exact permission that should be used?

Second problem I see are timeouts (in parallel running tests) which I guess mean I added a deadlock situation?
https://api.cirrus-ci.com/v1/artifact/task/5906611702595584/html/sys-podman-debian-13-rootless-host-sqlite.log.html

I guess looking at the code this unlock/lock again thing I did is indeed completely broken and unsafe due ABBA deadlock, i.e. in putlayer we also hold the containerStore lock so only unlocking the layer store makes it possible that another process can get the layer lock and then blocks on the still gold container store thus both process handing forever.
I haven't checked all the code paths but I guess with the locking order requirements what I did is basically impossible to achieve anyway and I have to indeed move this out to before we get the lock?

mtrmac · 2025-10-15T17:03:30Z

Issue 1 I see is that I just use the 700 permission from the tmpdir due the rename instead of the proper diff dir creation permissions that are in the driver.create() code

Not sure if I should expose that into the tmpdir creation logic, I guess that makes the most sense since only the dirver should now the exact permission that should be used?

I think that could work.

I was thinking StageAddition does not actually need to create (os.Create/os.Mkdir) the tmpAddPath at all. All of that happens inside a lock-protected td.tempDirPath, so there is ~nothing special, that I can see, about populating tmpAddPath — the provided callback can create the staged item without any help. (That could also mean StageDirectoryAddition and StageFileAddition could be consolidated into one. And I’m not immediately sure we need a callback — StageAddition could return a newStagingPathToPopulate — but I also didn’t now carefully re-read the tempdir package.)

I haven't checked all the code paths but I guess with the locking order requirements what I did is basically impossible to achieve anyway and I have to indeed move this out to before we get the lock?

Per the locking hierarchy documented at the top of store, I think you’re right here.

Luap99 · 2025-10-15T17:15:36Z

And I’m not immediately sure we need a callback — StageAddition could return a newStagingPathToPopulate — but I also didn’t now carefully re-read the tempdir package.)

Yeah my thinking was that the callback provides a "lifetime" of when the path is safe to use, if I return a string/struct with the path then the caller can cleanup/commit and then still use the path afterwards. This is really where I start to hate go because in rust this would be trivial to enforce so that there could only ever be one call to commit and then render the object useless afterwards.

But yes usage wise this callback is indeed getting quite ugly to the point where just returning the path is much simpler and well how go works in general. I do like the suggestion of just returning the path to consolidate both tmpdir functions into one so I will go with that.

Luap99 · 2025-10-22T19:09:23Z

@mtrmac I will push this into podman and run more tests tomorrow but I think like this it should be workable now. I fix the minor lint issues here of course on the next push. Let me know if this approach seem right to you, I guess the code could need some more better comments/function names likely.

Luap99 · 2025-10-23T17:46:07Z

Ok last issue noticed in podman the idmapping logic cannot be implemented unlocked I fear.

We have this code in putLayer()

	if options.HostUIDMapping {
		options.UIDMap = nil
	}
	if options.HostGIDMapping {
		options.GIDMap = nil
	}
	uidMap := options.UIDMap
	gidMap := options.GIDMap
	if parent != "" {
		var ilayer *Layer
		for _, l := range append([]roLayerStore{rlstore}, rlstores...) {
			lstore := l
			if lstore != rlstore {
				if err := lstore.startReading(); err != nil {
					return nil, -1, err
				}
				defer lstore.stopReading()
			}
			if l, err := lstore.Get(parent); err == nil && l != nil {
				ilayer = l
				parent = ilayer.ID
				break
			}
		}
		if ilayer == nil {
			return nil, -1, ErrLayerUnknown
		}
		parentLayer = ilayer

		if err := s.containerStore.startWriting(); err != nil {
			return nil, -1, err
		}
		defer s.containerStore.stopWriting()
		containers, err := s.containerStore.Containers()
		if err != nil {
			return nil, -1, err
		}
		for _, container := range containers {
			if container.LayerID == parent {
				return nil, -1, ErrParentIsContainer
			}
		}
		if !options.HostUIDMapping && len(options.UIDMap) == 0 {
			uidMap = ilayer.UIDMap
		}
		if !options.HostGIDMapping && len(options.GIDMap) == 0 {
			gidMap = ilayer.GIDMap
		}
	} else {
		// FIXME? It’s unclear why we are holding containerStore locked here at all
		// (and because we are not modifying it, why it is a write lock, not a read lock).
		if err := s.containerStore.startWriting(); err != nil {
			return nil, -1, err
		}
		defer s.containerStore.stopWriting()

		if !options.HostUIDMapping && len(options.UIDMap) == 0 {
			uidMap = s.uidMap
		}
		if !options.HostGIDMapping && len(options.GIDMap) == 0 {
			gidMap = s.gidMap
		}
	}
	if s.canUseShifting(uidMap, gidMap) {
		options.IDMappingOptions = types.IDMappingOptions{HostUIDMapping: true, HostGIDMapping: true, UIDMap: nil, GIDMap: nil}
	} else {
		options.IDMappingOptions = types.IDMappingOptions{
			HostUIDMapping: options.HostUIDMapping,
			HostGIDMapping: options.HostGIDMapping,
			UIDMap:         copySlicePreferringNil(uidMap),
			GIDMap:         copySlicePreferringNil(gidMap),
		}
	}

However we extract the layer with the caller specified layerOptions.UIDMap, layerOptions.GIDMap in stageWithUnlockedStore() before this code can get run.

I guess the s.uidMap case could be done without a lock but not the parent lookups? So I guess the simple solution would be to only use the unlocked extract path when options.HostGIDMapping is true.

mtrmac · 2025-10-23T18:01:52Z

(I didn’t look at the current code in this PR yet.) AFAICS layer.[UG]IDMap never changes once the layer is created, so it should be safe to look up the parent, read these values, and then unlock.

IIRC the plan was to start layer creation with an ID lookup (so that we don’t start expensively staging it if it already exists), so the parent lookup could be done within the same lock scope.

giuseppe · 2025-10-23T20:08:05Z

storage/layers.go

+			Mappings: idtools.NewIDMappingsFromMaps(layerOptions.UIDMap, layerOptions.GIDMap),
+			// FIXME: What to do here? We have no lock and assigned label yet.
+			// Overlayfs should not need it anyway so this seems fine for now.
+			MountLabel: "",


I think this is correct

storage/layers.go

storage/drivers/overlay/overlay.go

storage/layers.go

mtrmac

Reading this commit by commit, this looks really great — the comments about documenting locking semantics etc. are basically the final polish.

(Note to self: Eventually it might be worth re-reading the final state as is, to check whether there is any opportunity to simplify.)

Around #378 (comment) and more recently with the parent’s mapping there was some tentative discussion about checking whether the layer exists before deciding to stage it — that’s still to be decided, I think. (In c/image, commitLayer does do a layer presence check before deciding to create it — but in case it is reusing an existing local layer by extracting it into a temporary tarball to be applied, there is still quite a window in which the layer could be concurrently created. Of course, c/image can add one more check to its caller — but if we happened to take locks to read the parent’s state, a lookup for an ID already existing would be ~free.)

storage/internal/tempdir/tempdir.go

storage/drivers/overlay/overlay.go

storage/store.go

storage/drivers/overlay/overlay.go

mtrmac · 2025-10-23T22:42:58Z

storage/store.go

+			}
+		}()
+		// FIXME: type case should be safe for now but really there should be a better way to do this
+		err = m.stageWithUnlockedStore(rlstore.(*layerStore), lOptions)


#378 (comment) talks about ID mapping values from the parent layer — if I’m reading the code right, that does not yet happen.

Luap99 · 2025-11-04T17:19:17Z

#!/usr/bin/env bpftrace

kfunc:fcntl_setlk
{
  $lockname = str(args->filp->f_path.dentry->d_name.name);

  if ($lockname == "storage.lock" || $lockname == "layers.lock" ||
      $lockname == "images.lock" || $lockname == "containers.lock" ) {

    @blockedTime[tid, $lockname] = nsecs;
  }
}

kretfunc:fcntl_setlk
{
  $lockname = str(args->filp->f_path.dentry->d_name.name);

  if ($lockname == "storage.lock" || $lockname == "layers.lock" ||
      $lockname == "images.lock" || $lockname == "containers.lock" ) {

    // lock duration in msec
    $lock_duration = (nsecs - @blockedTime[tid, $lockname])/1000000;
    if ($lock_duration) {
      printf("blocked %s time: %lld msec\n", $lockname, $lock_duration);
    }
    delete(@blockedTime[tid, $lockname]);
    @lockholdTime[pid, args->fd] = (nsecs, $lockname);
  }
}

tracepoint:syscalls:sys_enter_close
{
  $a = @lockholdTime[pid, args->fd];
  if ($a.0) {
    // lock duration in msec
    $lock_duration = (nsecs - $a.0)/1000000;
    if ($lock_duration) {
      printf("lock %s time: %lld msec\n", $a.1, $lock_duration);
    }
    delete(@lockholdTime[pid, args->fd]);
  }
}

FYI this is the script I used to measure lock times during my demo last week, not sure if there is a decent place to store this. I guess it could be helpful in the future to find more lock contention, should I add it under hack/ maybe?

Luap99 · 2025-11-11T13:33:18Z

Podman/buildah tests seems to be passing (minus unrelated flakes AFAICT)
containers/podman#27251 containers/buildah#6414

This should be good to review again. I think I addressed the comments.

mtrmac

Highlights:

Locking stores that contain the parent layer while creating a child
The pre-existing discussion about MountLabel — I think the current feature set is fine, but stageWithUnlockedStore should refuse to stage if the input provides a mountLabel, just to be sure we are not introducing a problem.

There are some minor pre-existing comments, and (note to self) I’d like to read the resulting code as a whole.

mtrmac · 2025-11-10T16:30:21Z

storage/internal/tempdir/tempdir.go

+// struct. The returned type StagedAddition has a Commit() function to move the content from
+// the temporary location to the final one.
+//
+// The caller MUST call Commit() before Cleanup(0 is called on the TempDir, otherwise the


Suggested change

// The caller MUST call Commit() before Cleanup(0 is called on the TempDir, otherwise the

// The caller MUST call Commit() before Cleanup() is called on the TempDir, otherwise the

mtrmac · 2025-11-14T17:49:07Z

storage/internal/tempdir/tempdir.go

+	if td.tempDirLock == nil {
+		return nil, fmt.Errorf("temp dir instance not initialized or already cleaned up")
+	}
+	fileName := fmt.Sprintf("%d-", td.counter) + "addition"


Suggested change

fileName := fmt.Sprintf("%d-", td.counter) + "addition"

fileName := fmt.Sprintf("%d-addition", td.counter)

(or, hypothetically, have the caller provide a name, but I doubt that would be worth the effort)

I switch this to strconv.FormatUint(td.counter, 10) + "-addition", while it shouldn't matter here it is more efficient as it doesn't need runtime type checking due the interface that the sprintf needs

mtrmac · 2025-11-14T17:56:36Z

storage/drivers/overlay/overlay.go


-	st := idtools.Stat{IDs: idPair, Mode: defaultPerms}
-
 	if parent != "" {


This part was copied into getLayerPermissions, so I think it can be deleted here.

mtrmac · 2025-11-14T18:19:01Z

storage/drivers/overlay/overlay.go

+		return t.Cleanup, nil, -1, err
+	}
+
+	_, forcedSt, st, err := d.getLayerPermissions(parent, options.Mappings.UIDs(), options.Mappings.GIDs())


This calls GetContainersOverrideXattr and stat on d.dir(parent), without holding locks.

I think that’s fine if we know that the parent layer existed = was previously fully created — otherwise we might race against Driver.create setting the fields, and see incorrect interim values. And we do know that, due to populateLayerOptions — so that’s all working fine, but it’s a very remote dependency, so worth carefully documenting in the StartStagingDiffToApply function and in callers across the call stack.

(If the layer existed and is later removed, we can fail with an error, but we won’t see invalid data. Failing with an error is semantically correct, creating a layer against a missing parent should fail.)

makes sense, one thing I wonder should we turn that ENOENT into ErrLayerUnknown? Callers that currently match that would still have the possibility to retry and create the parent while an ENOENT could mean anything.

mtrmac · 2025-11-14T18:37:17Z

storage/layers.go

+	if err != nil {
+		return -1, err
+	}
+	defer tarSplitFile.Close()


I’m wondering whether this should have error checks… generally, the kernel can defer writes until close(2). OTOH we call .Sync(), so that should be good enough.

Just as a matter of comfort / lack of risk, I’d prefer error checks on the close operations, but I can’t think of a specific reason why that should matter.

(If this were changed, applies also to the other use of createTarSplitFile.)

I think we should good with sync(). I can certainly add error checks but also that pattern of missing error checks for close is far more than this call here where it shouldn't matter due the sync.

mtrmac · 2025-11-14T20:36:20Z

storage/store.go

 	defer rlstore.stopWriting()
-	return s.putLayer(rlstore, rlstores, id, parent, names, mountLabel, writeable, lOptions, diff, nil)
+
+	if parentLayer == nil {


if options == nil, isn’t it? Some layers have no parents.

And then the comment just below can be dropped.

mtrmac · 2025-11-14T20:39:42Z

storage/store.go

+		if !compareIdMappings(freshLayer.UIDMap, parentLayer.UIDMap) || !compareIdMappings(freshLayer.GIDMap, parentLayer.GIDMap) {
+			// Fatal problem. Mappings changed so the parent must be considered different now.
+			// Since we consumed the diff there is no we to recover, return error to caller. The caller would need to retry.
+			// How likely is that and would need to return a special error so c/image could do the retries?


(This should currently not happen for c/image-created layers, because those have content-based IDs. It could happen only with a non-c/image writer that does something different. But I don’t know whether we will continue to have deterministic IDs with the sha512 work, I suspect we might not.

Either way, c/image pulls are currently not resilient to concurrent layer removals, so a hard failure here is ~fine. We’d need some kind of “lease a layer” feature that allows the pull to hold a found parent layer while downloading a child, and that does not exist.)

mtrmac · 2025-11-14T20:50:17Z

storage/drivers/template.go

@@ -1,52 +0,0 @@
-package graphdriver


(Unrelated: It turns out there are ~no non-trivial implementations of CreateFromTemplate now, so that method could be deleted and the caller simplified a bit not to compute its extra parameters. But that’s way out of scope.)

mtrmac · 2025-11-14T21:07:55Z

storage/layers.go

+// a layer potentially before we even take a look if the driver implements the
+// ApplyDiffStaging interface.
+// This should be initialized with layerStore.newMaybeStagedLayerExtraction()
+type maybeStagedLayerExtraction struct {


(Absolutely non-blocking: it might make sense to inline this into stagedLayerOptions/layerCreationContents/whatever — I think it would be nice, long term, for the chunked and non-chunked staging paths to be unified, by making the chunked code also use StagedAddition and the like. [That’s 100% out of scope.]

OTOH right now the semantics of “caller must call .cleanup()” makes more sense when the two structs are distinct.)

mtrmac · 2025-11-14T21:16:50Z

storage/store.go

+			// How likely is that and would need to return a special error so c/image could do the retries?
+			return nil, -1, fmt.Errorf("error during staged layer apply, parent layer %q changed id mappings while the content was extracted, must retry layer creation", parent)
+		}
+		// FIXME: do we need to validate something else that would affect the extracted content?


This is fine, but it’s a somewhat remote dependency on populateLayerOptions; I’d like some fairly explicit comments linking the ID mapping comparison condition and the code in populateLayerOptions… or maybe even wrapping this condition into populateLayerOptionsInputsUnchanged function.

struct populateLayerOptionsInput { uidMap, gidMap } func (… populateLayerOptionsInput) equals(populateLayerOptionsInput) bool {…} func populateLayerOptionsInputFromLayer(*Layer) populateLayerOptionsInput {…} // only consume parentLayer via populateLayerOptionsInput(parentLayer) in populateLayerOptionsInput

would be way too overengineered and not worth the mental overhead, but I think we need something.

Add a new function to stage additions. This should be used to extract the layer content into a temp directory without holding the storage lock and then under the lock just rename the directory into the final location to reduce the lock contention. Signed-off-by: Paul Holzinger <[email protected]>

It is not clear to me when it will hit the code path there, by normal layer creation we always pass a valid parent so this branch is never reached AFAICT. Let's remove it and see if all tests still pass in podman, buildah and others... Signed-off-by: Paul Holzinger <[email protected]>

Split out the layer permission gathering from the main create() function so it can be reused elsehwere, see the next commit. Signed-off-by: Paul Holzinger <[email protected]>

Add a function to apply the diff into a tmporary directory so we can do that unlocked and only rename under the lock. Signed-off-by: Paul Holzinger <[email protected]>

I cannot see any reason why we should buffer the full tar split content in memory before writing it. That layer is still mark partial at this point and the store is locked so there is no concurrent access either thus we do not need the atomic rename here. Signed-off-by: Paul Holzinger <[email protected]>

Split it into multiple function to make it reusable without having a layer and so that it can be used unlocked see the following commits. Signed-off-by: Paul Holzinger <[email protected]>

The extracting of the tar under the store lock is a bottleneck as many concurrent processes might hold the locks for a long time on big layers. To address this move the layer extraction before we take the locks if possible. Currently this only work when using the overlay driver as the implementation requires driver specifc details in order for a rename() to work. Signed-off-by: Paul Holzinger <[email protected]>

It doesn't seem needed here so don't take it. Signed-off-by: Paul Holzinger <[email protected]>

A minor rework to enable more changes in following commits. Note the caller still must hold the layer store locks so ensure we return the layer locked and let the caller unlock instead. Signed-off-by: Paul Holzinger <[email protected]>

Make it reusable for other callers, see next commit. Also while at it remove the dedupeStrings() call, as pointed out by Miloslav the work it is doing is more expensive than just checking the same name several times as it does a O(1) map lookup. Also most callers won't pass duplicated names to begin with. Signed-off-by: Paul Holzinger <[email protected]>

Just be safe based on the review feedback from the PR. containers#378 Signed-off-by: Paul Holzinger <[email protected]>

podmanbot · 2025-11-17T17:38:05Z

✅ A new PR has been created in buildah to vendor these changes: containers/buildah#6508

Just be safe based on the review feedback from the PR. containers#378 Signed-off-by: Paul Holzinger <[email protected]>

The untar can be quite expensive so check for id, name conflicts right away. Also we must populate the idmappings so we extract with the right uids/gids. Signed-off-by: Paul Holzinger <[email protected]>

This function was added in commit c577a81 and used by older drivers we no longer suppor, such as aufs and windows. As such this is dead code and can be removed. Signed-off-by: Paul Holzinger <[email protected]>

It is unused in all drivers now, so it can be removed. Signed-off-by: Paul Holzinger <[email protected]>

We use this this typo all the time now so make the naming a bit more clear. Signed-off-by: Paul Holzinger <[email protected]>

Just be safe based on the review feedback from the PR. containers#378 Signed-off-by: Paul Holzinger <[email protected]>

github-actions bot added the storage Related to "storage" package label Oct 8, 2025

podmanbot pushed a commit to podmanbot/buildah that referenced this pull request Oct 8, 2025

dnm: Vendor changes from containers/container-libs#378

6d83f9f

podmanbot mentioned this pull request Oct 8, 2025

Sync: WIP: Staged layer creation containers/buildah#6414

Closed

mtrmac reviewed Oct 8, 2025

View reviewed changes

storage/store.go Outdated Show resolved Hide resolved

Luap99 force-pushed the staged-layer-creation branch from 348a11e to b7780f2 Compare October 13, 2025 13:14

podmanbot pushed a commit to podmanbot/buildah that referenced this pull request Oct 13, 2025

dnm: Vendor changes from containers/container-libs#378

ac18582

mtrmac reviewed Oct 13, 2025

View reviewed changes

Luap99 force-pushed the staged-layer-creation branch from b7780f2 to bbb2266 Compare October 15, 2025 14:59

podmanbot pushed a commit to podmanbot/buildah that referenced this pull request Oct 15, 2025

dnm: Vendor changes from containers/container-libs#378

45eda1a

Luap99 force-pushed the staged-layer-creation branch from bbb2266 to 74d0e97 Compare October 22, 2025 18:42

podmanbot pushed a commit to podmanbot/buildah that referenced this pull request Oct 22, 2025

dnm: Vendor changes from containers/container-libs#378

4ce027b

Luap99 force-pushed the staged-layer-creation branch from 74d0e97 to 5cf326c Compare October 23, 2025 10:31

podmanbot pushed a commit to podmanbot/buildah that referenced this pull request Oct 23, 2025

dnm: Vendor changes from containers/container-libs#378

284f29f

Luap99 force-pushed the staged-layer-creation branch from 5cf326c to e60d339 Compare October 23, 2025 12:15

podmanbot pushed a commit to podmanbot/buildah that referenced this pull request Oct 23, 2025

dnm: Vendor changes from containers/container-libs#378

e21be83

giuseppe reviewed Oct 23, 2025

View reviewed changes

mtrmac reviewed Oct 23, 2025

View reviewed changes

Luap99 force-pushed the staged-layer-creation branch from e60d339 to 91fcdbb Compare October 30, 2025 19:06

podmanbot pushed a commit to podmanbot/buildah that referenced this pull request Oct 30, 2025

dnm: Vendor changes from containers/container-libs#378

e20dc5d

Luap99 force-pushed the staged-layer-creation branch from 91fcdbb to 651c8f5 Compare November 4, 2025 14:46

podmanbot pushed a commit to podmanbot/buildah that referenced this pull request Nov 4, 2025

dnm: Vendor changes from containers/container-libs#378

31d42ea

podmanbot pushed a commit to podmanbot/buildah that referenced this pull request Nov 10, 2025

dnm: Vendor changes from containers/container-libs#378

babffb1

Luap99 marked this pull request as ready for review November 11, 2025 14:51

Luap99 changed the title ~~WIP: Staged layer creation~~ Staged layer creation Nov 11, 2025

mtrmac reviewed Nov 14, 2025

View reviewed changes

Luap99 added 10 commits November 17, 2025 14:58

overlay: add getLayerPermissions()

f85a32b

Split out the layer permission gathering from the main create() function so it can be reused elsehwere, see the next commit. Signed-off-by: Paul Holzinger <[email protected]>

overlay: add StartStagingDiffToApply()

1127faf

Add a function to apply the diff into a tmporary directory so we can do that unlocked and only rename under the lock. Signed-off-by: Paul Holzinger <[email protected]>

storage: rework applyDiffWithOptions()

92a81e3

Split it into multiple function to make it reusable without having a layer and so that it can be used unlocked see the following commits. Signed-off-by: Paul Holzinger <[email protected]>

storage: don't lock container store in putLayer

058bbc7

It doesn't seem needed here so don't take it. Signed-off-by: Paul Holzinger <[email protected]>

Luap99 added a commit to Luap99/container-libs that referenced this pull request Nov 17, 2025

storage: don't use stagedLayerExtract when mountLabel is set

9c7995a

Just be safe based on the review feedback from the PR. containers#378 Signed-off-by: Paul Holzinger <[email protected]>

Luap99 force-pushed the staged-layer-creation branch from be42227 to 9c7995a Compare November 17, 2025 17:31

podmanbot pushed a commit to podmanbot/buildah that referenced this pull request Nov 17, 2025

dnm: Vendor changes from containers/container-libs#378

a871211

podmanbot mentioned this pull request Nov 17, 2025

Sync: Staged layer creation containers/buildah#6508

Draft

Luap99 added a commit to Luap99/container-libs that referenced this pull request Nov 17, 2025

storage: don't use stagedLayerExtract when mountLabel is set

bac8c67

Just be safe based on the review feedback from the PR. containers#378 Signed-off-by: Paul Holzinger <[email protected]>

Luap99 force-pushed the staged-layer-creation branch from 9c7995a to bac8c67 Compare November 17, 2025 17:55

podmanbot pushed a commit to podmanbot/buildah that referenced this pull request Nov 17, 2025

dnm: Vendor changes from containers/container-libs#378

abb1229

Luap99 added 5 commits November 17, 2025 19:42

storage: do conflict checks before staging untar

69cb961

The untar can be quite expensive so check for id, name conflicts right away. Also we must populate the idmappings so we extract with the right uids/gids. Signed-off-by: Paul Holzinger <[email protected]>

storage: remove NaiveCreateFromTemplate()

1ad8e62

This function was added in commit c577a81 and used by older drivers we no longer suppor, such as aufs and windows. As such this is dead code and can be removed. Signed-off-by: Paul Holzinger <[email protected]>

storage: remove parent param from ApplyDiff()

7022fc7

It is unused in all drivers now, so it can be removed. Signed-off-by: Paul Holzinger <[email protected]>

storage: rename stagedLayerOptions to layerCreationContents

f962645

We use this this typo all the time now so make the naming a bit more clear. Signed-off-by: Paul Holzinger <[email protected]>

storage: don't use stagedLayerExtract when mountLabel is set

ceb1809

Just be safe based on the review feedback from the PR. containers#378 Signed-off-by: Paul Holzinger <[email protected]>

Luap99 force-pushed the staged-layer-creation branch from bac8c67 to ceb1809 Compare November 17, 2025 18:43

podmanbot pushed a commit to podmanbot/buildah that referenced this pull request Nov 17, 2025

dnm: Vendor changes from containers/container-libs#378

490aab8

	// The caller MUST call Commit() before Cleanup(0 is called on the TempDir, otherwise the
	// The caller MUST call Commit() before Cleanup() is called on the TempDir, otherwise the

	fileName := fmt.Sprintf("%d-", td.counter) + "addition"
	fileName := fmt.Sprintf("%d-addition", td.counter)


		st := idtools.Stat{IDs: idPair, Mode: defaultPerms}

		if parent != "" {

Staged layer creation #378

Are you sure you want to change the base?

Staged layer creation #378

Uh oh!

Conversation

Luap99 commented Oct 8, 2025

Uh oh!

podmanbot commented Oct 8, 2025

Uh oh!

Luap99 commented Oct 8, 2025

Uh oh!

mtrmac left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mtrmac left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Luap99 commented Oct 15, 2025

Uh oh!

mtrmac commented Oct 15, 2025

Uh oh!

Luap99 commented Oct 15, 2025

Uh oh!

Luap99 commented Oct 22, 2025

Uh oh!

Luap99 commented Oct 23, 2025

Uh oh!

mtrmac commented Oct 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mtrmac left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Luap99 commented Nov 4, 2025

Uh oh!

Luap99 commented Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mtrmac left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

mtrmac commented Oct 23, 2025 •

edited

Loading

Luap99 commented Nov 11, 2025 •

edited

Loading