-
Notifications
You must be signed in to change notification settings - Fork 16
Add OCI image support #191
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
henrybear327
wants to merge
8
commits into
sysprog21:main
Choose a base branch
from
henrybear327:oci/setup
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
a669d01
Extract elfuse_launch and add guest launch flags
henrybear327 d4209c2
Emulate procfs and devices for container guests
henrybear327 3a34ec6
Confine guest filesystem access to the sysroot
henrybear327 5ca7c90
Add elfuse-container CLI for OCI images
henrybear327 e2bca1c
Add macOS sparsebundle rootfs with COW clones
henrybear327 ddc53a5
Add OCI lifecycle commands and reachability GC
henrybear327 ab55034
Add OCI conformance checks and lifecycle CI
henrybear327 7193812
Document OCI image support
henrybear327 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,128 @@ | ||
| package main | ||
|
|
||
| import ( | ||
| "errors" | ||
| "fmt" | ||
| "os" | ||
| "path/filepath" | ||
| "syscall" | ||
| ) | ||
|
|
||
| // Per-digest bundle locks. | ||
| // | ||
| // A case-sensitive sparsebundle bundle (<store>/cs/<algo>/<hex>/) is shared | ||
| // mutable state: concurrent `run`s of one digest share its attached volume, | ||
| // while prune --cache and rmi --force want to detach and remove it. Liveness | ||
| // is decided by advisory flocks, not by pids or directory scans -- a held | ||
| // lock proves a live holder regardless of pid reuse, and a free lock proves | ||
| // the holder is gone regardless of what the directory contains. | ||
| // | ||
| // Two lock files live in the bundle directory, deliberately OUTSIDE the | ||
| // mounted volume: hdiutil detach -force revokes descriptors inside the | ||
| // volume, which would silently drop a lock held there, and the locks must be | ||
| // probeable while the volume is not attached at all. | ||
| // | ||
| // - attach.lock: exclusive, serializes bundle lifecycle transitions. A run | ||
| // holds it (blocking) across provisioning -- stale-mount recovery, | ||
| // hdiutil create/attach -- and the last-one-out detach; a sweep holds it | ||
| // (non-blocking) for its whole reap-detach-remove sequence. | ||
| // - run.lock: every live run holds it shared from before the volume is | ||
| // attached until the guest exits (the process exiting releases it, so a | ||
| // killed run cannot leak liveness). Anyone holding it exclusively has | ||
| // proven there are zero live runs: sweeps take it non-blocking (busy => | ||
| // skip the bundle), and provision probes it to tell a stale leftover | ||
| // mount from one that is live. | ||
| // | ||
| // Lock ordering: attach.lock is always acquired before run.lock is taken | ||
| // exclusively. That makes the EX->SH downgrade in provision safe (flock | ||
| // downgrades by release-and-reacquire, but no EX taker can slip in without | ||
| // attach.lock, which the downgrader holds) and rules out lock-order cycles | ||
| // with the store-level .lock, which prune/rmi already hold around the sweep | ||
| // while runs never take bundle locks under the store lock. | ||
|
|
||
| // errCacheBusy reports that a bundle lock is held by a live run (or an | ||
| // in-flight provision), so the caller must not detach or remove the bundle. | ||
| var errCacheBusy = errors.New("in use by a live run") | ||
|
|
||
| func attachLockPath(bundle string) string { return filepath.Join(bundle, "attach.lock") } | ||
| func runLockPath(bundle string) string { return filepath.Join(bundle, "run.lock") } | ||
|
|
||
| // flockFile is an open file holding (or having held) an advisory flock. | ||
| type flockFile struct { | ||
| f *os.File | ||
| } | ||
|
|
||
| // acquireFlock opens path (creating it if absent) and takes the flock mode | ||
| // `how` (syscall.LOCK_SH or LOCK_EX, optionally |LOCK_NB). A non-blocking | ||
| // request that loses returns errCacheBusy (wrapped with the path). | ||
| // | ||
| // A sweeper removes the whole bundle directory -- lock files included -- | ||
| // while holding both locks. A racing acquirer may then have opened the path | ||
| // just before the unlink and be holding a lock on an orphaned inode no later | ||
| // process can observe. Guard against that: after locking, verify the path | ||
| // still resolves to the locked inode; otherwise retry against the recreated | ||
| // file. The retry count is a defense bound, not a correctness knob -- one | ||
| // retry per concurrent unlink is the worst case. | ||
| func acquireFlock(path string, how int) (*flockFile, error) { | ||
| for range 16 { | ||
| f, err := os.OpenFile(path, os.O_CREATE|os.O_RDWR, 0o644) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| if err := flockRetryIntr(int(f.Fd()), how); err != nil { | ||
| f.Close() | ||
| if errors.Is(err, syscall.EWOULDBLOCK) || errors.Is(err, syscall.EAGAIN) { | ||
| return nil, fmt.Errorf("%s: %w", path, errCacheBusy) | ||
| } | ||
| return nil, fmt.Errorf("lock %s: %w", path, err) | ||
| } | ||
| var pathSt, fdSt syscall.Stat_t | ||
| if err := syscall.Stat(path, &pathSt); err != nil { | ||
| f.Close() | ||
| if errors.Is(err, syscall.ENOENT) { | ||
| continue // unlinked under us; retry on the recreated file | ||
| } | ||
| return nil, fmt.Errorf("lock %s: %w", path, err) | ||
| } | ||
| if err := syscall.Fstat(int(f.Fd()), &fdSt); err != nil { | ||
| f.Close() | ||
| return nil, fmt.Errorf("lock %s: %w", path, err) | ||
| } | ||
| if pathSt.Dev == fdSt.Dev && pathSt.Ino == fdSt.Ino { | ||
| return &flockFile{f: f}, nil | ||
| } | ||
| f.Close() // path now names a different file; lock that one instead | ||
| } | ||
| return nil, fmt.Errorf("lock %s: persistent unlink race", path) | ||
| } | ||
|
|
||
| // flockRetryIntr issues flock, retrying on EINTR (a blocking acquisition may | ||
| // be interrupted by the signal forwarding the run wrapper installs). | ||
| func flockRetryIntr(fd, how int) error { | ||
| for { | ||
| err := syscall.Flock(fd, how) | ||
| if !errors.Is(err, syscall.EINTR) { | ||
| return err | ||
| } | ||
| } | ||
| } | ||
|
|
||
| // Downgrade converts a held exclusive lock to shared. flock implements this | ||
| // as release-then-reacquire, so it is race-free only while the caller holds | ||
| // attach.lock: every exclusive taker of run.lock acquires attach.lock first, | ||
| // so none can slip into the gap. | ||
| func (l *flockFile) Downgrade() error { | ||
| return flockRetryIntr(int(l.f.Fd()), syscall.LOCK_SH) | ||
| } | ||
|
|
||
| // Close releases the lock and closes the file. Safe on nil and after a prior | ||
| // Close. | ||
| func (l *flockFile) Close() error { | ||
| if l == nil || l.f == nil { | ||
| return nil | ||
| } | ||
| _ = syscall.Flock(int(l.f.Fd()), syscall.LOCK_UN) | ||
| err := l.f.Close() | ||
| l.f = nil | ||
| return err | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,119 @@ | ||
| package main | ||
|
|
||
| import ( | ||
| "errors" | ||
| "os" | ||
| "path/filepath" | ||
| "syscall" | ||
| "testing" | ||
| ) | ||
|
|
||
| func TestAcquireFlockSharedCoexists(t *testing.T) { | ||
| path := filepath.Join(t.TempDir(), "run.lock") | ||
| a, err := acquireFlock(path, syscall.LOCK_SH) | ||
| if err != nil { | ||
| t.Fatal(err) | ||
| } | ||
| defer a.Close() | ||
| b, err := acquireFlock(path, syscall.LOCK_SH|syscall.LOCK_NB) | ||
| if err != nil { | ||
| t.Fatalf("second shared lock: %v, want success", err) | ||
| } | ||
| defer b.Close() | ||
| } | ||
|
|
||
| func TestAcquireFlockExclusiveBlockedIsCacheBusy(t *testing.T) { | ||
| path := filepath.Join(t.TempDir(), "run.lock") | ||
| a, err := acquireFlock(path, syscall.LOCK_SH) | ||
| if err != nil { | ||
| t.Fatal(err) | ||
| } | ||
| defer a.Close() | ||
| _, err = acquireFlock(path, syscall.LOCK_EX|syscall.LOCK_NB) | ||
| if !errors.Is(err, errCacheBusy) { | ||
| t.Fatalf("exclusive over shared err = %v, want errCacheBusy", err) | ||
| } | ||
|
|
||
| // Releasing the shared lock frees the exclusive probe. | ||
| if err := a.Close(); err != nil { | ||
| t.Fatal(err) | ||
| } | ||
| b, err := acquireFlock(path, syscall.LOCK_EX|syscall.LOCK_NB) | ||
| if err != nil { | ||
| t.Fatalf("exclusive after release: %v, want success", err) | ||
| } | ||
| defer b.Close() | ||
| } | ||
|
|
||
| func TestFlockDowngradeAdmitsSharedBlocksExclusive(t *testing.T) { | ||
| path := filepath.Join(t.TempDir(), "run.lock") | ||
| a, err := acquireFlock(path, syscall.LOCK_EX) | ||
| if err != nil { | ||
| t.Fatal(err) | ||
| } | ||
| defer a.Close() | ||
| if _, err := acquireFlock(path, syscall.LOCK_SH|syscall.LOCK_NB); !errors.Is(err, errCacheBusy) { | ||
| t.Fatalf("shared over exclusive err = %v, want errCacheBusy", err) | ||
| } | ||
| if err := a.Downgrade(); err != nil { | ||
| t.Fatalf("Downgrade: %v", err) | ||
| } | ||
| b, err := acquireFlock(path, syscall.LOCK_SH|syscall.LOCK_NB) | ||
| if err != nil { | ||
| t.Fatalf("shared after downgrade: %v, want success", err) | ||
| } | ||
| defer b.Close() | ||
| if _, err := acquireFlock(path, syscall.LOCK_EX|syscall.LOCK_NB); !errors.Is(err, errCacheBusy) { | ||
| t.Fatalf("exclusive after downgrade err = %v, want errCacheBusy", err) | ||
| } | ||
| } | ||
|
|
||
| // TestAcquireFlockUnlinkRace pins the verify-retry: when a sweeper unlinks | ||
| // the lock file while another process still holds a lock on the orphaned | ||
| // inode, a fresh acquire must land on the recreated file -- not block on or | ||
| // share fate with the orphan. | ||
| func TestAcquireFlockUnlinkRace(t *testing.T) { | ||
| path := filepath.Join(t.TempDir(), "run.lock") | ||
| orphan, err := acquireFlock(path, syscall.LOCK_EX) | ||
| if err != nil { | ||
| t.Fatal(err) | ||
| } | ||
| defer orphan.Close() | ||
| // Simulate the sweeper's RemoveAll of the bundle: the path is gone while | ||
| // the orphan's lock is still held on the old inode. | ||
| if err := os.Remove(path); err != nil { | ||
| t.Fatal(err) | ||
| } | ||
| fresh, err := acquireFlock(path, syscall.LOCK_EX|syscall.LOCK_NB) | ||
| if err != nil { | ||
| t.Fatalf("acquire after unlink: %v, want success on recreated file", err) | ||
| } | ||
| defer fresh.Close() | ||
| } | ||
|
|
||
| func TestFlockCloseIdempotentAndNilSafe(t *testing.T) { | ||
| path := filepath.Join(t.TempDir(), "run.lock") | ||
| a, err := acquireFlock(path, syscall.LOCK_EX) | ||
| if err != nil { | ||
| t.Fatal(err) | ||
| } | ||
| if err := a.Close(); err != nil { | ||
| t.Fatal(err) | ||
| } | ||
| if err := a.Close(); err != nil { | ||
| t.Fatalf("second Close: %v, want nil", err) | ||
| } | ||
| var nilLock *flockFile | ||
| if err := nilLock.Close(); err != nil { | ||
| t.Fatalf("nil Close: %v, want nil", err) | ||
| } | ||
| } | ||
|
|
||
| func TestBundleLockPaths(t *testing.T) { | ||
| if got := attachLockPath("/store/cs/sha256/ab"); got != "/store/cs/sha256/ab/attach.lock" { | ||
| t.Fatalf("attachLockPath = %q", got) | ||
| } | ||
| if got := runLockPath("/store/cs/sha256/ab"); got != "/store/cs/sha256/ab/run.lock" { | ||
| t.Fatalf("runLockPath = %q", got) | ||
| } | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.