Adopt swift-subprocess when running unit tests

[owenv]

Alternative to #9191

Begin adopting swift-subprocess, beginning with test execution, in an effort to standardize on one implementation for process output streaming across platforms.

This required a bit of additional refactoring of the parallel test runner to make it async

Depends on swiftlang/swift#84947

[jakepetroules]

I think this is unsafe. Because deregister is synchronous, it may return before the cancellation block has finished executing, meaning teardown could be called when the process is already dead. It is NOT safe to use execution once the body block has returned, as the pid may have already been recycled or the pidfd may have become invalid. This is one reason why Execution would probably ideally be ~Escapable...

Subprocess.run already response to Swift Concurrency task cancellation, so you'd want something like this:

extension Cancellator {
    func run<T: Sendable>(_ block: @escaping @Sendable () async throws -> T) async throws -> T {
        let task = Task { try await block() }
        let token = register(name: "Test Execution", handler: { _ in task.cancel() })
        defer {
            token.map { deregister($0) }
        }
        return try await task.value
    }
}

try await cancellator.run {
    try await Subprocess.run(...)
}

That said, do you even need the cancellator, or is cooperative cancellation on its own, enough?

[owenv]

fixed. I think we do need the explicit cancellation here to ensure tests can be interrupted interactively

[jakepetroules]

nit but also kinda seriously: I'd either use an AsyncStream or at least use a LockedValue or similar abstraction.

[owenv]

I copied this in the refactor, but looking at it more closely this lock seems unnecessary, I deleted it

[bkhouri]

Can we refactor the SwiftTestCommand so we can write 'small' or 'medium' tests, that is, we test the modified functions/code in isolation, as opposed to relying on 'large' (ie: end-to-end tests)?

[bkhouri]

Meant to mark this as required changed instead of approve

[owenv]

Depends on swiftlang/swift-subprocess#201 for the CMake build

[owenv]

Can we refactor the SwiftTestCommand so we can write 'small' or 'medium' tests, that is, we test the modified functions/code in isolation, as opposed to relying on 'large' (ie: end-to-end tests)?

This PR isn intended to be NFC so I've been relying on the existing tests so far

[owenv]

@swift-ci test

[bkhouri]

Can we refactor the SwiftTestCommand so we can write 'small' or 'medium' tests, that is, we test the modified functions/code in isolation, as opposed to relying on 'large' (ie: end-to-end tests)?

This PR isn intended to be NFC so I've been relying on the existing tests so far

I understand we are relying on existing tests, but we really should use the Test Pyramid to validate the code changes instead of relying on all the tests. Although the code changes may be exercises via existing tests, they tests to no guarantee we will exercise all the code paths. As such, we should try to write tests at different layers of the Test Pyramid.