Greetings, traveler!

An AI agent finishes a task and reports: “Fixed the concurrency issue. All tests pass.” That sounds reassuring, but it leaves several questions unanswered. Which tests ran? Did they exercise the failure? Was the final version of the code tested, or did the agent make another edit afterward? Did the command succeed, or did it fail before the tests started?

These questions matter because an agent can produce a plausible explanation without having enough information to support it. The same is true of any engineering report, but agents can generate and revise code quickly enough that the gap between a conclusion and its evidence is easy to miss.

A useful verification process connects each important claim to an observation and records what remains uncertain.

Start with the claim

Suppose a teammate reports a bug: when a user refreshes an activity feed twice, an older response can replace a newer one. An agent changes the loading code and says the issue is resolved.

There are several distinct claims hidden in that sentence:

• The modified target compiles.
• The original sequence no longer produces the stale result.
• Other loading behavior still works.
• The solution handles the relevant cancellation and error cases.
• The evidence applies to the version of the code under review.

A build can support the first claim. It cannot establish the others. A targeted test can exercise the original sequence, but a passing test tells us only about the behavior it actually checks and the conditions it creates.

This distinction is useful even before an agent starts working. Instead of asking it to “fix and test the bug,” describe the expected outcome: if request A starts first, request B starts second, and A finishes last, the UI must continue to display B’s result. That statement gives the agent a concrete behavior to investigate and gives the reviewer something to verify.

Commands produce observations, not conclusions

An agent may say it ran xcodebuild test. That is an action. To understand the outcome, we need the command’s exit status, the test summary, and enough context to know which scheme, destination, and tests were selected.

A zero exit status is stronger than “the command was started,” but it still needs interpretation. A test plan might exclude the relevant target. A filtered run might execute one test while the report implies that the entire suite passed. A command can also finish without exercising the specific race the change was meant to address.

The same applies to artifacts:

• A build log shows what happened during a build, not whether the feature behaves correctly at runtime.
• An XCTest result bundle records a test run, but its value depends on which tests ran and what they asserted.
• A screenshot can show a layout at one moment; it cannot establish that VoiceOver navigation is usable.
• A sanitizer report can expose certain runtime issues on executed paths; a clean run does not prove the absence of concurrency bugs.

A good report connects these observations to appropriately narrow conclusions. “The targeted regression test passed on the iPhone simulator” is more useful than “the race is impossible now.”

A concurrency bug that compiles successfully

Consider this simplified SwiftUI model. The example is hypothetical; the snippets illustrate the behavior to test, not a measurement from a production app.

struct Activity: Equatable, Sendable {
    let title: String
}

protocol ActivityLoading: Sendable {
    func load() async throws -> [Activity]
}

@MainActor
final class ActivityFeedModel {
    private let loader: any ActivityLoading
    private(set) var activities: [Activity] = []

    init(loader: any ActivityLoading) {
        self.loader = loader
    }

    func refresh() async throws {
        let loaded = try await loader.load()
        activities = loaded
    }
}

struct Activity: Equatable, Sendable {
    let title: String
}

protocol ActivityLoading: Sendable {
    func load() async throws -> [Activity]
}

@MainActor
final class ActivityFeedModel {
    private let loader: any ActivityLoading
    private(set) var activities: [Activity] = []

    init(loader: any ActivityLoading) {
        self.loader = loader
    }

    func refresh() async throws {
        let loaded = try await loader.load()
        activities = loaded
    }
}

The state is isolated to the main actor, so individual accesses are protected by that isolation. The logic can still be wrong. While the first call is suspended at await, a second call can start. If the newer request finishes first and the older request finishes later, the older result overwrites it.

This is an ordering problem, not necessarily a memory data race. Strict concurrency checking and Thread Sanitizer are valuable, but neither replaces a test of the intended ordering.

One possible fix is to associate each refresh with an identifier. Only the most recently started refresh may update the feed. The following class replaces the previous ActivityFeedModel; keep Activity and ActivityLoading unchanged.

import Foundation

@MainActor
final class ActivityFeedModel {
    private let loader: any ActivityLoading
    private var latestRequestID: UUID?
    private(set) var activities: [Activity] = []

    init(loader: any ActivityLoading) {
        self.loader = loader
    }

    func refresh() async throws {
        let requestID = UUID()
        latestRequestID = requestID

        let loaded = try await loader.load()
        try Task.checkCancellation()
        guard latestRequestID == requestID else { return }

        activities = loaded
    }
}

import Foundation

@MainActor
final class ActivityFeedModel {
    private let loader: any ActivityLoading
    private var latestRequestID: UUID?
    private(set) var activities: [Activity] = []

    init(loader: any ActivityLoading) {
        self.loader = loader
    }

    func refresh() async throws {
        let requestID = UUID()
        latestRequestID = requestID

        let loaded = try await loader.load()
        try Task.checkCancellation()
        guard latestRequestID == requestID else { return }

        activities = loaded
    }
}

This example uses Foundation for UUID. The exact policy is a product decision: here, a newer refresh makes an older one ineligible to update the feed even if the newer request fails or is cancelled. Errors still propagate to the caller; a real view model must also prevent an older error from replacing a newer loading or error state. If the desired policy differs, the implementation and tests must reflect it.

The cancellation check prevents a task already cancelled at that point from applying its result. It is not a guarantee that cancellation cannot occur immediately afterward, and it does not cancel network work automatically. Those behaviors need their own design and tests when they matter.

Reproduce the ordering deliberately

A regression test for this issue should control the completion of both requests. It should not depend on one network call happening to be slower than another or on a particular delay chosen with Task.sleep.

The following XCTest uses an actor as a controlled loader. Each request stays suspended until the test supplies its result. Expectations confirm when each request reaches the loader and when each refresh finishes. Add the test to the app’s test target and import the module that contains the model.

import XCTest
@testable import MyApp

private actor ControlledActivityLoader: ActivityLoading {
    private var pending: [Int: CheckedContinuation<[Activity], Error>] = [:]
    private var requestCount = 0
    private var isClosed = false
    private let started: [XCTestExpectation]

    init(started: [XCTestExpectation]) {
        self.started = started
    }

    func load() async throws -> [Activity] {
        guard !isClosed else { throw CancellationError() }
        let index = requestCount
        requestCount += 1

        return try await withCheckedThrowingContinuation { continuation in
            pending[index] = continuation
            if index < started.count {
                started[index].fulfill()
            }
        }
    }

    func complete(_ index: Int, with activities: [Activity]) {
        pending.removeValue(forKey: index)?.resume(returning: activities)
    }

    func close() {
        isClosed = true
        let waiting = pending.values
        pending.removeAll()
        for continuation in waiting {
            continuation.resume(throwing: CancellationError())
        }
    }
}

@MainActor
final class ActivityFeedModelTests: XCTestCase {
    func testOlderResponseDoesNotReplaceNewerResponse() async {
        let startedA = expectation(description: "Request A started")
        let startedB = expectation(description: "Request B started")
        let finishedA = expectation(description: "Refresh A finished")
        let finishedB = expectation(description: "Refresh B finished")
        let loader = ControlledActivityLoader(started: [startedA, startedB])
        let model = ActivityFeedModel(loader: loader)

        let first = Task {
            do {
                try await model.refresh()
            } catch is CancellationError {
                // Cleanup after a timeout can cancel a pending request.
            } catch {
                XCTFail("Refresh A failed: \(error)")
            }
            finishedA.fulfill()
        }
        await fulfillment(of: [startedA], timeout: 2)

        let second = Task {
            do {
                try await model.refresh()
            } catch is CancellationError {
                // Cleanup after a timeout can cancel a pending request.
            } catch {
                XCTFail("Refresh B failed: \(error)")
            }
            finishedB.fulfill()
        }
        await fulfillment(of: [startedB], timeout: 2)

        await loader.complete(1, with: [Activity(title: "New")])
        await fulfillment(of: [finishedB], timeout: 2)
        await loader.complete(0, with: [Activity(title: "Old")])
        await fulfillment(of: [finishedA], timeout: 2)

        // Also release suspended requests if an expectation timed out.
        await loader.close()
        first.cancel()
        second.cancel()
        await first.value
        await second.value

        XCTAssertEqual(model.activities, [Activity(title: "New")])
    }
}

import XCTest
@testable import MyApp

private actor ControlledActivityLoader: ActivityLoading {
    private var pending: [Int: CheckedContinuation<[Activity], Error>] = [:]
    private var requestCount = 0
    private var isClosed = false
    private let started: [XCTestExpectation]

    init(started: [XCTestExpectation]) {
        self.started = started
    }

    func load() async throws -> [Activity] {
        guard !isClosed else { throw CancellationError() }
        let index = requestCount
        requestCount += 1

        return try await withCheckedThrowingContinuation { continuation in
            pending[index] = continuation
            if index < started.count {
                started[index].fulfill()
            }
        }
    }

    func complete(_ index: Int, with activities: [Activity]) {
        pending.removeValue(forKey: index)?.resume(returning: activities)
    }

    func close() {
        isClosed = true
        let waiting = pending.values
        pending.removeAll()
        for continuation in waiting {
            continuation.resume(throwing: CancellationError())
        }
    }
}

@MainActor
final class ActivityFeedModelTests: XCTestCase {
    func testOlderResponseDoesNotReplaceNewerResponse() async {
        let startedA = expectation(description: "Request A started")
        let startedB = expectation(description: "Request B started")
        let finishedA = expectation(description: "Refresh A finished")
        let finishedB = expectation(description: "Refresh B finished")
        let loader = ControlledActivityLoader(started: [startedA, startedB])
        let model = ActivityFeedModel(loader: loader)

        let first = Task {
            do {
                try await model.refresh()
            } catch is CancellationError {
                // Cleanup after a timeout can cancel a pending request.
            } catch {
                XCTFail("Refresh A failed: \(error)")
            }
            finishedA.fulfill()
        }
        await fulfillment(of: [startedA], timeout: 2)

        let second = Task {
            do {
                try await model.refresh()
            } catch is CancellationError {
                // Cleanup after a timeout can cancel a pending request.
            } catch {
                XCTFail("Refresh B failed: \(error)")
            }
            finishedB.fulfill()
        }
        await fulfillment(of: [startedB], timeout: 2)

        await loader.complete(1, with: [Activity(title: "New")])
        await fulfillment(of: [finishedB], timeout: 2)
        await loader.complete(0, with: [Activity(title: "Old")])
        await fulfillment(of: [finishedA], timeout: 2)

        // Also release suspended requests if an expectation timed out.
        await loader.close()
        first.cancel()
        second.cancel()
        await first.value
        await second.value

        XCTAssertEqual(model.activities, [Activity(title: "New")])
    }
}

Replace MyApp with the actual module name. The acknowledgement before starting B establishes the order in which the model enters refresh(). Waiting for B to finish before completing A establishes the response order. If an expectation times out, XCTest records a failure; closing the loader releases pending continuations and causes any later load to fail rather than wait indefinitely. The timeouts bound the waits and are not used to arrange the race.

With the original model, this sequence is expected to leave Old in the feed, so the final assertion should fail. With the identifier check, it should retain New. These are expected outcomes for the example, not results from a test run. Run the test against both versions in the project to verify them. Test target settings and concurrency diagnostics may require adapting the example to the project’s Swift and Xcode versions.

A single ordering test does not establish correctness for every interleaving. This test checks only successful responses. Additional tests should exercise error and cancellation paths if the app relies on them.

The targeted test is only one part of the review. Depending on the model’s responsibilities, further tests might cover a successful single refresh, a failed latest request, cancellation, and a third overlapping request. The reviewer should also inspect how loading and error states are updated. A correct array of activities alongside a stale error message would still be a broken experience.

Keep the evidence attached to the code

Test results describe a particular run in a particular environment. They do not automatically apply to later edits.

A practical report identifies the commit or working-tree state, the command, the destination, and the result. A commit identifies the tested code most clearly when the working tree is clean. If it contains uncommitted work, preserve both staged and unstaged changes and the contents of relevant untracked files. Record the Xcode version, test plan, resolved dependencies, destination, and other configuration needed to interpret or reproduce the run.

For example, a team can retain an Xcode result bundle and a plain-text log:

set -o pipefail

xcodebuild test \
  -workspace MyApp.xcworkspace \
  -scheme MyApp \
  -destination 'platform=iOS Simulator,id=SIMULATOR-UDID' \
  -only-testing:MyAppTests/ActivityFeedModelTests \
  -resultBundlePath "$PWD/TestResults/ActivityFeedModelTests.xcresult" \
  2>&1 | tee "$PWD/TestResults/activity-feed-tests.log"

set -o pipefail

xcodebuild test \
  -workspace MyApp.xcworkspace \
  -scheme MyApp \
  -destination 'platform=iOS Simulator,id=SIMULATOR-UDID' \
  -only-testing:MyAppTests/ActivityFeedModelTests \
  -resultBundlePath "$PWD/TestResults/ActivityFeedModelTests.xcresult" \
  2>&1 | tee "$PWD/TestResults/activity-feed-tests.log"

Create TestResults first, choose an installed simulator and a scheme that includes the test target, and use a new result-bundle path for each run. Replace the placeholders with values from the project. pipefail helps preserve a failing xcodebuild status when its output is piped through tee; without it, the pipeline may report the status of tee instead. Confirm the actual test count and failures in the output or result bundle, especially when using a filter.

If the agent changes the production code or a relevant test after this run, it should reassess and rerun the affected checks. A documentation-only edit may not invalidate a test result, but the reason for keeping it should be clear. CI results for the final commit are often a convenient reference; local results remain useful for diagnosis and checks that CI does not perform.

Artifacts also need sensible handling. Test logs can include account identifiers, server responses, or other sensitive data. Retain enough to make the result inspectable, and follow the project’s rules for access and redaction.

Define completion before the agent starts

A quality gate is a condition that must be satisfied before work moves to the next stage. It can be automated, reviewed by a person, or combine both.

For the feed bug, a team might agree that completion requires:

• The affected target builds with the project’s intended configuration.
• The ordering regression test passes on the final code.
• Relevant existing model tests pass.
• A reviewer checks the handling of stale results, errors, and cancellation.
• The final report identifies any scenario that was not checked.

These conditions should be proportionate to the task and decided with the team’s existing process in mind. A required check that fails, cannot run, or returns no usable result has not passed. An infrastructure failure may call for a retry or a different environment; it should not be translated into a successful test result.

It is also useful to distinguish blocking checks from non-blocking checks. A blocking check must pass, or be explicitly handled through the team’s exception process, before the agreed milestone. A non-blocking check can provide useful information without preventing that milestone. Its failure still deserves a visible record and, where appropriate, an owner or follow-up. An approved exception can allow work to proceed, but the check remains failed or unrun in the record; it does not become a pass.

Classification depends on the change. A manual check of a rare layout variation might be non-blocking for a small internal screen. A test covering a payment authorization failure should not quietly become optional because it is inconvenient to run. Define exceptions and their consequences before relying on them.

Match verification to the risk

Running every available test for every edit is expensive, and it does not guarantee that the right behavior was exercised. Choose checks based on what could fail, how serious that failure would be, and how confidently the change is understood.

These are starting points, not universal test plans. A seemingly small UI change can be high risk if it obscures a critical warning or changes an accessibility label. An authentication change can also affect storage, account switching, server contracts, and audit requirements. The scope of the change matters more than its line count.

The cost of verification includes developer attention, test maintenance, runtime, and investigation of failures. Start with checks that directly address the change, then add broader checks where the risk justifies them. Do not remove an existing required project check solely because a smaller set seems sufficient.

Know what automation cannot tell you

A passing suite can contain weak assertions, and a snapshot can faithfully preserve an incorrect design. A deterministic check may consistently test the wrong requirement.

Some questions need judgment: does an error message help the user recover? Does the new authentication behavior match the security policy? Is the additional abstraction worth maintaining? An agent can gather context and suggest answers, but its explanation is still a proposal to evaluate.

Human review has limits too. A reviewer cannot reliably inspect every execution path by reading a diff. Automated checks and review work best when each addresses a different source of uncertainty: tests exercise selected behavior, tooling catches specific classes of problems, and people assess requirements, tradeoffs, and evidence.

Ask for a report that can be checked

Compare the original completion message with a more useful example. The following report is illustrative; its results and paths are not from an actual test run.

Change: Prevent an older feed refresh from replacing a newer result.

Code under test: commit <commit-id> plus the attached working-tree diff.

Checks:
- ActivityFeedModelTests: passed, 6 tests on iPhone simulator.
  Command: <exact command>
  Result bundle: <artifact location>
- Affected app scheme build: passed.
  Command and log: <command and artifact location>
- Original implementation: ordering regression test failed as expected.

Not checked:
- Integration against the live service.
- VoiceOver behavior; the change does not modify accessibility code.

Remaining consideration:
- Confirm the intended behavior when the latest refresh fails while an
  earlier request is still in flight.

The report gives the reviewer a path from the change to the observations. It also exposes an unresolved product decision that a green test run could otherwise hide. If the intended failure behavior has already been specified and verified, the report can say so and identify the relevant test instead.

Make verification part of the task

Evidence-driven work does not require a separate platform or a long checklist for every change. Start with a clear expected behavior, choose a few checks that can detect meaningful failures, and ask the agent to report the results with their scope and limitations.

When a check fails, investigate the failure instead of repeatedly changing code until the output turns green. When a check cannot run, preserve that uncertainty. And when the code changes after verification, reconsider which results still apply.

Before accepting the change, another developer should be able to see which behaviors were checked, on which version of the code, and which risks remain open.