Livsy Code

Deterministic Testing for iOS: Reproducing the Failures That Matter

Greetings, traveler!

A profile screen works when the backend responds immediately, but occasionally gets stuck in a loading state when a request times out. An engineer asks an AI agent to fix it. The agent adds a timeout message, runs the tests, and reports success. The trouble is that the tests never produced a timeout: they passed because the server happened to respond.

To verify the change, the engineer needs a way to put the application in the failing state on demand. That means controlling the response to a request, knowing when the application has processed it, and starting each run from a known state. The same setup helps an agent investigate a defect without repeatedly waiting for a rare production condition.

The examples below use a hypothetical iOS application with a profile endpoint. They are intended to show where to introduce control into an existing networking layer; the exact response model and error handling should follow your API contract.

What makes a test reproducible?

A test is reproducible when the same relevant inputs and initial conditions consistently lead to the same observable outcome. In an application, those inputs extend well beyond the arguments passed to a function. They include persisted data, the current date, the order in which asynchronous work completes, network responses, and even another test running at the same time.

A test becomes flaky when it passes and fails without a relevant change to the behavior being tested. Common causes include shared state, live services, implicit ordering, unseeded random values, and assertions that run before asynchronous work finishes. A flaky test can also reveal a real race, so a failure should be investigated before it is dismissed as a test problem.

Complete determinism is rarely practical for an entire application. The useful goal is narrower: control the factors that affect a particular assertion and make any remaining variability visible. A controlled HTTP response can verify error mapping, for example, while a separate integration check verifies that the actual service still honors its contract.

Choose the boundary you need to control

A networking test can substitute behavior at several points:

BoundaryWhat remains realUseful forWhat it misses
API client or repositoryCalling code and state transitionsFast tests of loading, retry decisions, and UI stateRequest construction and response decoding
URLProtocol in a configured URLSessionMuch of the app’s request construction and response handlingHTTP status mapping, headers, decoding, transport errorsA real socket, DNS, TLS, proxies, and backend behavior
HTTP mock serverThe app’s HTTP client and a real connection to the mockCross-process flows, UI tests, shared scenarios, request logsThe real backend and, unless specifically configured, production network conditions
Test backendClient and deployed serviceContract and integration checksComplete control over failures and other sources of variation

These are choices about scope, not strict test categories. A test using URLProtocol may be called a unit or integration test depending on what it exercises. What matters is stating which components are real and which are substituted.

For a view model that needs to show a signed-out state, a small fake repository may be enough. To check whether a 401 response is translated into the correct domain error, inject a session using a custom protocol. To check the complete screen flow in a separately launched app, an HTTP server gives both the app and the test runner a common endpoint.

Make the session replaceable

Many existing networking layers construct URLSession.shared inside the client. That makes it difficult to provide a session with test behavior. A small initializer change is often enough:

import Foundation

final class ProfileAPI {
    private let session: URLSession
    private let baseURL: URL

    init(baseURL: URL, session: URLSession = .shared) {
        self.baseURL = baseURL
        self.session = session
    }

    func loadProfile() async throws -> Profile {
        let url = baseURL.appendingPathComponent("profile")
        let (data, response) = try await session.data(from: url)

        guard let response = response as? HTTPURLResponse else {
            throw ProfileError.invalidResponse
        }

        switch response.statusCode {
        case 200:
            return try JSONDecoder().decode(Profile.self, from: data)
        case 401:
            throw ProfileError.sessionExpired
        case 429:
            throw ProfileError.rateLimited
        case 500...599:
            throw ProfileError.serverUnavailable
        default:
            throw ProfileError.unexpectedStatus(response.statusCode)
        }
    }
}

Profile and ProfileError represent types already owned by the application. The initializer preserves the production default while allowing tests to provide another session. In a real client, status handling, authentication and decoding may live in different objects. Keep those boundaries if they already serve the project.

One detail deserves attention: URLSession does not automatically treat an HTTP 401 or 500 as a thrown networking error. The request can complete successfully at the transport level while the response carries an unsuccessful status. The client must interpret that status itself.

Exercise HTTP handling with URLProtocol

A custom URLProtocol can intercept requests made through a session configured to use it. The following deliberately narrow example intercepts HTTP and HTTPS requests in its test session. It returns 401 only for the expected request and fails every other intercepted request. It avoids mutable global handlers, so parallel tests cannot overwrite one another’s response configuration.

import Foundation

final class UnauthorizedProfileProtocol: URLProtocol {
    override class func canInit(with request: URLRequest) -> Bool {
        guard let scheme = request.url?.scheme?.lowercased() else { return false }
        return scheme == "http" || scheme == "https"
    }

    override class func canonicalRequest(for request: URLRequest) -> URLRequest {
        request
    }

    override func startLoading() {
        guard let url = request.url,
              url.scheme == "https",
              url.host == "profile.test",
              url.path == "/profile",
              url.query == nil,
              (request.httpMethod ?? "GET") == "GET" else {
            client?.urlProtocol(self, didFailWithError: URLError(.unsupportedURL))
            return
        }

        guard let response = HTTPURLResponse(
                  url: url,
                  statusCode: 401,
                  httpVersion: "HTTP/1.1",
                  headerFields: ["Content-Type": "application/json"]
              ) else {
            client?.urlProtocol(self, didFailWithError: URLError(.badURL))
            return
        }

        let body = Data(#"{"error":"session_expired"}"#.utf8)
        client?.urlProtocol(self, didReceive: response, cacheStoragePolicy: .notAllowed)
        client?.urlProtocol(self, didLoad: body)
        client?.urlProtocolDidFinishLoading(self)
    }

    override func stopLoading() {}
}

The protocol answers immediately and has no pending work to cancel. If you add delayed or asynchronous responses, stopLoading() must cancel that work and prevent callbacks after cancellation. A production-quality test helper also needs synchronization around mutable scenario state. Merely marking a shared handler as safe for concurrency does not make it safe.

The test creates a session specifically for this scenario:

import XCTest

final class ProfileAPITests: XCTestCase {
    func testUnauthorizedResponseExpiresSession() async throws {
        let configuration = URLSessionConfiguration.ephemeral
        configuration.protocolClasses = [UnauthorizedProfileProtocol.self]
        let session = URLSession(configuration: configuration)
        defer { session.invalidateAndCancel() }

        let api = ProfileAPI(
            baseURL: try XCTUnwrap(URL(string: "https://profile.test")),
            session: session
        )

        do {
            _ = try await api.loadProfile()
            XCTFail("Expected the client to reject an expired session")
        } catch ProfileError.sessionExpired {
            // Expected domain error.
        } catch {
            XCTFail("Unexpected error: \(error)")
        }
    }
}

This assumes ProfileError is visible to the test target. A wrong method, host, path or query produces a transport error and fails this particular test. This does not assert headers or bodies; add explicit checks for those when they are part of the contract. The interception applies only to this configured session, not to other sessions or networking APIs in the process. The test checks the client’s public result; it does not prove that a login screen appears. That requires a separate test at the view model or UI boundary.

For a larger suite, use a test harness that isolates each scenario and captures requests safely. Assert the method, path and relevant headers when they are part of the behavior under test. Avoid a single unsynchronized static closure that every test replaces: concurrent runs can return another test’s response and produce failures that are difficult to explain. Custom protocol classes are also unavailable for background sessions, so background transfers need a different strategy.

Model failures at the right layer

Different failures can look similar on screen while exercising different code paths. A useful scenario set for the profile request might include:

ConditionControlled inputExample assertion
Expired sessionHTTP 401 with the expected error bodyClient returns a session-expired error; the app follows its authentication policy
Rate limitHTTP 429 and, if supported by the API, Retry-AfterClient exposes the limit or applies the agreed retry policy
Server failureHTTP 500App shows a recoverable error and does not decode it as a profile
Invalid payloadHTTP 200 with malformed or incompatible JSONDecoding failure is surfaced and loading finishes
Transport timeoutURLError(.timedOut) from a controlled transportTimeout-specific handling runs
Slow responseA response held until the test releases it, or delayed by a serverLoading state is visible before completion and clears afterward

An HTTP 408 is a response from a server; a URLError.timedOut is a transport failure. They may lead to the same message, but they should not be treated as the same test input. A delayed server response can exercise an actual session timeout if the configured timeout expires, but it introduces elapsed time and may be sensitive to the environment. timeoutIntervalForRequest limits how long a task waits for additional data and resets when data arrives; timeoutIntervalForResource limits the overall resource transfer. The correct setting depends on the behavior being tested. Injecting a timeout error is faster for testing error handling; a separate integration test can cover the actual timeout configuration when that behavior matters.

For retry logic, verify more than the final result. Record how many requests were made, whether retryable failures were retried, and whether a non-retryable response stopped the sequence. Define limits and cancellation behavior explicitly. Retrying a state-changing request also requires an API contract that makes retries safe, such as an appropriate idempotency mechanism.

Use an HTTP mock server for complete flows

A mock server runs outside the app and listens for HTTP requests. Point a test build at its address, configure a scenario, then launch the application as the test normally would. This is useful when a UI test runs in a separate process or when the same scenario should be available to a developer, a QA engineer and an agent.

A scenario for the profile flow can specify a 401 response and an error body. The test still has to establish the initial app state, such as a signed-in user, and then assert the observable result. A request log can show that the app called /profile; it cannot by itself prove that the user reached the login screen.

For a small local experiment, the following server returns a selected status for GET /profile. It binds to loopback and deliberately returns a diagnostic 404 for other paths:

# mock_profile.py
import json
import os
from http.server import BaseHTTPRequestHandler, HTTPServer

status = int(os.environ.get("PROFILE_STATUS", "401"))
payloads = {
    200: {"id": "test-user", "name": "Alex"},
    401: {"error": "session_expired"},
    429: {"error": "rate_limited"},
    500: {"error": "server_error"},
}
if status not in payloads:
    raise SystemExit("Unsupported PROFILE_STATUS")

class Handler(BaseHTTPRequestHandler):
    def do_GET(self):
        if self.path != "/profile":
            self.send_error(404)
            return
        body = json.dumps(payloads[status]).encode("utf-8")
        self.send_response(status)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        self.wfile.write(body)

HTTPServer(("127.0.0.1", 8787), Handler).serve_forever()

Start it with PROFILE_STATUS=401 python3 mock_profile.py. In a separate terminal, curl -i http://127.0.0.1:8787/profile should return a 401 with the JSON body; the server prints received requests. The 200 fixture is illustrative and must match the application’s Profile model before it can test decoding. Use another supported status to select a different response. This small server has no built-in request assertions, scenario isolation or test lifecycle management.

To exercise the screen, configure the test build’s base URL as http://127.0.0.1:8787, establish a signed-in test state, launch the app and open Profile. Assert the app’s expected authentication behavior in the UI test, then inspect the server log to confirm the request arrived. Stop the server after the run. For CI, start it as a managed subprocess, wait for readiness, capture its output and guarantee cleanup on failure. Use separate ports and state for concurrent tests. A simulator running on the same Mac can use the loopback address; a physical device needs a reachable address and a server bound accordingly.

When choosing a server, check how it matches requests and what happens when no route matches. Some tools match method and path only; others can match headers, query parameters or body content. A permissive fallback may make an incorrectly constructed request look successful. Prefer an explicit failure for unexpected requests and inspect logs when a scenario fails.

For parallel runs, isolate server ports, scenario state, logs and app data. Ensure the server is ready before launching the app and stop it after the test. A local server on the development Mac may be reachable from the simulator through a host address; a physical device requires a reachable network address and appropriate configuration. Plain HTTP may also require an App Transport Security exception. Keep test endpoints and exceptions out of release configuration, and never point a test at production by accident.

A mock server can introduce latency and status codes, but that does not automatically reproduce DNS failures, TLS errors, packet loss or an operating system changing connectivity. Use a suitable transport fake, proxy or network-conditioning tool for the specific behavior you want to exercise.

Control completion instead of guessing how long it takes

An asynchronous test often starts with a sleep:

try await Task.sleep(nanoseconds: 1_000_000_000)
XCTAssertEqual(viewModel.state, .loaded)

The assertion is tied to an elapsed duration, not to the operation finishing. It can run too early on a busy CI worker and waste time when the operation finishes quickly.

When the method under test is async, await it directly. For a callback-based API, fulfill an XCTest expectation from the callback and await that expectation with a bounded timeout. The timeout is a failure bound, not the mechanism that makes the work finish.

Testing an intermediate loading state requires another point of control. For example, inject a test repository that announces when its request starts and suspends until the test releases a response. A test can then follow this sequence:

  1. Start loading in a task.
  2. Wait until the fake confirms that the request has started.
  3. Assert the loading state.
  4. Release the response and await the task’s completion.
  5. Assert the final state.

The fake must synchronize its state and resume each continuation at most once, including when cancellation occurs. A continuation that is never resumed can hang a test; a test timeout and cleanup path are still needed. Awaiting a callback or task also does not automatically guarantee that every downstream UI update has completed. Wait for the specific observable condition when the update happens separately.

The following compact example shows that approach with a callback-based service. All types share the main actor, so the test controls the completion without sleeping or coordinating unsynchronized state:

import XCTest

@MainActor
protocol ProfileLoading {
    func load(completion: @escaping (Result<String, Error>) -> Void)
}

@MainActor
final class ProfileModel {
    enum State: Equatable {
        case idle, loading, loaded(String), failed
    }

    private let loader: any ProfileLoading
    private(set) var state: State = .idle

    init(loader: any ProfileLoading) {
        self.loader = loader
    }

    func load() {
        state = .loading
        loader.load { [weak self] result in
            guard let self else { return }
            switch result {
            case .success(let name): self.state = .loaded(name)
            case .failure: self.state = .failed
            }
        }
    }
}

@MainActor
final class ControlledLoader: ProfileLoading {
    private var completion: ((Result<String, Error>) -> Void)?

    func load(completion: @escaping (Result<String, Error>) -> Void) {
        precondition(self.completion == nil)
        self.completion = completion
    }

    func succeed(with name: String) {
        guard let completion else { preconditionFailure("No pending request") }
        self.completion = nil
        completion(.success(name))
    }
}

final class ProfileModelTests: XCTestCase {
    @MainActor
    func testLoadingThenSuccess() {
        let loader = ControlledLoader()
        let model = ProfileModel(loader: loader)

        model.load()
        XCTAssertEqual(model.state, .loading)

        loader.succeed(with: "Alex")
        XCTAssertEqual(model.state, .loaded("Alex"))
    }
}

This fake models one pending request and invokes its callback only when the test releases it. The production implementation must deliver its callback on the main actor to meet the protocol’s isolation contract. For an async service, use an appropriately synchronized test double and await explicit start and completion signals; do not assume that starting a Task means it has already reached its first suspension point. Adapt the example to the project’s deployment target and Swift concurrency settings.

Time-dependent behavior needs its own seam. Inject a clock or a small application-specific delay abstraction into retry, debounce or expiration logic. In a test, use a controllable implementation that lets the test wait until a timer is registered and then advance time. This checks a ten-second backoff without spending ten real seconds on every run. Keep a few integration tests with real scheduling when you need confidence in the wiring, and recognize that a fake clock does not reproduce every scheduler or network race.

Start every run from a known state

A reliable HTTP response is only one part of a scenario. A previous run can leave an authentication token, cached profile, database row or pending task behind. The next run may then skip the request entirely.

Give each test a clean or isolated store, predictable preferences and a known account state. For UI tests, provide a test launch configuration that selects the environment and seeds the necessary data before the first screen appears. Reset keychain items, databases and caches through explicit test-support paths where appropriate. A fresh application launch alone does not guarantee that persistent state has been cleared.

Resetting shared state is not enough when tests run concurrently. Give each test its own database, temporary directory and server scenario where practical. If a dependency cannot be isolated, serialize the affected tests and document why. Restore modified process settings and tear down sessions or servers even after failures.

Fixtures need maintenance too. A small payload with only required fields is easy to understand, but it can miss compatibility problems with optional fields, pagination or unexpected values. Keep representative examples of important API variants and review them when the contract changes.

Record and replay with care

Recording traffic can turn an intermittent issue into a repeatable local scenario. Capture the relevant request and response, remove sensitive information, and replay the result while investigating. Recordings are especially helpful when the payload is difficult to construct by hand.

Replay has limits. Tokens, timestamps, identifiers and request ordering may differ between runs. Matching rules must distinguish the request you intend to reproduce from unrelated traffic. A captured successful response also says little about failures that were never recorded. Review and sanitize recordings before committing them: headers and bodies may contain credentials or personal data.

Recorded traffic can become stale just like handwritten fixtures. Pair replay with contract checks against a current schema or a controlled test service, and update recordings deliberately when the API changes. A replayed response proves how the client handles that recording; it does not prove the backend still produces it.

Let an agent use the same scenarios

An agent can help set up scenarios, add focused tests and interpret failures, provided it can observe the inputs and results. A useful task for the profile timeout might specify:

Reproduce the profile request timing out. Verify that loading ends and the user can retry. Use the existing test infrastructure, report the command and result, and identify any behavior that the test does not cover.

For the opening timeout defect, the agent can locate the client and screen state handling, then configure a fake transport to return URLError(.timedOut). A focused test should verify that loading ends and the retry action remains available. The agent runs that test before the fix and inspects whether it fails at the intended assertion; after the fix, it reruns the test and relevant neighboring checks. A separate mock-server UI flow can verify screen wiring, while a real timeout configuration check remains a separate integration concern. A compile error or broken fixture is not evidence that the desired behavior was reproduced. Review the failure before accepting it as a useful starting point.

For a UI scenario, preserve the scenario configuration, test result and relevant request log together. If a screenshot or UI assertion is part of the check, record it separately from the HTTP evidence. This makes it possible to distinguish a request that was never sent from a correctly handled request followed by an unrelated UI failure.

Do not give the agent unrestricted access to real accounts or production data merely to make a test convenient. Use test accounts and synthetic payloads. A scenario should also identify what it does not simulate, so the agent does not mistake a local pass for proof of production behavior.

Keep a path back to reality

Controlled tests are most useful when they run frequently and fail for understandable reasons. They do not replace checks against an actual service. A balanced suite can use focused tests for error mapping and state transitions, a smaller number of mock-server UI flows, and contract or integration checks against a managed test environment.

Measure whether the setup helps: track unexpected failures, time spent diagnosing them, and whether a reported defect can be reproduced from a saved scenario. If the team must edit several unrelated fixtures or restart shared services to reproduce a single 401, the test environment itself needs attention.

Start with one failure that has already cost time. Make its input controllable, assert the visible behavior, and keep the scenario small enough for another engineer to understand. Once that works reliably, extend the same approach to the next failure mode.

It might be interesting: