Timeout Patterns and Partial Results in Java 21 | Engineering Notes

engnotes.devengnotes.dev

Was this article helpful?

to mark as helpful

Enjoyed this article?

Get more engineering insights delivered weekly.

Comments

to join the discussion

Note This article uses the Java 21 preview structured concurrency API (StructuredTaskScope, JEP 453). API shape changed in later previews. See Part 9 for Java 21 -> Java 25 migration guidance. Compile and run with --enable-preview.

Why Timeout Design Matters

In distributed systems, timeouts are normal, not exceptional. The important design question is whether timeout handling is:

all-or-nothing, or
able to return useful partial data.

Structured concurrency gives you clearer control over both behaviors.

Baseline: All-or-Nothing Timeout

java

public <T> T runInScopeWithTimeout(Callable<T> task, Duration timeout) throws Exception {
    Instant deadline = Instant.now().plus(timeout);

    try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
        var future = scope.fork(task);
        scope.join();

        if (Instant.now().isAfter(deadline)) {
            throw new TimeoutException("Operation exceeded timeout: " + timeout);
        }

        scope.throwIfFailed();
        return future.get();
    }
}

This is straightforward and often correct for endpoints where all fields are mandatory.

Partial Results Pattern (Java 21 Preview)

When some sections are optional, return what completed before the deadline and mark missing sections explicitly.

java

public <T> List<Optional<T>> executeWithPartialResults(List<Callable<T>> tasks, Duration timeout)
        throws InterruptedException {
    Instant deadline = Instant.now().plus(timeout);

    try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
        List<StructuredTaskScope.Subtask<T>> subtasks = new ArrayList<>();
        for (Callable<T> task : tasks) {
            subtasks.add(scope.fork(task));
        }

        try {
            scope.joinUntil(deadline);
        } catch (TimeoutException ignored) {
            // Deadline reached: return what is available.
        }

        // Use Optional to distinguish missing vs null results.
        List<Optional<T>> results = new ArrayList<>(subtasks.size());
        for (var subtask : subtasks) {
            if (subtask.state() == StructuredTaskScope.Subtask.State.SUCCESS) {
                results.add(Optional.of(subtask.get()));
            } else {
                results.add(Optional.empty());
            }
        }

        scope.shutdown(); // Stop unfinished work when returning partial data.
        return results;
    }
}

Optional streaming progress can be added with a callback:

For simple cases, poll subtask.state() after joinUntil(...).

java

@FunctionalInterface
public interface ProgressCallback<T> {
    // Optional: for streaming progress; simple polling in Java 21 preview.
    void onProgress(int taskIndex, T result);
}

public void executeWithProgressCallback(List<Callable<T>> tasks,
                                        ProgressCallback<T> callback)
        throws InterruptedException {

    try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
        List<StructuredTaskScope.Subtask<T>> subtasks = new ArrayList<>();

        for (Callable<T> task : tasks) {
            subtasks.add(scope.fork(task));
        }

        boolean[] completed = new boolean[tasks.size()];
        int totalCompleted = 0;
        long startTime = System.currentTimeMillis();
        long timeout = 10000;

        while (totalCompleted < tasks.size() &&
               (System.currentTimeMillis() - startTime) < timeout) {

            try {
                scope.joinUntil(Instant.now().plusMillis(10));
            } catch (TimeoutException e) {

            }

            for (int i = 0; i < subtasks.size(); i++) {
                if (!completed[i] &&
                        subtasks.get(i).state() == StructuredTaskScope.Subtask.State.SUCCESS) {

                    completed[i] = true;
                    totalCompleted++;
                    try {
                        callback.onProgress(i, subtasks.get(i).get());
                    } catch (Exception e) {

                    }
                } else if (!completed[i] &&
                        subtasks.get(i).state() == StructuredTaskScope.Subtask.State.FAILED) {
                    completed[i] = true;
                    totalCompleted++;
                }
            }
        }

        try {
            scope.join();
        } catch (Exception e) {

        }
    } catch (Exception e) {
        throw new RuntimeException("Progressive execution timeout", e);
    }
}

This keeps timeout behavior explicit and avoids hidden stale work after response completion.

Choosing Full Failure vs Partial Results

Use full failure when:

response correctness requires all fields,
downstream decisions depend on complete data,
partial responses are more dangerous than retries.
Example: core identity/profile data that drives authentication or authorization.

Use partial responses when:

some fields are optional/enrichment,
degraded UX is acceptable,
you can clearly signal missing sections.
Example: recommendation or personalization enrichments.

Java 21 Preview Limitations

joinUntil(...) timeout does not automatically define your response policy; you still need to decide whether to fail, return partials, or retry.
ShutdownOnFailure is fail-fast by exceptions, not business semantics; for optional sections, avoid converting normal absence into hard failures.
In timeout code paths, remember scope.shutdown() when you choose to return early. In Java 21 experiments, forgetting explicit shutdown() after timeout was a frequent source of leaked work.
Keep response contracts explicit; include metadata for missing sections instead of silent null handling.

Test Cases to Add

For timeout-sensitive endpoints, test at least:

All dependencies fast.
One dependency slow.
One dependency throws.
Multiple dependencies slow.
Deadline reached with partial success.

Include assertions on cancellation behavior and late work cleanup.

Build and Runtime Reminder

Structured concurrency is preview in Java 21.

bash

javac --release 21 --enable-preview ...
java --enable-preview ...

Previous: Part 1 - Introduction to Structured Concurrency in Java 21
Next: Part 3 - Conditional Cancellation and Circuit Breaker Patterns
Code Repository: project-loom

Resources