Series navigation
Code repositoryproject-loom
J
Written by
Jagdish Salgotra
Distributed systems, cloud-native architecture, and the JVM. mostly shipping, occasionally reading.
Was this article helpful? or email →anonymous · no account needed
Should a missed deadline fail the request or return what made it back in time? Three timeout shapes and where each fits.
Written by
Distributed systems, cloud-native architecture, and the JVM. mostly shipping, occasionally reading.
All code in this series targets Java 21 preview APIs. Part 9 covers migration to Java 25.
Part 1 focused on ownership: a scope gives related concurrent work one visible lifetime.
Timeouts are the next layer. Once a parent owns the lifetime of its subtasks, it has to decide what happens when time runs out.
That decision is not just a technical detail. A timeout is a product decision hiding as an engineering one.
Is this response useless unless every branch completes? Can the caller use a partial response? Should the first successful branch win? Should slow optional work be cancelled? Should missing sections be visible in the response?
Structured concurrency does not choose those answers for you. What it gives you is a place to make the choice.
This article is still learning material. The examples are small on purpose. They are not production claims, and structured concurrency is still preview. The goal is to understand timeout shapes before mixing them into larger request flows.
The easiest timeout policy is full failure.
If all fields are required, the parent can say: either every subtask completes before the deadline, or the whole operation fails.
In Java 21 preview syntax, that shape looks like this:
public <T> T runWithTimeout(Callable<T> task, Instant deadline) throws Exception {
try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
Subtask<T> result = scope.fork(task);
scope.joinUntil(deadline);
scope.throwIfFailed();
return result.get();
}
}
There are two ideas packed into this small method.
First, the deadline belongs to the scope join, not just to the code after the work has finished. If you call join() and only then check the clock, you did not stop waiting at the deadline. You only noticed late.
Second, timeout policy and failure policy are separate. ShutdownOnFailure says what happens when a subtask fails. joinUntil(deadline) says how long the parent is willing to wait. The response contract still belongs to you.
If a timeout means "this response is invalid," throw. If a timeout means "return what is ready," collect partials. If a timeout means "use fallback," make that fallback visible.
Do not hide those choices behind a generic helper name like runFast.
The companion repository has a useful teaching example in ConcurrentServiceLayer.shortTimeoutExample().
The method sets a 300ms deadline and forks two tasks:
fast-service: 100ms,slow-service: 500ms.The migrated Java 25 version joins the scope and then checks whether the deadline passed. When I ran /timeout/short, it returned a timeout response after 508ms:
HTTP/1.1 500 Internal Server Error
[23b18024] Request timed out after 508ms
The focused wrk run showed the same behavior:
wrk -t4 -c40 -d10s http://localhost:8082/timeout/short
Latency avg: 505.02ms
Requests/sec: 75.32
Non-2xx responses: 760
That is not a result to explain away. It is exactly the lesson.
The code had a 300ms deadline, but because it waited for scope.join() before checking the clock, the parent waited for the 500ms sibling anyway. The timeout was detected, but it was not enforced at the waiting boundary.
For Java 21 preview code, this is why joinUntil(deadline) matters. The deadline has to participate in the wait. Otherwise the deadline becomes a label attached to a late response.
Not every timeout should fail the response.
Sometimes you ask multiple places for equivalent or acceptable data:
If the cache wins, waiting for the database is waste. If the fallback wins, waiting for the primary may still be waste. This is where a first-success policy teaches a different timeout shape.
In Java 21 preview syntax:
try (var scope = new StructuredTaskScope.ShutdownOnSuccess<String>()) {
scope.fork(() -> getFromPrimary());
scope.fork(() -> getFromFallback());
scope.fork(() -> getFromCache());
scope.joinUntil(deadline);
return scope.result();
}
The policy is not "everything must finish before the deadline." The policy is "the first useful answer wins before the deadline."
The companion endpoint ConcurrentServiceLayer.gracefulTimeoutExample() models this shape with three branches:
primary-service: 400ms,fallback-service: 200ms,cache-service: 50ms.The single request returned the cache result in 60ms:
Graceful Timeout Result: cache-service-ok (Duration: 60ms)
The load check stayed near that shape:
wrk -t4 -c40 -d10s http://localhost:8082/timeout/graceful
Latency avg: 55.29ms
Requests/sec: 720.68
Requests: 7,266
Again, this is not a production benchmark. It is a sanity check. The fastest useful branch is 50ms, and the measured request sits close to that value. The scope shape matches the behavior.
The third shape is a strict deadline where all siblings are required, but each sibling is expected to finish within the budget.
The companion endpoint ConcurrentServiceLayer.strictDeadlineExample() uses:
task-1: 200ms,task-2: 300ms,task-3: 400ms,The endpoint returned all three results in 407ms:
Deadline Results: task-1-within-deadline, task-2-within-deadline, task-3-within-deadline
Duration: 407ms
The load check matched the slowest required sibling:
wrk -t4 -c40 -d10s http://localhost:8082/deadline/strict
Latency avg: 405.55ms
Requests/sec: 95.18
Requests: 960
This is the normal fan-out/fan-in pattern from Part 1, with a budget attached. Every sibling is required. The parent waits near the slowest sibling. If one branch exceeds the deadline, the parent should fail the whole operation.
That is different from first-success and different from partial results. The code should make that difference visible.
Partial results are not just "whatever finished."
If the parent returns partial data, the response must say what happened. A caller should not have to guess whether a missing section means:
null,In Java 21 preview syntax, the shape is:
public <T> List<Optional<T>> executeWithPartialResults(
List<Callable<T>> tasks,
Instant deadline) throws InterruptedException {
try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
List<StructuredTaskScope.Subtask<T>> subtasks = new ArrayList<>();
for (Callable<T> task : tasks) {
subtasks.add(scope.fork(task));
}
try {
scope.joinUntil(deadline);
} catch (TimeoutException timeout) {
scope.shutdown();
}
List<Optional<T>> results = new ArrayList<>();
for (var subtask : subtasks) {
if (subtask.state() == StructuredTaskScope.Subtask.State.SUCCESS) {
results.add(Optional.of(subtask.get()));
} else {
results.add(Optional.empty());
}
}
return results;
}
}
This example uses Optional only to keep the teaching code small. In an HTTP response, I would usually prefer explicit metadata:
{
"sections": {
"profile": { "status": "complete" },
"recommendations": { "status": "timeout" },
"activity": { "status": "complete" }
}
}
The important point is that partial response design belongs at the scope boundary. The parent knows which subtasks were started. The parent knows which ones completed. The parent is the right place to decide what the caller sees.
The companion AdvancedStructuredPatterns demo has a timeout-with-partial-results example with four simulated tasks:
The run produced:
Completed: 2/4 tasks
Results: [Quick result, Medium result]
Timed out: [2, 3]
That is the partial-results contract in miniature. The fast and medium tasks made the deadline. The slow and very slow tasks did not. The response does not pretend all data exists, and it does not silently drop the missing branches.
One migration detail matters for this example. This article is learning material. The main branch now builds with OpenJDK 25.0.2 and uses the Java 25 preview structured-concurrency API, with the Java 21 version separately managed in the feature/java-21 branch. The measurements below were generated from the current Java 25 code. The Java 21 preview syntax shown here remains valid for learning purposes, and Part 9 covers the migration details. This specific companion demo uses polling around futures to model the partial-results response shape.
The clean service also has BusinessService.performTimedOperation(), which calls ScopedRequestHandler.runInScopeWithTimeout().
That path uses a 2-second timeout around a simulated slow service that sleeps for 1.5 seconds. A single request returned successfully in 1506ms:
Slow-slow-task (Duration: 1506ms)
The focused load check stayed near the same number:
wrk -t2 -c20 -d10s http://localhost:8085/timed/operation?op=slow-task
Latency avg: 1.51s
Requests/sec: 11.89
Requests: 120
This is an all-or-nothing wrapper where the task completed before its budget. It is useful, but it should not be confused with partial results or first-success behavior. A timeout wrapper can enforce a budget for one operation. It does not define what a multi-section response should look like.
That response contract still needs to be designed.
Use full failure when the response is unsafe or misleading without every required branch.
Use first success when several branches can provide an acceptable answer and only one answer is needed.
Use partial results when some sections are optional and the caller can make sense of missing sections.
Use strict deadlines when every branch is required but each branch should finish inside a shared request budget.
The trap is pretending these are all the same. They are not. They have different failure semantics, different cancellation behavior, and different response contracts.
Structured concurrency helps because the scope becomes the place where the policy is visible.
For timeout-sensitive code, test the edges not the happy path. Does the full-failure path actually fail when one required branch misses the deadline? Does the partial-results path mark missing sections explicitly rather than silently dropping them? Do slow siblings actually cancel according to the chosen policy, or do they keep running after the parent returns? Those three questions cover the timeout behavior worth verifying.
The Article 2 local run ended with the port 8082 metrics reporting 8,308 total requests, 801 timeout responses, and 145.16ms average response time across the mixed timeout checks. Those aggregate numbers are less important than the per-endpoint behavior, but they are useful as a sanity check that the timeout path was actually exercised.
Timeouts decide how long the parent waits.
Part 3 moves to conditional cancellation: cases where the parent should stop siblings not because the clock ran out, but because one branch produced a business result that makes the remaining work wasteful.
The companion repository at github.com/salgotraja/project-loom has runnable examples for every post in this series. The README covers build setup and the scripts used to reproduce the checks.