Timeouts in distributed code are routine, not exceptional. The real question is whether a missed deadline should fail the whole request or return what made it back in time. The three timeout shapes we run and when each one fits.
Written by
Distributed systems, cloud-native architecture, and the JVM. mostly shipping, occasionally reading.
Was this useful?
Note This article uses the Java 21 preview structured concurrency API (
StructuredTaskScope, JEP 453). API shape changed in later previews. See Part 9 for Java 21 -> Java 25 migration guidance. Compile and run with--enable-preview.
In distributed systems, timeouts are normal, not exceptional. The important design question is whether timeout handling is:
Structured concurrency gives you clearer control over both behaviors.
public <T> T runInScopeWithTimeout(Callable<T> task, Duration timeout) throws Exception {
Instant deadline = Instant.now().plus(timeout);
try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
var future = scope.fork(task);
scope.join();
if (Instant.now().isAfter(deadline)) {
throw new TimeoutException("Operation exceeded timeout: " + timeout);
}
scope.throwIfFailed();
return future.get();
}
}
This is straightforward and often correct for endpoints where all fields are mandatory.
When some sections are optional, return what completed before the deadline and mark missing sections explicitly.
public <T> List<Optional<T>> executeWithPartialResults(List<Callable<T>> tasks, Duration timeout)
throws InterruptedException {
Instant deadline = Instant.now().plus(timeout);
try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
List<StructuredTaskScope.Subtask<T>> subtasks = new ArrayList<>();
for (Callable<T> task : tasks) {
subtasks.add(scope.fork(task));
}
try {
scope.joinUntil(deadline);
} catch (TimeoutException ignored) {
// Deadline reached: return what is available.
}
// Use Optional to distinguish missing vs null results.
List<Optional<T>> results = new ArrayList<>(subtasks.size());
for (var subtask : subtasks) {
if (subtask.state() == StructuredTaskScope.Subtask.State.SUCCESS) {
results.add(Optional.of(subtask.get()));
} else {
results.add(Optional.empty());
}
}
scope.shutdown(); // Stop unfinished work when returning partial data.
return results;
}
}
Optional streaming progress can be added with a callback:
For simple cases, poll subtask.state() after joinUntil(...).
@FunctionalInterface
public interface ProgressCallback<T> {
// Optional: for streaming progress; simple polling in Java 21 preview.
void onProgress(int taskIndex, T result);
}
public void executeWithProgressCallback(List<Callable<T>> tasks,
ProgressCallback<T> callback)
throws InterruptedException {
try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
List<StructuredTaskScope.Subtask<T>> subtasks = new ArrayList<>();
for (Callable<T> task : tasks) {
subtasks.add(scope.fork(task));
}
boolean[] completed = new boolean[tasks.size()];
int totalCompleted = 0;
long startTime = System.currentTimeMillis();
long timeout = 10000;
while (totalCompleted < tasks.size() &&
(System.currentTimeMillis() - startTime) < timeout) {
try {
scope.joinUntil(Instant.now().plusMillis(10));
} catch (TimeoutException e) {
}
for (int i = 0; i < subtasks.size(); i++) {
if (!completed[i] &&
subtasks.get(i).state() == StructuredTaskScope.Subtask.State.SUCCESS) {
completed[i] = true;
totalCompleted++;
try {
callback.onProgress(i, subtasks.get(i).get());
} catch (Exception e) {
}
} else if (!completed[i] &&
subtasks.get(i).state() == StructuredTaskScope.Subtask.State.FAILED) {
completed[i] = true;
totalCompleted++;
}
}
}
try {
scope.join();
} catch (Exception e) {
}
} catch (Exception e) {
throw new RuntimeException("Progressive execution timeout", e);
}
}
This keeps timeout behavior explicit and avoids hidden stale work after response completion.
Use full failure when:
Use partial responses when:
joinUntil(...) timeout does not automatically define your response policy; you still need to decide whether to fail, return partials, or retry.
ShutdownOnFailure is fail-fast by exceptions, not business semantics; for optional sections, avoid converting normal absence into hard failures.
In timeout code paths, remember scope.shutdown() when you choose to return early. In Java 21 experiments, forgetting explicit shutdown() after timeout was a frequent source of leaked work.
Keep response contracts explicit; include metadata for missing sections instead of silent null handling.
For timeout-sensitive endpoints, test at least:
Include assertions on cancellation behavior and late work cleanup.
Structured concurrency is preview in Java 21.
javac --release 21 --enable-preview ...
java --enable-preview ...