Was this article helpful?
to mark as helpful
Enjoyed this article?
Get more engineering insights delivered weekly.
Comments
to join the discussion
Note This series uses Java 21 as the baseline. Structured concurrency snippets in this part (
StructuredTaskScope, JEP 453) use preview APIs and require--enable-preview.
TL;DR
- Structured concurrency keeps related concurrent tasks in one lifecycle boundary
- Automatic cancellation means one failure can stop sibling tasks early
- In one sample benchmark, StructuredTaskScope showed lower coordination overhead than CompletableFuture
- The programming model stays close to try-with-resources
- Hierarchical task management keeps orchestration readable
- Scope-based cleanup reduces resource-leak risk in failure paths
The Async Programming Problem in Production
Long CompletableFuture chains look clean in reviews and painful in incident calls.
The common issues are predictable: hard-to-follow stack traces, cleanup gaps that appear only under load, and cancellation logic that behaves inconsistently between happy-path and failure-path traffic.
"Why is Service C still running when Service A failed?" is a common production question with ad-hoc async orchestration.
The CompletableFuture Reality Check
Here's what most of us end up writing when we need to call multiple services:
java
// From StructuredConcurrencyComparison
CompletableFuture<String> cf1 = CompletableFuture.supplyAsync(() -> simulateService("Service-A", 200));
CompletableFuture<String> cf2 = CompletableFuture.supplyAsync(() -> simulateService("Service-B", 300));
CompletableFuture<String> cf3 = CompletableFuture.supplyAsync(() -> simulateService("Service-C", 100));
CompletableFuture.allOf(cf1, cf2, cf3).join();
String result = String.format("Results: %s, %s, %s",
cf1.get(), cf2.get(), cf3.get());
logger.info(result);
Common failure patterns:
- Resource leak risk: failed tasks may continue running and consuming resources
- Complex exception paths: nested handling across async boundaries
- Debugging overhead: stack traces and failure lineage are harder to follow
- Manual cleanup burden: cancellation and timeout paths are easy to miss
- Memory pressure: incomplete futures can retain resources longer than expected
Structured Concurrency Approach
Structured concurrency is based on a simple principle: related tasks should be managed together as a unit. When one fails, the group can fail fast and clean up automatically.
Let's see how the same dashboard service looks with structured concurrency:
java
// From StructuredExample
try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
Subtask<String> fetch1 = scope.fork(() -> fetchFromService1());
Subtask<String> fetch2 = scope.fork(() -> fetchFromService2());
Subtask<String> fetch3 = scope.fork(() -> fetchFromService3());
scope.join();
scope.throwIfFailed();
String result = fetch1.get() + fetch2.get() + fetch3.get();
logger.info("Combined result: {}", result);
}
What changes in practice:
- 10 lines vs 40 lines: significantly less code to maintain
- Automatic cleanup: try-with-resources pattern handles everything
- Lower leak risk: cleanup is tied to scope exit
- Readable code: Linear flow that any developer can understand
- Fail fast: First error cancels all related tasks immediately
Deep Dive: How Structured Concurrency Actually Works
The Architecture
The key point is lifecycle scoping: task start, failure, cancellation, and cleanup are managed together.
Pattern 1: ShutdownOnFailure - Fail Fast and Clean
Perfect for operations where all results are needed:
java
try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
Subtask<String> fetch1 = scope.fork(() -> fetchFromService1());
Subtask<String> fetch2 = scope.fork(() -> fetchFromService2());
Subtask<String> fetch3 = scope.fork(() -> fetchFromService3());
scope.join();
scope.throwIfFailed();
String result = fetch1.get() + fetch2.get() + fetch3.get();
logger.info("All services completed: {}", result);
}
What cleanup behavior gives you:
- Any task failure immediately cancels all other running tasks
- Resources are automatically released when leaving the try block
- Lower risk of long-running orphaned tasks
Pattern 2: ShutdownOnSuccess - First Success Wins
Perfect for redundant services or fastest-wins scenarios:
java
try (var scope = new StructuredTaskScope.ShutdownOnSuccess<String>()) {
scope.fork(() -> slowService("Service-A", 1000));
scope.fork(() -> slowService("Service-B", 500));
scope.fork(() -> slowService("Service-C", 200));
scope.join();
String result = scope.result();
logger.info("First successful result: {}", result);
}
Use this when redundant providers can satisfy the same request and only the first successful result matters.
Real-World Performance Analysis
Benchmark Setup
These benchmark numbers are from one simplified test environment. Real workloads vary widely based on downstream behavior, JVM configuration, and traffic shape.
Test Environment:
- Java 21 with preview features enabled
- 16GB RAM, 8-core CPU
- 1000 iterations per test (after 100 warmup runs)
- Virtual threads for both approaches
Test 1: Service Orchestration Performance
Scenario: User dashboard requiring data from 3 services:
- User Service: 200ms response time
- Order Service: 300ms response time
- Profile Service: 100ms response time
Expected optimal time: ~300ms (limited by slowest service)
java
try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
var task1 = scope.fork(() -> simulateService("Service-A", 200));
var task2 = scope.fork(() -> simulateService("Service-B", 300));
var task3 = scope.fork(() -> simulateService("Service-C", 100));
scope.join();
scope.throwIfFailed();
String result = String.format("Results: %s, %s, %s",
task1.get(), task2.get(), task3.get());
logger.info(result);
}
Performance Results (1000 iterations):
code
PERFORMANCE COMPARISON
Expected time: ~300ms
StructuredTaskScope: 312ms (overhead: 12ms)
CompletableFuture: 328ms (overhead: 28ms)
DETAILED METRICS
StructuredTaskScope average: 2.47ms per operation
CompletableFuture average: 3.21ms per operation
Performance improvement: 23% faster with StructuredTaskScope
Memory efficiency: 18% less memory allocation per operation
How to read these numbers:
- Lower overhead in this run
- Consistent performance profile under this test setup
- Lower temporary allocation in this run
Test 2: Error Handling and Resource Cleanup
Failure scenario: One service fails after 50ms. How quickly can we detect failure and stop related work?
java
try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
Subtask<String> fetch1 = scope.fork(() -> fetchFromService("Service-1", 300, false));
Subtask<String> fetch2 = scope.fork(() -> fetchFromService("Service-2", 200, true));
Subtask<String> fetch3 = scope.fork(() -> fetchFromService("Service-3", 100, false));
scope.join();
scope.throwIfFailed();
} catch (Exception e) {
logger.error("One or more services failed: {}", e.getMessage());
logger.error("All remaining tasks were cancelled automatically");
}
Resource Cleanup Results:
code
ERROR HANDLING PERFORMANCE
Structured Concurrency:
- Error detection: 52ms
- Automatic cancellation: YES
- Resource cleanup: AUTOMATIC
- Wasted compute time: 0ms
CompletableFuture:
- Error detection: 50ms
- Automatic cancellation: NO
- Resource cleanup: MANUAL REQUIRED
- Wasted compute time: 450ms (other tasks keep running)
Critical Finding: StructuredTaskScope prevents resource waste by
automatically cancelling related tasks on first failure.
The practical takeaway is straightforward: cancellation behavior is built into the scope instead of bolted on per call chain.
Validate Gains in Your Environment
- Re-run benchmarks with realistic request mix and concurrency ramps
- Compare p50/p95/p99 latency and error rates, not only averages
- Measure cancellation behavior under induced failures and timeouts
- If running on virtual threads, use JFR
jdk.VirtualThreadPinnedevents for diagnosis - Track allocation and GC impact during sustained runs
Common Limitations (Java 21 Preview)
- Forgetting
scope.throwIfFailed()can hide task failures; this is a common review item in Java 21 preview code
Advanced Patterns
Pattern 3: Timeout with Graceful Degradation
Use this when you need a strict deadline and can return partial or fallback data.
java
public String shortTimeoutExample() throws Exception {
Instant deadline = Instant.now().plusMillis(300);
try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
var slowTask = scope.fork(() -> simulateSlowService("slow-service", 500));
var fastTask = scope.fork(() -> simulateSlowService("fast-service", 100));
scope.join();
if (Instant.now().isAfter(deadline)) {
throw new TimeoutException("Request exceeded 300ms deadline");
}
scope.throwIfFailed();
return String.format("Timeout Results: %s, %s", slowTask.get(), fastTask.get());
}
}
Pattern 4: Hierarchical Task Management
Use this for workflows with staged dependencies.
java
public String processOrder(String orderId) throws Exception {
var validationResult = scopedHandler.runInParallel(
() -> validatePayment(orderId),
() -> validateInventory(orderId),
() -> validateShipping(orderId)
);
if (allValidationsPassed(validationResult)) {
return scopedHandler.runInParallel(
() -> chargePayment(orderId),
() -> reserveInventory(orderId),
() -> scheduleShipping(orderId)
).toString();
}
return "Order validation failed: " + validationResult;
}
Migration Strategy: From CompletableFuture to Structured Concurrency
Phase 1: Identify the Pain Points
Signs your CompletableFuture code needs structured concurrency:
- Exception handling takes more lines than business logic
- You have manual cancellation code that's never tested
- Resource leak monitoring alerts fire regularly
- New developers avoid touching the async orchestration code
- Production issues involve long-running orphaned tasks consuming resources
Phase 2: The Gradual Migration
A practical migration sequence:
Week 1-2: Replace Simple Cases
java
// Before: Complex exception handling
CompletableFuture<String> cf1 = CompletableFuture.supplyAsync(() -> simulateService("A", 1));
CompletableFuture<String> cf2 = CompletableFuture.supplyAsync(() -> simulateService("B", 1));
CompletableFuture.allOf(cf1, cf2).join();
cf1.get();
cf2.get();
// After: Clean structured concurrency
try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
var task1 = scope.fork(() -> simulateService("A", 1));
var task2 = scope.fork(() -> simulateService("B", 1));
scope.join();
scope.throwIfFailed();
task1.get();
task2.get();
}
Week 3-4: Tackle Complex Orchestrations
- Replace CompletableFuture chains with hierarchical structured tasks
- Remove manual cancellation paths where scope cancellation already applies
- Simplify custom exception plumbing where scope failure semantics are sufficient
Week 5+: Monitor and Optimize
- Watch resource usage and cancellation behavior under sustained load
- Measure reduced memory allocation
- Verify operational behavior during failures and timeouts
Phase 3: Practical Best Practices
DO:
- Start with ShutdownOnFailure: Covers 80% of use cases
- Use try-with-resources: Leverage automatic cleanup
- Keep tasks lightweight: Virtual threads make this easy
- Embrace exceptions: Let them propagate naturally up the scope
- Test failure scenarios: Structured concurrency makes this easier
DON'T:
- Mix patterns casually: avoid layering complex CompletableFuture chains inside structured scopes
- Rely on manual cleanup by default: use scope lifecycle first, then add explicit handling only where needed
- Ignore timeouts: Use
joinUntil()for time-sensitive operations - Over-engineer: Prefer simple scopes over deeply nested orchestration
Best Practices for Deployment
Error Handling That Actually Works
java
public <T> T runWithFallback(Callable<T> primary, Callable<T> fallback) throws Exception {
try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
var primaryFuture = scope.fork(primary);
scope.join();
try {
scope.throwIfFailed();
return primaryFuture.get();
} catch (Exception e) {
logger.warn("Primary task failed, using fallback: {}", e.getMessage());
return fallback.call();
}
}
}
Monitoring Structured Concurrency
java
private static String generatePrometheusMetrics() {
StringBuilder metrics = new StringBuilder();
MemoryUsage heapUsage = memoryBean.getHeapMemoryUsage();
metrics.append("# HELP jvm_memory_used_bytes Used memory in bytes\n");
metrics.append("# TYPE jvm_memory_used_bytes gauge\n");
metrics.append("jvm_memory_used_bytes{area=\"heap\"} ").append(heapUsage.getUsed()).append("\n");
metrics.append("# HELP jvm_memory_max_bytes Maximum memory in bytes\n");
metrics.append("# TYPE jvm_memory_max_bytes gauge\n");
metrics.append("jvm_memory_max_bytes{area=\"heap\"} ").append(heapUsage.getMax()).append("\n");
ManagementFactory.getGarbageCollectorMXBeans().forEach(gc -> {
metrics.append("# HELP jvm_gc_collection_seconds Time spent in GC\n");
metrics.append("# TYPE jvm_gc_collection_seconds counter\n");
metrics.append("jvm_gc_collection_seconds{gc=\"").append(gc.getName()).append("\"} ")
.append(gc.getCollectionTime() / 1000.0).append("\n");
});
return metrics.toString();
}
What's Next?
In Part 5, we'll explore advanced structured concurrency patterns for real-world scenarios: conditional cancellation, progressive result collection, and fault-tolerant orchestration.
We'll cover:
- Timeout patterns that prevent cascading failures
- Conditional cancellation for complex business workflows
- Resource-aware scheduling and the bulkhead pattern
- Building circuit breakers with structured concurrency
- Production monitoring and debugging strategies
Resources
- Complete Code: StructuredConcurrencyComparison.java
- Try It Yourself: Run the examples with Java 21+ and
--enable-preview - Official Documentation: JEP 453: Structured Concurrency
- Performance Tests: Run repeated tests and compare behavior under failure scenarios
Part 4 complete. We focused on making parallel orchestration easier to reason about and safer to operate.
Series Navigation:
- Previous: Part 3: Real-World Microservices →
- Next: Part 5: Advanced Structured Concurrency Patterns →
Failure-handling differences often become apparent first in migrated paths.