Related articles: Also see applying k6 to real-world peak traffic scenarios, a complete frontend performance testing guide to pair with load testing, and chaos engineering as the next step after stress testing.
Load vs. Stress vs. Soak Testing: When to Use Each One Before a High-Traffic Event
You've shipped your application. It works perfectly in development. Your unit tests pass. Integration tests look good. Then Black Friday arrives, traffic spikes 10x, and your carefully crafted system crumbles under load. Database connections exhaust. Response times spike. Memory leaks that took hours to manifest in testing now crash your app in minutes.
Sound familiar?
Most teams test functionality thoroughly but treat performance as an afterthought. They might run a few load tests before launch, declare victory when the system handles 1,000 concurrent users, and call it a day. Then production tells a different story.
The problem isn't that they didn't test performance�it's that they ran the wrong kind of performance test for the questions they needed to answer.
This guide explains the three fundamental types of performance testing�load testing, stress testing, and soak testing�and, critically, when to use each one. You'll learn practical implementation techniques with modern tools like k6 and JMeter, and how to interpret results to build truly resilient, scalable systems.
The Performance Testing Landscape
Performance testing is an umbrella term covering various techniques that evaluate system behavior under load. The three most important types every engineer should understand are:
graph LR
A[Performance Testing] --> B[Load Testing]
A --> C[Stress Testing]
A --> D[Soak Testing]
B --> B1[Expected Traffic]
B --> B2[Normal Operation]
B --> B3[Find Bottlenecks]
C --> C1[Beyond Capacity]
C --> C2[Breaking Points]
C --> C3[Failure Modes]
D --> D1[Extended Duration]
D --> D2[Memory Leaks]
D --> D3[Resource Exhaustion]
style B fill:#c5e1a5
style C fill:#ffccbc
style D fill:#bbdefb
Each type serves a different purpose and answers different questions about your system's behavior.
Load Testing: Can Your System Handle Expected Traffic?
Load testing validates that your system performs acceptably under expected real-world load. It simulates normal and peak traffic conditions to ensure you can handle your target user base.
When to Use Load Testing
- Before launching a new feature or product
- After infrastructure changes
- To validate autoscaling configuration
- To establish performance baselines
- Before major events (Black Friday, product launches)
What Load Testing Reveals
- Average response times under typical load
- Throughput (requests per second)
- Resource utilization (CPU, memory, database connections)
- Bottlenecks in your architecture
- Whether you meet SLAs and SLOs
Load Testing Characteristics
| Aspect | Description |
|---|---|
| Duration | 10 minutes to 1 hour |
| Traffic Pattern | Gradual ramp-up to expected peak |
| User Count | Target concurrent users (e.g., 1,000) |
| Expected Result | System remains stable, response times acceptable |
| Failure Condition | Response times exceed SLA or errors occur |
Load Testing Example with k6
// load-test.js
import http from 'k6/http';
import { check, sleep } from 'k6';
import { Rate } from 'k6/metrics';
// Custom metrics
const errorRate = new Rate('errors');
// Test configuration
export const options = {
stages: [
{ duration: '2m', target: 100 }, // Ramp up to 100 users over 2min
{ duration: '5m', target: 100 }, // Stay at 100 users for 5min
{ duration: '2m', target: 500 }, // Ramp up to peak 500 users
{ duration: '10m', target: 500 }, // Stay at peak for 10min
{ duration: '2m', target: 0 }, // Ramp down
],
thresholds: {
http_req_duration: ['p(95)<500', 'p(99)<1000'], // 95% under 500ms, 99% under 1s
http_req_failed: ['rate<0.01'], // Error rate under 1%
errors: ['rate<0.01'],
},
};
// Simulated user behavior
export default function () {
// Homepage
let response = http.get('https://api.example.com/');
check(response, {
'homepage status 200': (r) => r.status === 200,
'homepage response time OK': (r) => r.timings.duration < 500,
}) || errorRate.add(1);
sleep(1);
// API request
response = http.get('https://api.example.com/api/products?limit=20');
check(response, {
'API status 200': (r) => r.status === 200,
'API response time OK': (r) => r.timings.duration < 300,
'returns products': (r) => JSON.parse(r.body).products.length > 0,
}) || errorRate.add(1);
sleep(2);
// Search
response = http.get('https://api.example.com/api/search?q=laptop');
check(response, {
'search status 200': (r) => r.status === 200,
}) || errorRate.add(1);
sleep(3);
}
Run the test:
k6 run load-test.js
Interpreting Load Test Results
Key metrics to watch:
// Good load test results
http_req_duration..............: avg=245ms min=89ms med=198ms max=892ms p(90)=387ms p(95)=456ms p(99)=723ms
http_req_failed................: 0.12% ? 23 ? 18977
http_reqs......................: 19000 63.3/s
vus............................: 500 min=0 max=500
vus_max........................: 500 min=500 max=500
// Problem indicators:
// - p(95) or p(99) exceeding thresholds
// - Error rate increasing with load
// - Response times degrading over time (potential memory leak)
Stress Testing: What Happens When Things Go Wrong?
Stress testing pushes your system beyond its expected capacity to identify breaking points and understand failure modes. It answers: "What happens when we get 10x our expected traffic?"
When to Use Stress Testing
- To find the maximum capacity
- To understand graceful degradation
- To test circuit breakers and fallbacks
- To validate monitoring and alerting
- To prepare for DDoS mitigation
What Stress Testing Reveals
- The absolute maximum throughput
- At what point the system becomes unstable
- How the system fails (gracefully vs. catastrophically)
- Whether error handling works under pressure
- If the system recovers after load decreases
Stress Testing Characteristics
| Aspect | Description |
|---|---|
| Duration | 15-30 minutes |
| Traffic Pattern | Aggressive ramp-up past capacity |
| User Count | Far beyond expected (5-10x) |
| Expected Result | System eventually fails but gracefully |
| Failure Condition | Catastrophic failure, can't recover |
Stress Testing Example with k6
// stress-test.js
import http from 'k6/http';
import { check, sleep } from 'k6';
export const options = {
stages: [
{ duration: '2m', target: 500 }, // Normal load
{ duration: '5m', target: 500 },
{ duration: '2m', target: 2000 }, // Spike to 4x
{ duration: '5m', target: 2000 },
{ duration: '2m', target: 5000 }, // Spike to 10x
{ duration: '5m', target: 5000 },
{ duration: '5m', target: 0 }, // Recovery period
],
thresholds: {
// More lenient thresholds - we EXPECT things to break
http_req_failed: ['rate<0.05'], // Allow 5% errors
},
};
export default function () {
const response = http.get('https://api.example.com/api/products');
check(response, {
'status is 200 or 503': (r) => r.status === 200 || r.status === 503,
'has rate limit headers': (r) => r.headers['X-RateLimit-Remaining'] !== undefined,
});
sleep(1);
}
// Lifecycle hooks to test recovery
export function teardown(data) {
// Give system time to recover
console.log('Stress test complete. Waiting 2 minutes for recovery...');
sleep(120);
// Verify system recovered
const response = http.get('https://api.example.com/health');
check(response, {
'system recovered': (r) => r.status === 200,
});
}
Stress Testing with JMeter
<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="5.0">
<hashTree>
<TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="Stress Test">
<elementProp name="TestPlan.user_defined_variables" elementType="Arguments">
<collectionProp name="Arguments.arguments">
<elementProp name="BASE_URL" elementType="Argument">
<stringProp name="Argument.name">BASE_URL</stringProp>
<stringProp name="Argument.value">https://api.example.com</stringProp>
</elementProp>
</collectionProp>
</elementProp>
</TestPlan>
<hashTree>
<!-- Ultimate Thread Group for complex load patterns -->
<kg.apc.jmeter.threads.UltimateThreadGroup guiclass="kg.apc.jmeter.threads.UltimateThreadGroupGui"
testclass="kg.apc.jmeter.threads.UltimateThreadGroup" testname="Stress Load Pattern">
<collectionProp name="ultimatethreadgroupdata">
<!-- Stage 1: Baseline -->
<collectionProp name="1">
<stringProp name="100">100</stringProp> <!-- threads -->
<stringProp name="30">30</stringProp> <!-- initial delay -->
<stringProp name="60">60</stringProp> <!-- startup time -->
<stringProp name="300">300</stringProp> <!-- hold load -->
<stringProp name="30">30</stringProp> <!-- shutdown -->
</collectionProp>
<!-- Stage 2: Stress -->
<collectionProp name="2">
<stringProp name="500">500</stringProp>
<stringProp name="120">120</stringProp>
<stringProp name="60">60</stringProp>
<stringProp name="300">300</stringProp>
<stringProp name="60">60</stringProp>
</collectionProp>
<!-- Stage 3: Break -->
<collectionProp name="3">
<stringProp name="2000">2000</stringProp>
<stringProp name="240">240</stringProp>
<stringProp name="120">120</stringProp>
<stringProp name="300">300</stringProp>
<stringProp name="120">120</stringProp>
</collectionProp>
</collectionProp>
</kg.apc.jmeter.threads.UltimateThreadGroup>
</hashTree>
</hashTree>
</jmeterTestPlan>
Run with:
jmeter -n -t stress-test.jmx -l results.jtl -e -o report/
Soak Testing: Can Your System Run Forever?
Soak testing (also called "endurance testing") runs your system at normal load for an extended period to uncover issues that only manifest over time, like memory leaks, connection pool exhaustion, or log file growth.
When to Use Soak Testing
- Before production deployments
- After changes to connection handling, caching, or resource management
- To validate that memory doesn't grow unbounded
- To test log rotation and cleanup jobs
- To verify connection pool configuration
What Soak Testing Reveals
- Memory leaks
- Connection pool exhaustion
- Disk space issues (logs, temp files)
- Database connection leaks
- Degrading performance over time
- Resource cleanup issues
Soak Testing Characteristics
| Aspect | Description |
|---|---|
| Duration | 12-72 hours |
| Traffic Pattern | Steady, consistent load |
| User Count | Normal production levels |
| Expected Result | Stable performance over entire duration |
| Failure Condition | Memory growth, increasing response times, crashes |
Soak Testing Example with k6
// soak-test.js
import http from 'k6/http';
import { check, sleep } from 'k6';
import { Counter, Trend } from 'k6/metrics';
// Custom metrics to track over time
const memoryLeakIndicator = Trend('memory_leak_indicator');
const responseTimeTrend = Trend('response_time_trend');
export const options = {
stages: [
{ duration: '5m', target: 200 }, // Ramp up
{ duration: '24h', target: 200 }, // Stay at load for 24 hours
{ duration: '5m', target: 0 }, // Ramp down
],
thresholds: {
http_req_duration: ['p(95)<500'],
http_req_failed: ['rate<0.01'],
// Key soak test threshold: response time shouldn't degrade over time
response_time_trend: ['p(95)<600'], // Slight buffer for variance
},
};
let requestCount = 0;
export default function () {
requestCount++;
const startTime = Date.now();
const response = http.get('https://api.example.com/api/products');
const duration = Date.now() - startTime;
// Track response time trend
responseTimeTrend.add(duration);
// Log periodically to detect trends
if (requestCount % 1000 === 0) {
console.log(`Request ${requestCount}: ${duration}ms`);
}
check(response, {
'status 200': (r) => r.status === 200,
'response time OK': (r) => r.timings.duration < 500,
'no memory errors': (r) => !r.body.includes('OutOfMemory'),
});
sleep(1);
}
// Check for memory leak indicators
export function handleSummary(data) {
// Compare first hour vs last hour response times
// Significant degradation suggests resource leak
const firstHourP95 = data.metrics.http_req_duration.values['p(95)'];
const lastHourP95 = data.metrics.http_req_duration.values['p(95)'];
if (lastHourP95 > firstHourP95 * 1.5) {
console.warn(`?? Response time degradation detected: ${firstHourP95}ms -> ${lastHourP95}ms`);
}
return {
'soak-test-summary.json': JSON.stringify(data, null, 2),
};
}
Monitoring During Soak Tests
Critical metrics to track throughout the soak test:
// monitoring/soak-test-monitor.ts
import { CloudWatchClient, GetMetricStatisticsCommand } from '@aws-sdk/client-cloudwatch';
interface SoakTestMetrics {
timestamp: Date;
memoryUsageMB: number;
cpuPercent: number;
activeConnections: number;
responseTimeP95: number;
errorRate: number;
diskUsagePercent: number;
}
async function monitorSoakTest(instanceId: string, durationHours: number): Promise<SoakTestMetrics[]> {
const cloudwatch = new CloudWatchClient({ region: 'us-east-1' });
const metrics: SoakTestMetrics[] = [];
const startTime = new Date();
const endTime = new Date(startTime.getTime() + durationHours * 60 * 60 * 1000);
const metricsToTrack = ['MemoryUtilization', 'CPUUtilization', 'DatabaseConnections', 'DiskSpaceUtilization'];
// Collect metrics every 5 minutes
for (let now = startTime; now < endTime; now.setMinutes(now.getMinutes() + 5)) {
const snapshot: Partial<SoakTestMetrics> = {
timestamp: new Date(now),
};
for (const metricName of metricsToTrack) {
const command = new GetMetricStatisticsCommand({
Namespace: 'AWS/EC2',
MetricName: metricName,
Dimensions: [{ Name: 'InstanceId', Value: instanceId }],
StartTime: new Date(now.getTime() - 5 * 60 * 1000),
EndTime: now,
Period: 300,
Statistics: ['Average', 'Maximum'],
});
const response = await cloudwatch.send(command);
// Process metrics...
}
metrics.push(snapshot as SoakTestMetrics);
// Alert if memory grows >20% from baseline
if (metrics.length > 12) {
// After 1 hour
const baseline = metrics[12].memoryUsageMB;
const current = snapshot.memoryUsageMB!;
if (current > baseline * 1.2) {
console.error(`?? MEMORY LEAK DETECTED: ${baseline}MB -> ${current}MB`);
}
}
}
return metrics;
}
Comparison: When to Use Each Test Type
| Scenario | Load Test | Stress Test | Soak Test |
|---|---|---|---|
| Pre-launch validation | ? Primary | ?? Recommended | ?? If time permits |
| Infrastructure change | ? Yes | ? Not necessary | ? Not necessary |
| Code deployment | ?? Quick smoke test | ? No | ? For major releases |
| Capacity planning | ? Yes | ? Yes | ? No |
| Memory leak investigation | ? No | ? No | ? Essential |
| Finding max throughput | ?? Indicates | ? Determines | ? No |
| Testing autoscaling | ? Perfect | ? Good | ? No |
| Validating error handling | ?? Basic | ? Comprehensive | ? No |
| SLA/SLO validation | ? Primary | ? No | ? Long-term |
Performance Testing Strategy: A Complete Flow
graph TD
A[New Feature/Release] --> B{Load Test}
B -->|Pass| C{Stress Test}
B -->|Fail| B1[Optimize]
B1 --> B
C -->|Pass| D{Critical Feature?}
C -->|Fail Gracefully| E[Document Limits]
C -->|Catastrophic Failure| C1[Fix Error Handling]
C1 --> C
D -->|Yes| F{Soak Test}
D -->|No| G[Deploy to Staging]
F -->|Pass| G
F -->|Memory Leak| F1[Fix Leak]
F1 --> F
F -->|Performance Degradation| F2[Investigate Resources]
F2 --> F
G --> H[Production Deployment]
E --> G
style B fill:#c5e1a5
style C fill:#ffccbc
style F fill:#bbdefb
style H fill:#fff9c4
Building a Performance Testing Pipeline
Integrate all three test types into your CI/CD:
# .github/workflows/performance-tests.yml
name: Performance Testing Pipeline
on:
push:
branches: [main]
schedule:
- cron: '0 2 * * 0' # Weekly soak test
jobs:
load-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Deploy to Test Environment
run: ./scripts/deploy-test.sh
- name: Run Load Test
uses: grafana/k6-action@v0.3.0
with:
filename: tests/load-test.js
- name: Upload Results
uses: actions/upload-artifact@v4
with:
name: load-test-results
path: summary.json
stress-test:
needs: load-test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run Stress Test
uses: grafana/k6-action@v0.3.0
with:
filename: tests/stress-test.js
- name: Analyze Breaking Point
run: |
MAX_RPS=$(jq '.metrics.http_reqs.rate' summary.json)
echo "Max throughput: $MAX_RPS req/s"
echo "MAX_RPS=$MAX_RPS" >> $GITHUB_ENV
- name: Update Capacity Docs
run: |
echo "Last tested: $(date)" > docs/capacity.md
echo "Max RPS: $MAX_RPS" >> docs/capacity.md
soak-test:
if: github.event_name == 'schedule'
runs-on: ubuntu-latest
timeout-minutes: 1500 # 25 hours
steps:
- uses: actions/checkout@v4
- name: Run 24h Soak Test
uses: grafana/k6-action@v0.3.0
with:
filename: tests/soak-test.js
- name: Analyze Memory Trends
run: |
python scripts/analyze-soak-results.py summary.json
- name: Alert on Memory Leak
if: failure()
uses: 8398a7/action-slack@v3
with:
status: failure
text: '?? Soak test detected memory leak or performance degradation'
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK }}
Common Performance Bottlenecks and How Each Test Reveals Them
| Bottleneck | Load Test | Stress Test | Soak Test |
|---|---|---|---|
| Database connection pool exhausted | ?? May see at peak | ? Definitely see | ? Won't exceed pool |
| Memory leak in application | ? Too short | ? Too short | ? Primary indicator |
| Inefficient database query | ? Slow response times | ? Database becomes bottleneck | ?? May worsen over time |
| Autoscaling too slow | ?? May see delay | ? Clear indicator | ? Irrelevant |
| CDN cache misses | ? Visible in metrics | ? Exacerbated | ?? Depends on test |
| Connection leak | ? Won't manifest | ? Too short | ? Primary indicator |
| Rate limiting misconfigured | ?? May trigger | ? Will definitely trigger | ? Unlikely |
Best Practices Across All Test Types
1. Test Production-Like Environments
# Bad: Testing against local dev environment
k6 run --vus 1000 load-test.js # Localhost can't handle this
# Good: Testing against staging that mirrors production
export BASE_URL=https://staging.example.com
k6 run --vus 1000 load-test.js
2. Use Realistic User Behavior
// Bad: Unrealistic constant hammering
export default function () {
http.get('https://api.example.com/products');
}
// Good: Realistic user journey with think time
export default function () {
// Homepage
http.get('https://example.com/');
sleep(randomIntBetween(2, 5));
// Browse products
http.get('https://example.com/products');
sleep(randomIntBetween(5, 10));
// Product detail
const productId = randomIntBetween(1, 1000);
http.get(`https://example.com/products/${productId}`);
sleep(randomIntBetween(10, 20));
// Only 10% add to cart
if (Math.random() < 0.1) {
http.post('https://example.com/cart', { productId });
sleep(randomIntBetween(3, 7));
}
}
3. Monitor System Metrics, Not Just Request Metrics
// Track infrastructure alongside application metrics
interface PerformanceSnapshot {
// Application metrics (from k6)
requestsPerSecond: number;
p95ResponseTime: number;
errorRate: number;
// Infrastructure metrics (from monitoring)
cpuUtilization: number;
memoryUtilization: number;
activeConnections: number;
diskIOPS: number;
// Database metrics
dbConnections: number;
dbQueryTime: number;
dbConnectionPoolUtilization: number;
}
Conclusion
Load testing, stress testing, and soak testing aren't interchangeable�they're complementary techniques that answer different critical questions about your system:
- Load testing validates you can handle expected traffic under normal conditions
- Stress testing reveals breaking points and ensures graceful failure modes
- Soak testing exposes time-based issues like memory leaks and resource exhaustion
A mature performance testing strategy incorporates all three:
- Every release: Run load tests to validate SLA compliance
- Before major events: Run stress tests to understand capacity limits
- Monthly or before major releases: Run soak tests to catch resource leaks
The most important lesson? Don't wait for production to discover your performance limits. Test early, test often, and test realistically.
Ready to integrate all three types of performance testing into your development workflow? Sign up for ScanlyApp and add automated load, stress, and soak testing to your CI/CD pipeline today.
