Skip to content

Commit abdd215

Browse files
Devesh-Skyflowclaude
Devesh-Skyflow
and
claude
committed
SK-3023 add configurable connect/read/write timeouts (v3)
Extends the v3 HTTP config (Phase 2 of CUST-4311) with per-attempt connectTimeout, readTimeout, and writeTimeout, in seconds, at both the client-wide (Skyflow.builder()) and per-vault (VaultConfig) levels, with the same per-field precedence as the existing timeout/retry knobs. Unset phase timeouts leave OkHttp's built-in 10s default in place, so existing integrations are unchanged. The overall `timeout` (callTimeout) remains the total ceiling that bounds the whole call including retries; the three phase timeouts bound a single phase of one attempt. - VaultConfig: connectTimeout/readTimeout/writeTimeout fields + accessors - Skyflow.SkyflowClientBuilder: client-wide setters + propagation - VaultClient: resolve per-field and apply only when configured - Tests: OkHttp-default-when-unset, vault/client overrides, per-field - README: settings table + "how the four timeouts relate" guidance - TimeoutLiveDemo: scenario showing a per-phase timeout firing first Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent ceae600 commit abdd215

7 files changed

Lines changed: 336 additions & 14 deletions

File tree

‎README.md‎

Lines changed: 16 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -180,14 +180,19 @@ You can control how long a request is allowed to run and whether failed requests
180180
| Setting | Unit | Default | Description |
181181
| --- | --- | --- | --- |
182182
| `timeout` | seconds | `60` | Overall time budget for a request, including any retries and backoff. |
183+
| `connectTimeout` | seconds | `10` | Time budget for establishing a connection, per attempt. |
184+
| `readTimeout` | seconds | `10` | Time budget for reading the response, per attempt. |
185+
| `writeTimeout` | seconds | `10` | Time budget for writing the request, per attempt. |
183186
| `maxRetries` | count | `0` | Number of retry attempts. `0` disables retries. |
184187
| `initialRetryDelayMillis` | milliseconds | `500` | Base delay before the first retry. |
185188
| `maxRetryDelayMillis` | milliseconds | `2000` | Upper bound on the delay between retries. |
186189

190+
**How the four timeouts relate:** `timeout` is the *total* ceiling for the whole call — every attempt, every backoff sleep, all counted together. `connectTimeout`, `readTimeout`, and `writeTimeout` each bound a single *phase of one attempt* (connecting, reading, writing). Because the phase timeouts are per attempt, their sum across retries can exceed `timeout` — but `timeout` always wins and cuts the call off. In practice, set `timeout` to cap worst-case blocking, and use the phase timeouts to fail faster on a stalled connection or a slow-but-not-dead server. The three phase timeouts default to the underlying HTTP client's `10` seconds; leaving them unset preserves that default.
191+
187192
Each setting is available at two levels:
188193

189-
- **Client-wide** — on `Skyflow.builder()`: `.timeout(int)`, `.maxRetries(int)`, `.initialRetryDelayMillis(long)`, `.maxRetryDelayMillis(long)`. Applies to every vault.
190-
- **Per vault** — on `VaultConfig`: `.setTimeout(int)`, `.setMaxRetries(int)`, `.setInitialRetryDelayMillis(long)`, `.setMaxRetryDelayMillis(long)`. Applies to that vault only.
194+
- **Client-wide** — on `Skyflow.builder()`: `.timeout(int)`, `.connectTimeout(int)`, `.readTimeout(int)`, `.writeTimeout(int)`, `.maxRetries(int)`, `.initialRetryDelayMillis(long)`, `.maxRetryDelayMillis(long)`. Applies to every vault.
195+
- **Per vault** — on `VaultConfig`: `.setTimeout(int)`, `.setConnectTimeout(int)`, `.setReadTimeout(int)`, `.setWriteTimeout(int)`, `.setMaxRetries(int)`, `.setInitialRetryDelayMillis(long)`, `.setMaxRetryDelayMillis(long)`. Applies to that vault only.
191196

192197
**Precedence:** a value set on `VaultConfig` (per vault) overrides the client-wide value set on `Skyflow.builder()`, which overrides the SDK default. Resolution is per field, so a vault can override just `timeout` and still inherit the client-wide retry settings.
193198

@@ -206,21 +211,28 @@ vaultConfig.setClusterId("<CLUSTER_ID>");
206211
vaultConfig.setEnv(Env.PROD);
207212
vaultConfig.setCredentials(credentials);
208213
vaultConfig.setTimeout(30); // seconds — overall request timeout
214+
vaultConfig.setConnectTimeout(5); // seconds — per-attempt connect timeout
215+
vaultConfig.setReadTimeout(20); // seconds — per-attempt read timeout
216+
vaultConfig.setWriteTimeout(5); // seconds — per-attempt write timeout
209217
vaultConfig.setMaxRetries(3); // retry attempts (0 = retries off)
210218
vaultConfig.setInitialRetryDelayMillis(1000L); // base backoff in milliseconds
211219
vaultConfig.setMaxRetryDelayMillis(4000L); // backoff cap in milliseconds
212220

213221
// Client-wide defaults: apply to every vault unless overridden on the vault (as above).
214222
Skyflow skyflowClient = Skyflow.builder()
215223
.timeout(60) // seconds — overall request timeout
224+
.connectTimeout(10) // seconds — per-attempt connect timeout
225+
.readTimeout(15) // seconds — per-attempt read timeout
226+
.writeTimeout(10) // seconds — per-attempt write timeout
216227
.maxRetries(2) // retry attempts (0 = retries off)
217228
.initialRetryDelayMillis(500L) // base backoff in milliseconds
218229
.maxRetryDelayMillis(2000L) // backoff cap in milliseconds
219230
.addVaultConfig(vaultConfig)
220231
.build();
221232

222-
// Result for this vault: timeout=30, maxRetries=3, initialRetryDelayMillis=1000, maxRetryDelayMillis=4000
223-
// (all overridden per vault). A vault that sets none of these inherits the client-wide values above.
233+
// Result for this vault: timeout=30, connectTimeout=5, readTimeout=20, writeTimeout=5,
234+
// maxRetries=3, initialRetryDelayMillis=1000, maxRetryDelayMillis=4000 (all overridden per vault).
235+
// A vault that sets none of these inherits the client-wide values above.
224236
```
225237

226238
# Vault

‎samples/src/main/java/com/example/vault/TimeoutLiveDemo.java‎

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
2+
package com.example.vault;
3+
4+
import com.skyflow.Skyflow;
5+
import com.skyflow.config.Credentials;
6+
import com.skyflow.config.VaultConfig;
7+
import com.skyflow.enums.Env;
8+
import com.skyflow.enums.LogLevel;
9+
import com.skyflow.vault.data.DetokenizeRequest;
10+
import com.skyflow.vault.data.DetokenizeResponse;
11+
import java.net.ServerSocket;
12+
import java.net.Socket;
13+
import java.util.ArrayList;
14+
import java.util.List;
15+
16+
/**
17+
* Local manual demo for the timeout behavior.
18+
*
19+
* <p>Starts a local server that accepts the TCP connection but never responds (so the TLS
20+
* handshake hangs), points the SDK at it, then issues a real detokenize. Two scenarios:
21+
* <ol>
22+
* <li><b>Overall {@code timeout}</b> — the whole call is bounded by {@code timeout} instead of
23+
* hanging forever.</li>
24+
* <li><b>Per-phase {@code readTimeout}</b> — the server accepts the TCP connection instantly (so
25+
* {@code connectTimeout} never fires) but never sends bytes, stalling the TLS-handshake read.
26+
* With {@code readTimeout} shorter than the overall {@code timeout}, the call is cut at
27+
* {@code readTimeout}, returning well before the overall ceiling. {@code connectTimeout} and
28+
* {@code writeTimeout} bound their own phases the same way.</li>
29+
* </ol>
30+
*
31+
* <p>Run: from samples/, mvn -o compile then
32+
* java -cp "target/classes:$(mvn -o -q dependency:build-classpath -Dmdep.outputFile=/dev/stdout)" \
33+
* com.example.vault.TimeoutLiveDemo
34+
*/
35+
public class TimeoutLiveDemo {
36+
public static void main(String[] args) throws Exception {
37+
// Local black-hole server: accept connections, never respond.
38+
ServerSocket serverSocket = new ServerSocket(0);
39+
int port = serverSocket.getLocalPort();
40+
Thread acceptor = new Thread(() -> {
41+
try { while (true) { Socket s = serverSocket.accept(); /* hold open, never respond */ } }
42+
catch (Exception ignored) { }
43+
});
44+
acceptor.setDaemon(true);
45+
acceptor.start();
46+
47+
Credentials credentials = new Credentials();
48+
credentials.setApiKey("sky-ab123-abcd1234cdef1234abcd4321cdef4321"); // token set locally, no network
49+
50+
List<String> tokens = new ArrayList<>();
51+
tokens.add("some-token");
52+
DetokenizeRequest req = DetokenizeRequest.builder().tokens(tokens).build();
53+
54+
// ---- Scenario 1: overall `timeout` bounds the whole call ----
55+
VaultConfig timeoutConfig = new VaultConfig();
56+
timeoutConfig.setVaultId("demo_vault");
57+
timeoutConfig.setVaultURL("https://localhost:" + port); // hangs during TLS handshake
58+
timeoutConfig.setEnv(Env.PROD);
59+
timeoutConfig.setCredentials(credentials);
60+
timeoutConfig.setTimeout(5); // seconds — change this to see the ceiling move
61+
62+
Skyflow timeoutClient = Skyflow.builder()
63+
.setLogLevel(LogLevel.ERROR)
64+
.addVaultConfig(timeoutConfig)
65+
.build();
66+
67+
int timeoutSec = timeoutConfig.getTimeout();
68+
System.out.println("[TIMEOUT DEMO] --- Scenario 1: overall timeout ---");
69+
System.out.println("[TIMEOUT DEMO] hanging server https://localhost:" + port + ", timeout=" + timeoutSec + "s");
70+
71+
// Call #1 (cold): includes one-time client build + class loading + SSL init (outside the callTimeout clock).
72+
long cold = timedCall(timeoutClient, req, "cold #1");
73+
// Call #2 (warm): client + classes already initialized -> ~= the pure callTimeout ceiling.
74+
long warm = timedCall(timeoutClient, req, "warm #2");
75+
76+
System.out.println("[TIMEOUT DEMO] cold call #1 : " + cold + " ms");
77+
System.out.println("[TIMEOUT DEMO] warm call #2 : " + warm + " ms (~= callTimeout ceiling = " + (timeoutSec * 1000) + " ms)");
78+
System.out.println("[TIMEOUT DEMO] one-time init overhead (cold - warm) = " + (cold - warm) + " ms");
79+
80+
// ---- Scenario 2: per-phase `readTimeout` fires before the overall `timeout` ----
81+
VaultConfig phaseConfig = new VaultConfig();
82+
phaseConfig.setVaultId("demo_vault");
83+
phaseConfig.setVaultURL("https://localhost:" + port); // same black-hole server; stalls the TLS-handshake read
84+
phaseConfig.setEnv(Env.PROD);
85+
phaseConfig.setCredentials(credentials);
86+
phaseConfig.setTimeout(30); // seconds — generous overall ceiling
87+
phaseConfig.setReadTimeout(2); // seconds — but each read must make progress within 2s
88+
89+
Skyflow phaseClient = Skyflow.builder()
90+
.setLogLevel(LogLevel.ERROR)
91+
.addVaultConfig(phaseConfig)
92+
.build();
93+
94+
System.out.println("[TIMEOUT DEMO] --- Scenario 2: readTimeout wins ---");
95+
System.out.println("[TIMEOUT DEMO] timeout=" + phaseConfig.getTimeout() + "s, readTimeout="
96+
+ phaseConfig.getReadTimeout() + "s");
97+
timedCall(phaseClient, req, "warm-up"); // absorb one-time init so the next call reflects the phase ceiling
98+
long readBounded = timedCall(phaseClient, req, "read-bounded");
99+
System.out.println("[TIMEOUT DEMO] read-bounded call : " + readBounded + " ms (~= readTimeout = "
100+
+ (phaseConfig.getReadTimeout() * 1000) + " ms, well under the " + (phaseConfig.getTimeout() * 1000)
101+
+ " ms overall timeout)");
102+
103+
serverSocket.close();
104+
}
105+
106+
private static long timedCall(Skyflow client, DetokenizeRequest req, String tag) throws Exception {
107+
long start = System.currentTimeMillis();
108+
// timeout surfaces in response.errors, does not throw
109+
com.skyflow.vault.data.DetokenizeResponse resp = client.vault().bulkDetokenize(req);
110+
long ms = System.currentTimeMillis() - start;
111+
System.out.println("[TIMEOUT DEMO] " + tag + " error surfaced: " + resp.getErrors());
112+
return ms;
113+
}
114+
}

‎v3/src/main/java/com/skyflow/Skyflow.java‎

Lines changed: 40 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,9 @@ public static final class SkyflowClientBuilder extends BaseSkyflowClientBuilder
5757
private final LinkedHashMap<String, VaultController> vaultClientsMap;
5858
// Client-wide HTTP config defaults (apply to all vaults unless a vault overrides). null => SDK default.
5959
private Integer timeout;
60+
private Integer connectTimeout;
61+
private Integer readTimeout;
62+
private Integer writeTimeout;
6063
private Integer maxRetries;
6164
private Long initialRetryDelayMillis;
6265
private Long maxRetryDelayMillis;
@@ -84,7 +87,8 @@ public SkyflowClientBuilder addVaultConfig(VaultConfig vaultConfig) throws Skyfl
8487
} else {
8588
this.vaultConfigMap.put(vaultConfigCopy.getVaultId(), vaultConfigCopy); // add new config in map
8689
VaultController controller = new VaultController(vaultConfigCopy, this.skyflowCredentials); // new controller with new config
87-
controller.setCommonHttpConfig(this.timeout, this.maxRetries, this.initialRetryDelayMillis, this.maxRetryDelayMillis);
90+
controller.setCommonHttpConfig(this.timeout, this.connectTimeout, this.readTimeout,
91+
this.writeTimeout, this.maxRetries, this.initialRetryDelayMillis, this.maxRetryDelayMillis);
8892
this.vaultClientsMap.put(vaultConfigCopy.getVaultId(), controller);
8993
LogUtil.printInfoLog(Utils.parameterizedString(
9094
InfoLogs.VAULT_CONTROLLER_INITIALIZED.getLog(), vaultConfigCopy.getVaultId()));
@@ -114,6 +118,39 @@ public SkyflowClientBuilder timeout(int timeout) {
114118
return this;
115119
}
116120

121+
/**
122+
* Client-wide per-attempt connection-establishment timeout in seconds. Applies to all vaults
123+
* unless a vault overrides it; when unset the underlying HTTP client default (10s) applies.
124+
* The overall {@code timeout} still bounds the whole call, including retries.
125+
*/
126+
public SkyflowClientBuilder connectTimeout(int connectTimeout) {
127+
this.connectTimeout = connectTimeout;
128+
propagateHttpConfig();
129+
return this;
130+
}
131+
132+
/**
133+
* Client-wide per-attempt response-read timeout in seconds. Applies to all vaults unless a
134+
* vault overrides it; when unset the underlying HTTP client default (10s) applies. The overall
135+
* {@code timeout} still bounds the whole call, including retries.
136+
*/
137+
public SkyflowClientBuilder readTimeout(int readTimeout) {
138+
this.readTimeout = readTimeout;
139+
propagateHttpConfig();
140+
return this;
141+
}
142+
143+
/**
144+
* Client-wide per-attempt request-write timeout in seconds. Applies to all vaults unless a
145+
* vault overrides it; when unset the underlying HTTP client default (10s) applies. The overall
146+
* {@code timeout} still bounds the whole call, including retries.
147+
*/
148+
public SkyflowClientBuilder writeTimeout(int writeTimeout) {
149+
this.writeTimeout = writeTimeout;
150+
propagateHttpConfig();
151+
return this;
152+
}
153+
117154
/** Client-wide retry attempt count. Applies to all vaults unless a vault overrides it. */
118155
public SkyflowClientBuilder maxRetries(int maxRetries) {
119156
this.maxRetries = maxRetries;
@@ -137,7 +174,8 @@ public SkyflowClientBuilder maxRetryDelayMillis(long maxRetryDelayMillis) {
137174

138175
private void propagateHttpConfig() {
139176
for (VaultController vault : this.vaultClientsMap.values()) {
140-
vault.setCommonHttpConfig(this.timeout, this.maxRetries, this.initialRetryDelayMillis, this.maxRetryDelayMillis);
177+
vault.setCommonHttpConfig(this.timeout, this.connectTimeout, this.readTimeout,
178+
this.writeTimeout, this.maxRetries, this.initialRetryDelayMillis, this.maxRetryDelayMillis);
141179
}
142180
}
143181

‎v3/src/main/java/com/skyflow/VaultClient.java‎

Lines changed: 36 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,9 @@ public class VaultClient {
4949
private String currentVaultURL = null;
5050
// Client-wide (Skyflow builder) HTTP config defaults; null => fall back to the SDK defaults below.
5151
private Integer commonTimeout;
52+
private Integer commonConnectTimeout;
53+
private Integer commonReadTimeout;
54+
private Integer commonWriteTimeout;
5255
private Integer commonMaxRetries;
5356
private Long commonInitialRetryDelayMillis;
5457
private Long commonMaxRetryDelayMillis;
@@ -84,9 +87,13 @@ protected void setCommonCredentials(Credentials commonCredentials) throws Skyflo
8487
* Client-wide HTTP timeout/retry defaults from the Skyflow builder. Nulls out the cached client
8588
* so the next call rebuilds with the new values.
8689
*/
87-
protected void setCommonHttpConfig(Integer timeout, Integer maxRetries,
90+
protected void setCommonHttpConfig(Integer timeout, Integer connectTimeout, Integer readTimeout,
91+
Integer writeTimeout, Integer maxRetries,
8892
Long initialRetryDelayMillis, Long maxRetryDelayMillis) {
8993
this.commonTimeout = timeout;
94+
this.commonConnectTimeout = connectTimeout;
95+
this.commonReadTimeout = readTimeout;
96+
this.commonWriteTimeout = writeTimeout;
9097
this.commonMaxRetries = maxRetries;
9198
this.commonInitialRetryDelayMillis = initialRetryDelayMillis;
9299
this.commonMaxRetryDelayMillis = maxRetryDelayMillis;
@@ -175,8 +182,12 @@ protected void updateExecutorInHTTP() {
175182
vaultConfig.getInitialRetryDelayMillis(), commonInitialRetryDelayMillis, DEFAULT_INITIAL_RETRY_DELAY_MILLIS);
176183
long maxRetryDelayMillis = resolveLong(
177184
vaultConfig.getMaxRetryDelayMillis(), commonMaxRetryDelayMillis, DEFAULT_MAX_RETRY_DELAY_MILLIS);
185+
// Per-attempt timeouts: null => leave OkHttp's built-in 10s default (backward compatible).
186+
Integer connectTimeout = resolveNullableInt(vaultConfig.getConnectTimeout(), commonConnectTimeout);
187+
Integer readTimeout = resolveNullableInt(vaultConfig.getReadTimeout(), commonReadTimeout);
188+
Integer writeTimeout = resolveNullableInt(vaultConfig.getWriteTimeout(), commonWriteTimeout);
178189

179-
sharedHttpClient = new OkHttpClient.Builder()
190+
OkHttpClient.Builder httpBuilder = new OkHttpClient.Builder()
180191
.connectionPool(new ConnectionPool(10, 1, TimeUnit.MINUTES))
181192
.callTimeout(timeoutSeconds, TimeUnit.SECONDS) // overall ceiling; bounds the whole call incl. retries
182193
.addInterceptor(new RetryInterceptor( // OUTER: retries (Fern generated; jitter default 0.2)
@@ -186,8 +197,18 @@ protected void updateExecutorInHTTP() {
186197
.header("Authorization", "Bearer " + this.token)
187198
.build();
188199
return chain.proceed(requestWithAuth);
189-
})
190-
.build();
200+
});
201+
// Per-attempt phase timeouts; only override when explicitly configured.
202+
if (connectTimeout != null) {
203+
httpBuilder.connectTimeout(connectTimeout, TimeUnit.SECONDS);
204+
}
205+
if (readTimeout != null) {
206+
httpBuilder.readTimeout(readTimeout, TimeUnit.SECONDS);
207+
}
208+
if (writeTimeout != null) {
209+
httpBuilder.writeTimeout(writeTimeout, TimeUnit.SECONDS);
210+
}
211+
sharedHttpClient = httpBuilder.build();
191212
apiClientBuilder.httpClient(sharedHttpClient);
192213
}
193214
}
@@ -200,6 +221,17 @@ private static int resolveInt(Integer vaultLevel, Integer clientLevel, int defau
200221
return clientLevel != null ? clientLevel : defaultValue;
201222
}
202223

224+
/**
225+
* Resolve an optional int setting: vault-level override, else client-wide default, else null.
226+
* Null means "not configured" — the caller leaves the underlying HTTP client default in place.
227+
*/
228+
private static Integer resolveNullableInt(Integer vaultLevel, Integer clientLevel) {
229+
if (vaultLevel != null) {
230+
return vaultLevel;
231+
}
232+
return clientLevel;
233+
}
234+
203235
/** Resolve a long setting: vault-level override, else client-wide default, else SDK default. */
204236
private static long resolveLong(Long vaultLevel, Long clientLevel, long defaultValue) {
205237
if (vaultLevel != null) {

0 commit comments

Comments
 (0)