Configuring connection pool setting – maxRequestsPerHost in openai-ai java
Summary A production Java service using the openai-java SDK experienced intermittent timeout errors and elevated latency when calling OpenAI APIs under load. The root cause was an unconfigured OkHttp connection pool, specifically the maxRequestsPerHost limit. The default OkHttp client settings (5 requests per host) bottlenecked concurrent API calls, causing requests to queue and eventually time … Read more