If your web application calls out to third-party APIs, set aside some time to review the timeouts surrounding those requests.
- Carefully choose a timeout for the initial connection attempt. Choose a low timeout and perhaps add a retry if you really need the call to go through.
- Carefully choose a timeout for the overall request. How long are you willing to make your users wait for the response?
- Do not rely on the default timeouts set by your HTTP client library, language runtime, or HTTP server. These defaults may not be sane. If they are sane right now, they may not be in the future. That is how life goes, sometimes.
- Consider the user experience, application performance, and business requirements.
- Be able to tell what the timeouts are by looking at your code.
- Consider how your application should proceed despite having timed out. It would be nice to float on, but sometimes one must show an error message or do something else.
- While you’re at it, make sure you are correctly handling edge-cases. What if you get an unexpected response from the API call? A new error code? A baby in a basket on your doorstep? What will you do then?
- Strongly consider logging latency, success count, failure count and timeout count for all API calls. Push the data into something like StatsD+Graphite+Grafana so you can graph it.
If you don’t carefully set timeouts, you are letting someone else control the quality and user experience of your web application. You may also be wasting precious CPU cycles waiting for mis-behaving third-party services. This is not something you should let happen. Program defensively against these things.