Challenges Faced While Testing 3rd-Party APIs
Many applications integrate third-party offerings into their product to build meaningful products, one famous way is via APIs. This kind of approach benefits both parties, the offerer can generate revenues and the product using it gets a relatively easy way to get the job done.
However, it presents many challenges while integrating and testing such integrations. Let's discuss a few of them.
- Lack of documentation
Lack of documentation can expand to many things, starting from proper API documentation, contracts, schemas, usages etc. E.g., if you don't have proper documentation of error codes from the third party then it would be difficult to debug the issues and hence handle them in the code. Another one is schema if the API schema changes based on the data then without that knowledge we will get false negatives. Also without knowing the limitation of the API later it might need unnecessary fixes which could have easily been handled during the dev cycle.
2. VPN issues
Suppose your third party allows connecting to their server via their VPN only. In that case, you lose control, plus as a customer to them, one needs to have proper alerts and key rotation mechanisms to proactively monitor if anything goes wrong. Low bandwidth can also hinder the performance of your product during peak times, hence outages can happen. Whitelisting IP addresses can also take time due to other dependencies. Also one needs to be vigilant about changes or expiration of SSL certificates or any other thing done by the third party so keep an eye on the emails.
3. Lack of testing environment
Often, third parties will not have the same number of test environments as you have, hence it becomes difficult to do E2E testing. A couple of ways in which one can unblock it is by pointing all the lower environments to the same IP which is used for testing and whitelisted by the third party or you can provide different IPs for each environment and ask them to whitelist it to their production and test servers. This approach reduces the effort of constantly maintaining the config at your end. Test data can also be a pain to generate in these cases, hence better to get some valid test data from the third party which can be used for integration testing. Having a mock server can also be very useful build-up an automated regression suite to check the corner cases.
4. Incorrect timeouts
Timeouts are a very tricky but effective way to prevent cascading failures. If there is latency from third party end and our services depend on the data but since that data is not available for downstream services in the given amount of time, it might choke up the funnel as it will hold the connections for a long time. Having logical read, socket and transactional timeouts can help to reduce these kinds of failures. Also, circuit breakers can prove effective in temporarily completely disconnecting the system from the third party, it goes without saying that the flow will be impacted however it can reduce the blast radius due to the failure.
5. Absence of fallback
Fallbacks are essential if one fears the business flow is prone to failure. While developing it is essential to explore the fallback strategies to ensure the business continues to work also testing this fallback is equally important as the logical flow. Adding a feature flag can be a lifesaver to turn off the feature if something goes wrong.
Nonetheless, having a well-thought-out deployment and scaling strategy can help to understand the behavior, for e.g. A/B mode. Based on the third party there could be different sets of challenges.
