Caching External Service API Calls

February 4, 2013 Link to post  Permalink

In a recent project, there was an API call to an external service to retrieve some data. The code was written in the easiest way possible, which is to make the service call when the data is needed. For this application, the data is needed to populate a drop-down in a form, so the user is waiting for this call to complete before they can see the page they requested.

This particular service takes at least 1 second, and usually 1.5 seconds to return the data needed. Every single time the user visits this page. This clearly needs to change.

Cache the Service Call

If you have service calls where the parameters to the call are likely to be re-used, and the results for a given combination of parameters is the same, then you can easily cache the service results and re-use them across requests.

In my project, the service call was performed inside a Rails view helper method during rendering of the content. The easiest way to cache this call is to use the Rails fragment cache view helper to avoid the call:

# foo.html.erb
<% cache [ "v1", "obj_name", @obj.param_1, @obj.param_2 ] do %>
  <%= helper_call_to_service @obj.param_1, @obj.param_2 %>
<% end %>

If the service data changes fairly frequently, you might want to add an expires_in parameter to the cache to cause the service to be called periodically to refresh the data. This cache refresh, although infrequent, is still performed while the user is waiting, and this isn’t ideal

Save the data permanently

An alternative to using the Rails caching and storing the data in memcache is to store the service result in the database. Now, on the face of it, this will be slower than using the memcache, but using the database gives you the opportunity to avoid future cache refresh calls from delaying the user.

Here’s the flow:

  1. Query the database - if data not found
    1. Make the service request
    2. Save result into database
  2. Return to user

Subsequent calls will then access only the database, and these calls can then be cached to improve speed.

So far, all we’ve added is complexity, where’s the payoff?

Now that we’ve got the data in a database, it’s easy to write a task that cycles through the cached data and makes service calls to get updates to that data. This service update code can be scheduled as necessary based on how often you think the data could change. After the first cache miss, the user will never need to wait for a service call, yet the service results can be kept as current as you need them to be.

Pre-cache the External Service

For the app that I am working with, there’s actually a possibility to pre-cache the service request. This is the holy grail of isolating the user from external services completely. Here’s how this can work.

Somewhere else in the application, the data that is used for the service request parameters is entered. If you have some form of queuing system in your Rails application, e.g. Resque or Sidekiq, you can add a message to the queue to have a background worker process perform the initial external service call, and hopefully this occurs before the user visits the page where the result is needed.

This is part of why message queuing is a useful part of high performance applications.

Summary

If you can cache external service calls, then you should. If you can store them for later and then de-couple the user interaction from the external service completely, then you’ll have better control over the response time your user experiences, with only a little extra complexity.

If you can avoid the external service call by pre-calling the service ahead of time, you can also avoid having the external service response times causing slowness for your users.

The next issue of my Faster Rails Newsletter will be be released at the end of February. Use the sign up form to the right now to ensure you get it when it is available.