Thursday, April 7, 2011

Using Goliath to Integrate With External Services

In this post I am going to explore how you can use Goliath, an asynchronous Ruby framework to enable browser integration for services that lack JSONP api's as well as how to integrate these external services with your existing synchronous web stack.

In a previous post I talked about JSONP and how you can add support for it to your Rails app. But what if you want to add JSONP to someone else's API? Chances are you can't. However all hope is not lost, we can use a asynchronous framework such as goliath to proxy the requests from an api that does not support JSONP. As you can see here we simply proxy the request using em-http-request:


Although this code doesn't look it thanks to em-synchrony it is asynchronous. The http request to get the external resource will not block the handling of further incoming requests as it would in a synchronous framework. It is important to note that calling to external services in your synchronous service is consider a scalability anti-pattern. By calling to external services inside a synchronous web request you are blocking the handling of further requests until the external service responds. If the external service becomes slow or you experience a burst in traffic to an action with an external call you will be serving 503's before you know it.

As you can see proxying external resources for use in the browser is very straight forward. However what if we actually need our existing synchronous Rails stack to be involved in the request? For example lets say that we want to re-use an existing partial to render the JSON that returns from an external service?

I wrote the forwarding support middleware to achieve this end. You can see the source here:


It takes the JSON that comes back from the external API and appends it to the forward_to url and sends the request on. The request flow would look something like this:



I think this is a really effective pattern to integrate a existing synchronous web stack with a asynchronous service. The asynchronous service is incredibly generic, it does one thing and does it very well. Its stateless nature makes it highly scalable. It also enables you to still build rich users experiences and integrate with external services while avoiding blocking requests in your synchronous web stack.

You can check out the complete source here: https://github.com/nolman/proxy_service