November 20, 2014

Service Design Patterns in Rails: Web Service Evolution

This is my fourth post on Service Design Patterns in Rails. These posts have been inspired by the Service Design Patterns book by Robert Daigneau.

The previous posts were:


The Web Service Evolution patterns talk about what to do to make the API evolve with the minimum breaking changes. What that means? It means that changing your API should not break clients that consume it, otherwise said, that your API is backward compatible.

From this chapter, three patterns apply to the Rails framework:

  • Tolerant Reader
  • Consumer driven contract
  • Versioning

Tolerant Reader refers to the client. It means that if you write a client, you should be "tolerant" on what you get as a response. Rails provides the ActiveResource object. This object will get the response and create an object from it. This object per se is very tolerant.

However, only using that object does not make you tolerant reader. ActiveResource will create a model object like ActiveRecord does. For example, you can have a Product object that inherits from ActiveResource, the same way you would with ActiveRecord

  class Product < ActiveResource
  end

ActiveResource will create an attribute for each attribute found in the response. Thus if you expect a "name" attribute, you would try:

  Product.find(1).name

However, what if name is not in the response? Then, if you really want to be tolerant, you should do

  p = Product.find(1)
  if p.respond_to?(:name)
    p.name
  end

Thus, with little effort you can have a tolerant reader, even thought Rails per se does not fully implement it.
 
Consumer driven contract means testing. Actually, it means having a test suite provided by your consumer on what it expects from the service. Testing is a common good practice in Rails and Rails provides you with a lot of resources for that (rspec, factory girl, stubs, mock ups). You only need to use them to specify what you expect from the API, or what a consumer should expect, depending on whether you are writing the producer or the consumer.

And finally, versioning. Versioning is a very good practice for an API. How you can implement versioning is very well explained on this rails cast, better than I could do myself, so I'd recommend you watching it, to see how to use namespaces in the routes file and in the controller implementation:

 
 
Web Service Evolution patterns are partly implemented on Rails. However, it needs you to do some implementation in order to follow them, because the typical example (the one you would generate with "rails generate") does not implement them, but give you the tools or the base so that you can do it.
 
For example, versioning is not implemented when you generate a new rails application, but you can easily do it with namespacing; tests are not implemented but rails gives you the tools to do so; tolerant reader can be implemented by using ActiveResource and respond_to? method.

Thus, my conclusion is that Rails, even thought it is very cool for simple applications since "rails generate" can give you what you need right away, when it comes to more serious things, you need to do some work yourself.
 
This means that, despite the fact that when you start using rails for simple things you don't need to know about patterns, if you want to do more, you need to know them.



November 16, 2014

Service Design Patterns in Rails: Web Service Implementation Styles

This is the third post on Service Design Patterns in Rails. The first two where



 In these series of posts, I am trying to map the patterns described in the Service Design Patterns book by Robert Daigneau to a typical Rails application.

So far, I've proved that Rails already implements some of the patterns described in the book and thus, by only using the Rails framework, you are already following those patterns. This means you already follow certain good practices on implementing services without actually knowing about them.

However, we've also seen that some patterns are not implemented by default. The common thing on them is that they are related to the response: Response Mapper, Response Data Transfer Object Mapper, and Linked Service.

The chapter about Web Service Implementation Styles, is about five patterns:

  • Transaction script
  • Datasource adapter
  • Operation script
  • Command invoker
  • Workflow connector

Transaction script pattern means writing a service that, without any helper, nor model, accesses the database. This is not the Rails way, since in Rails we access the database through a domain model object. Personally, I think it is good that this pattern is not in the Rails framework. I am surprised to have find out this pattern in the book.

Datasource adapter means having a class that will get the data from the database, making the controller unaware of, for example, if the data is stored on a postgresql db or sqlite, and managing metadata, like the relationships between different entities (i.e. one-to-many, belong-to, many-to-many). Does it sound familiar? Yes! This looks like ActiveRecord, doesn't it?

Operation script means delegating work to a class that can be used by different web-services. Translating that to Rails means implementing features in the model so that different controllers can reuse it.

Command invoker means having a class that can be either call from a web service or put to a queue to be run later. For example, a class with a perform method can be run by calling the perform method, or queued to a delayed job queue. This may not be the preferred way of using delayed job, but certainly can be done (See Custom Jobs) . This class will then call domain model methods.

Workflow connector is something more complicated. It means designing a workflow where each step can be a web service. However, you need to maintain information on the state and be able to rollback (typically done by compensating, by calling a web service that performs the negative function).

I don't think you can implement a workflow in Rails with what the framework provides, nor with the typical gems. From what I know, you will need to add additional gems/tools.


So far, I've seen that rails implements the patterns for a basic REST API, which is cool. However, when it comes to more complex workflows where the services you implement are part of a bigger services infraestucture, rails does not implement patterns for that, like the Workflow Connector, the Response Mapper or the Linked Service.

November 10, 2014

Service Design Patterns in Rails: Request and Response Management

This is the second post on the Service Design Patterns in Rails. The first one was about Client-Service Interaction styles patterns. This one is about Request and Response Management patterns (from the book Service Design Patterns by Robert Daigneau).

The Request and Response Management patterns are:
  • Service Controller
  • Data Transfer Object
  • Request Mapper
  • Response Mapper

Service Controller pattern means that there is a class that decides which controller should be called. The decision is made based on the Request (for example "GET /customer/123") and a set of rules.

Does it sound familiar? Yes! This is the routes.rb file. Here is the simplest rule of all:

resources :products

which means, any request to /products will be routed to the products controller.

The Data Transfer Object is an intermediate object used in the request or the response instead of using the domain objects. That is, instead of serializing the Products object on the typical rails example, you would instead use another object populated from the values in the Product.

On the request side, this is achieved through the params object. See this code which is typically in the typical Products controller example:


    def set_product
      @product = Product.find(params[:id])
    end


this is how we find in the database a product with id "params[:id]".
"params[:id]" is the 123 from "GET products/123" request.

Another example, when we want to update, how we update it with the params object:

def update
  respond_to do |format|
    if @product.update(params)
    ....
    else
    ...
    end
  end
end


However, in the response side I have bad news. A rails typical application is not using a Data Transfer Object. See the typical example:

class ProductsController < ApplicationController
  def index  
    @products = Products.all        
    respond_to do |format|                                                      
      format.json { render json @products}   

    end
  end
end

This code means that the object used in the response, is the Products object serialized as json. Under the hoods, the "ActiveRecord::to_json" call is made for this purpose. This means that the response is highly coupled with the domain object.

What we should do instead is rendering the response in the view, by using a template.

Rails does come with builder for "xml templating". If we were using xml for the response, the following code will work as long as we had an index.xml.builder file.

class ProductsController < ApplicationController
  def index  
    @products = Products.all        
    respond_to do |format|                                                      
      format.xml  
    end
  end
end

Thus, for json to work, we would expect something like:


class ProductsController < ApplicationController
  def index  
    @products = Products.all        
    respond_to do |format|                                                      
      format.json
    end
  end
end


However, rails does not come by default with "json templating" and thus the previous code won't work unless we add some "json templating" gem.

Researching a bit on google, I've found this post that explains the case very well and much better than I could and gives some options on "json templating":



Finally, Request and Response Mapper are two patterns that can be used to interact with different web services, different on the syntax, but semantically equivalent. In that situation, you need an object that maps different request/responses to the same controller.


In the case of the Request Mapper, we get back to the routes.rb file, where you can declare matching patterns. For example if you were authenticating against github, you may have this line on your routes.rb file

  get "/auth/:provider/callback" => "sessions#create"

The Response Mapper is used to construct a response but used by different web services. This can be used when you need to create a response that matches some kind of agreement between different parties. Sadly I don't know of a typical rails example that does that. If you know one, please tell me.
 
 

November 6, 2014

Service Design Patterns in Rails: Client-Service Interaction styles

I have started reading this book that have been on my shelf for several months accumulating dust. The book is called "Service Design Patterns" by Robert Daigneau. Actually it looked cool on my shelf next to the "Design Patterns in Ruby".

Anyway, I started reading it some days ago and while reading it, I've been trying to match the patterns described in it with what I know from having worked on Rails for the last few years. The examples in the book are Java and C#. It was a nice surprise to see that a typical Rails application already implements most of them, thus if you use Rails, you are already using them, the same way you use the MVC pattern.

Thus, today I'll write a post on how Rails implements the Client-Service Interaction Styles patterns described in this book. Hopefully, I won't put the book back to the shelf for more dust and this post will be followed by other posts on the other patterns.

Client-Service Interaction Styles patterns are:

  • Request/Response
  • Request/Acknowledge
  • Media Type Negotiation
  • Linked Service


Request/Response

This is the simplest pattern and the default one. The Request/Response is nothing more than sending a Response to a Request. For example, if we take the typical Rails example application for maintaining a Product, the product controller would contain:

class ProductsController < ApplicationController
  def index
    @products = Product.all
  end
end

Thus, a request like 

GET /products.json

will return a Response with all the Products.


Request/Acknowledge

Request Acknowledge means that a Request will return without actually processing the data and no returning any data but an acknowledge code, and the data will get processed by another process in the background.

This can be accomplish with the delayed_job gem. In short, the delayed_job stores a task in the database instead of running that task, and then a background job will get that task and run it. This is very appropriate for long-running tasks so that you can give back control to the client and do the task later in the background.

For a task to be delayed, you just call:

@post.delay.some_long_running_task

where some_long_running_task is the task you want to be delayed.

This is the typical architecture of having a worker dyno running a delayed job in Heroku.


Media Type Negotiation

Getting back to the typical Products example, media type negotiation means that the client asks for a media type and the server creates a different response. In rails this means that these two requests:

GET /products.json
GET /products.xml

will produce different responses with the same data. For example:

class ProductsController < ApplicationController
  def index  
    @products = Products.all        
    respond_to do |format|                                                             format.xml  { render xml  @products}
      format.json { render json @products}                                           end
  end
end


Linked Service

The Linked Service patterns means that the response includes links to other services, so that the client should parse that information before deciding the next request.

Unfortunately, I haven't found any example nor in the rails itself nor in the gems that I know of such a pattern. If you know of a linked service example typically used in Rails, please respond to this post.

October 31, 2014

Dice - A light weight build service

Last week was Hackweek on SUSE and for 2 days I hacked on Marcus' project "Dice - A light weight build service".

It was fun and Marcus code was very easy to understand, very well structured and with comprehensive tests.

Dice is a simple build service for KIWI images using virtual instances controlled by vagrant or a directly contacted build machine. It can be used to fire up build jobs on e.g public cloud instances.

What that means is that you can do:

>dice build myimage

and that will either:

1- start a virtual machine on your workstation/laptop and build your image IN that virtual machine

2- connect to a virtual machine on the cloud (i.e. google cloud) and build your image IN that cloud virtual machine


And why all the trouble? the reasons:

1- setting up an environment for building images on your laptop/workstation can be sometimes paintful

2- running multiple builds on your laptop/workstation will make your host performance get low. Builds take time, thus you normally are doing something else meanwhile, and running the build on the cloud can be very good so that you can use your resources for something else

3- security: building an image implies running custom scripts. If you have done this scripts, fine, but if not, better not run it on your laptop/ws.

4- availability: having a build service on the cloud, makes it available to others that won't have to invest time on setting it up

During those 2 days, I just implemented the ssh command as:

> dice ssh myimage

which will open an ssh connection to the build node, either virtual machine on you laptop/ws or in the cloud, so that you can easily debug when a build fails.