Why REST ?

Mon 08 June 2009

Retrieving comment count

Introduction

It is becoming apparent that even as it becomes popular, REST (Representational State Transfer) is not yet as well understood. This might seem a surprising statement, but a lot of us use REST thanks to many frameworks supporting REST like interfaces, have a sense of what REST like interfaces are like (even if such an understanding is not sufficiently accurate), and exercise our common sense in using such interfaces. Having said that, let me clarify that while the internet is full of documentation about the semantics of REST, its actually quite light on the rationale for REST (including Roy Fielding's dissertation which is the reference document for REST). Thus I have had to intersperse REST semantics and historical narrative with some personal opinions. So treat this as a part opinion and feel free to question my thought where you think it does not make sense.

Audience

If you are a REST expert, you are likely to have figured out much of this any ways by now. If you would like to understand specific technical semantics about REST, again this may not be the best article to read. However if you are curious about REST and would like to read a perspective on why and how it makes sense read on.

Flow

I shall be meandering through a historical narrative in the first half before starting to make the points I wish to make in the second. Lot of the points I make in the first half are likely to be those you already are aware of. However these are being made to allow an immediate recall when you read the second half. It is quite important to have read the first half to understand the perspective I put together in the second.

Historical Narrative

FTP

I am sure you have used FTP (File Transfer Protocol) a few times even though nowhere as frequently as HTTP (HyperText Transfer Protocol). Let me quickly present some characteristics of FTP

So FTP allows us to upload and download files. But does it allow us to 'do things' ? Sure, so long as you combine it with a few more pieces in the puzzle. Let us say, we are back in the late 80s (prior to the invention of HTTP) and I want to send and a list of purchase orders collected by a local office to a central office for further processing. This requires the following elements to be added to the mix.

Ho Hum. This was all stuff you knew. But the reason I brought it up is that FTP and how it was leveraged then has a lot to do with the principles that govern REST as we shall later see.

RPC

Back then in the 80's FTP wasn't the only mechanism to transfer data between machines. One more of the many other options was RPC (Remote Procedure Call). It not only allowed you to transfer data across machines, it actually had built into itself, a contract to remotely execute software. Unlike FTP which merely transferred data (in well understood units called files), RPC allowed you to invoke remote procedures by supporting an ability to pass messages which included the message name and the values for all the parameters necessary to be supplied to the message. Unlike FTP which was meant to do data transfer across a network, RPC was geared to do things remotely.

A contrast between FTP and RPC

If the objective of network computing is to use the computers and networks to 'do things' one would assume that many more people would use RPC than FTP for the same. While RPC did get used as a technical substrate, at a business processing level FTP got used far more (eg. to send and remotely process the list of purchase orders). There are some important reasons that we need to understand here.

FTP required understanding of very few basic verbs (ls, cd, get, put). Thus the training required to understand FTP semantics was far less than that for RPC. This was partially due to the fact that RPC had a programmatic interface. To the best of my knowledge there are no widely used command line clients for human interaction with RPC services. In addition, each procedure required a set of semantic data (parameter) associated with it. This was no different than FTP which also required similar data to be shipped over the network. Turns out there were a few distinctions. First, the nature of design of RPC services often required combining application data with control data, and there was also often a sequential expectation due to the RPC business transaction being broken up into multiple RPC calls perhaps for the sake of efficiency. Moreover each time, new procedures were added or parameters added, these required programmatic changes. FTP on the other hand was simpler. In most cases the entire data (including some redundant data perhaps) was sent in one block (or file in FTP parlance). By dealing with a file as the least common denominator, the FTP stack decoupled itself from any application specific semantics. Moreover, depending upon the agreed upon format, the files could be edited at either end by by human actors using specific software such as plain text editors or word processors or spreadsheets. Moreover if the formats changed, wherever such files were being manually edited, no programmatic changes were required. Irrespective of the changes you made to the file formats, file processing software, the FTP stack itself did not change - it remained stable.

Less is more

A theme I shall come back to again is many a times less is more. FTP had far fewer training requirements (few basic verbs). FTP did not deal with parameter value formatting (though other pieces of software subsequently might have to). FTP was just so much easier to start working with. FTP did not actually preclude any of the capabilities of RPC from being introduced, it merely allowed this to be added subsequently as additional optional layers (or subsequent elements in the processing pipeline). Finally FTP allowed users to deal with the data in the units and the formats and the tools they understood the best - their day to day application software components and simply focused on only transferring files, while imposing only one requirement - each end should work with a file as a unit, and both ends should understand the file formats. By focusing on file as a unit, each business user could focus on the data he/she wanted to deal with in the format that was most appropriate (an analogy in REST would be a resource .. but I'm getting ahead of myself). And at the end of the day, by doing less, FTP ended up being much more popular and thus doing more.

Email

Other protocols widely used on the net were SMTP / POP which were used for email. Email eventually was considered the killer app for the internet. Similar to FTP email focused on the users getting to learn only a few basic verbs and exchanging the basic unit of data transfer (messages) using these verbs. Again, even though email itself didn't get things done, it contributed far more heavily than RPC to getting things done, by having other manual or software actors at either end of the messages who did the necessary processing required.

WWW (World Wide Web)

While email was the killer app for the internet, the one that really brought it to masses was the world wide web which was based on the HTTP protocol. While HTTP could be used to ship documents of a variety of types (often classified by their mime-types), the defacto type of document used the HyperText Markup Language (HTML). Unlike FTP and email, this required the authors to understand a new language, but used a simple markup syntax to keep the learning curve to the minimum. It however introduced a very powerful element - the embedded hyperlink. While the earlier technologies supported a uniform identifier for each document / message, the hyperlink allowed references to other documents / messages to be embedded thus converting the document pool into a document web. We now had the ability to navigate from one document to another and such navigation retained the contextual relevance by embedding the hyperlinks. There were other enhancements as well such as introduction of more verbs (eg. POST and DELETE, the latter not really being supported by any of the browsers). Allow me to state the salient points of WWW despite the obvious duplications with some of the points I listed under FTP (for the sake of emphasis). Note that the scenario I describe below is primarily describing static web serving (except to the extent of file uploads) and does not address the presence of a dynamic web application.

Again the reason I listed these characteristics is that these have a tremendous commonality with those of REST (except that what I refer to as a document above may get referred to as a resource in REST parlance).

SOA : DCE, Tuxedo, CORBA, RMI

Even as the web was evolving other technologies which allowed for more sophisticated remote service invocations were being developed. Along with RPC, these were essentially different technical manifestations of Service Oriented Architecture (SOA) principles. While these are substantial developments in their own right, the relevant points to be made in the context of this article are :

CGI, dynamic web applications and Web Services

Clearly as WWW started getting used far more, people were only too keen to use it for much more than storing or retrieving documents. This led to the development of CGI and subsequently other dynamic web application technologies (eg. LAMP, J2EE etc.) which would allow us to use the web to 'do something'. Since these were clearly offshoots of the SOA world, being mapped onto the WWW infrastructure, the characteristics of such dynamic applications often had a lot in common with SOA, and they started dropping many characteristics of the traditional static WWW. Thus was born the child of the world wide web and distributed service oriented architectures - web services. This led to newer SOA technologies such as WS-* and SOAP.

Like the typical scenarios after the discovery of any highly profitable opportunity, the early rush was to leverage the opportunity and it was only a little later when the dust died down, that people started wondering if they had sacrificed something in the heat and dust of the moment. That stock taking resulted in the realisation, that some of the very basic characteristics of the extraordinarily successful internet technologies (FTP / SMTP / WWW) had been diluted, and even if such dilution still allowed immediate progress to have occurred, some of them would need to be corrected to be able to continue the explosive growth that had been seen so far. One such exercise in my opinion is the laying down of the REST architecture style.

REST Semantics

While REST brings back many of the characteristics that made internet so successful back to application design, it should be noted that many of these are not precluded by Web Services or SOA. However what are mandatory characteristics in REST are in some cases missing from but in most cases quite feasible to implement in traditional (non REST) web services by using additional best practices. Also note that each characteristic is not necessarily universally superior. So do evaluate it in your context to see if it makes sense. However before we get to the benefits of REST, a quick synopsis of REST technical characteristics might be in order.

While a full description of the REST technical aspects is completely beyond the scope of this post, I summarise these below. You might notice the strong parallels between the characteristics of FTP and WWW and those of REST even as REST adds a few more capabilities. The reason I portray them in the form below in a manner quite similar to the way I portrayed the characteristics of FTP and WWW is to emphasise that REST actually continues to leverage the same characteristics that made these technologies so popular and globally scalable, even as it just adds those few minimally necessary capabilities to achieve the same scalability for not just transferring documents or rendering pages but to 'do something'. In other words it brings together the characteristics which made the internet technologies so popular and applies them to the inter application integration, component and service orientation, and application mashup scenarios to allow them to achieve similarly large adoption and to perform the tasks necessary in the given context (or 'do someting' as I have continuously referred to).

REST characteristics

REST benefits

Having described many of the REST characteristics the following could be interpreted as the benefits of adopting a REST style architecture.

I have used the word scalability above in the context of the ability to service the runtime demands of a larger number of clients. However REST helps makes your software artifacts become scalable in one more way. By providing a basic and minimal uniform interface requirements, REST allows your applications / services / components a low entry barrier path into being a participant in a broad web of similar others who all agree on the basic REST semantics. This substantially increases the potential number of clients to your services since they can leverage these services easily and with low entry barriers. While traditional SOA technologies attempt to provide the universal access to all possible consumers, REST with its emphasis on minimalism, simplicity and low entry barriers actually makes it practical. Similarly REST makes it easy for you to start consuming other services and mashing them up with others to service your clients (pun intended) quickly. Finally REST takes the the very characteristics that made document and message sharing so easy to use and popular (characteristics which are not necessarily found in all conventional SOA implementations) and combines them with the necessary elements to achieve transaction processing, application integration and mashups (use the web to 'do something') on a truly global scale even as it makes these capabilities easily available and cost effective to leverage.

Some concluding comments

While not directly relevant as REST rationale or REST benefits, I thought it might be useful to add a few more associated comments within the context of REST usage and adoption.

Simplicity and bottom up adoption

I must confess, my biases show up quite strongly in this paragraph (so feel free to treat this as a partially prejudiced statement). Simplicity is not per se a characteristic of REST. However it does stem from the nature of genesis of the competing options. While most internet technologies using an incremental, evolutionary approach, most SOA technologies have been designed by a committee. This is why the consulting and development budgets required to implement FTP / email / Web especially on a per utility basis are far different than those likely for implementing DCE, Tuxedo, CORBA and SOAP. Part of the reason is also due to the fact that most internet technology adoption is bottom up, while that of SOA often is top down. While top down may seem attractive, it may seem sobering to realise that most top down processes break down beyond a particular scale. Thats why free markets on the whole have trounced centrally planned economies (though some recent happenings do point to limitations of the same as well). Thats why internet scale simple inter application API integration and mashups took off even as intranet scale application integration was mired in budgeting, territorial, enterprise modeling and governance issues. Thats why the LAMP stack (eg. PHP) which hasn't been particularly strong in the non web arena, is deeply entrenched in the web based application space. Sometimes it just is more productive to quickly implement a simpler technology and incrementally enhance it rather than attempting to cover all possibilities, options, and border conditions by putting a committee in place. At its very core, REST requires only incremental understanding of newer technologies, is easier to incrementally adopt and is less likely to get mired in organisational issues. Precisely the characteristics that FTP, EMail and WWW had.

Simpler abstractions win

I have generally found that simpler abstractions even though harder to deal with initially, often win in the long run. Notice the fact that the bare bones rendering functionality of HTML/WWW completely trounced the rich UI and application integration capabilities then available (eg. Windows/Java and DCOM/CORBA/RMI). This is not to suggest that the extra capabilities are not required. That is why Rich User Interfaces on WWW continue to be a dominant part of the internet technology wishlist. However the simpler, cleaner and minimalistic abstractions often are far more important than feature richness. A point I would want to make in favour of REST even as I admit that conventional SOA technologies are far more feature rich than REST.

REST is not SOA

I must confess, for a long time I believed REST was merely a specific usecase of SOA. However recent thoughts lead me to believe otherwise. There is indeed a reason for such potential confusion. REST based architectures and SOA may often attempt to service similar goals. To the extent of servicing such goals, REST may look like a substitutable component for other SOA technologies such as SOAP. However even as they attempt to meet similar goals, REST attempts to view at your architecture artifacts differently. REST encourages you to view and model your architecture as a set of resources rather than services. There are important implications of this not just in terms of the many benefits I describe above available under REST but also in terms of the design and architecture characteristics of the implementation. Treating REST as just another way to implement SOA sometimes encourages one to miss out on the subtleties. These however are beyond the scope of this post, and I intend to cover the same apart from the implications of REST on software design in my next post.

comments powered by Disqus