Month: April 2009

Is a large corporate making money off open source or open standards an oxymoron ? In a Sun / Java Context

Posted by – April 22, 2009

The recent acquisition of Sun by Oracle reignited a thought that had been going through my mind for a long time. It simply boils down to whether a large corporate can make money off open source or open platforms. Now quite obviously it in itself is not a truism. But the point remains that large corporates which have become large in the conventional economy do find the going a little difficult when trying to make money off open software.

The way I perceived it, Sun was going through similar difficulties. The hardware business was delivering dwindling margins post the dot com bust, and the software business was under threat from upstarts such as Linux which offered a substantially similar software stack at near zero licensing costs. One of the crown jewels asset Sun had in its stable was Java. And while it was (and continues to be) a wonderful asset, it just was incredibly difficult to make money off it. Now java hasn’t been open source exactly for most of its life, but it was a sufficiently open stack nevertheless to cause the same difficulties that open source software would in terms of monetisation.

There’s an excellent blog post written a year and half ago by Michel Bauwens, Can the experience economy be capitalist? which does refer to some of the underlying issues. I suggest you do read it, but would like to quote a few points from it below.

First of all, in the field of the immaterial, we are no longer dealing with scarce goods, but with marginal reproduction costs and non-rival goods. With such goods, sharing does not diminish the enjoyment of the good, since all parties retain their ability to use them. The emergence of peer production shows a new form of creating value, that is in fundamental aspects “outside the market”. Typically, in commons-based production we have a common pool, accessible to everyone (Linux, Wikipedia), around which an ecology of business can form to create and sell scarcities (usually services and experiences). In sharing-oriented production (YouTube, Google documents), we have proprietary platforms that enable and empower the sharing, but at the same time, sell the aggregated attention (a scarcity), to the advertising market. Finally, in the third crowdsourcing mode, companies try to integrate participation in their own value chain and framework.

So the good news is that indeed business is possible. But I would like the readers to entertain the following proposition, nl. That:

1) The creation of non-monetary value is exponential

2) The monetization of such value is linear

In other words, we have a growing discrepancy between the direct creation of use value through social relationships and collective intelligence (open platforms create near infinite value through the operations of the laws of Metcalfe and Reed), but only a fraction of that value can actually be captured by business and money. Innovation is becoming social and diffuse, an emergent property of the networks rather than an internal R & D affair within corporations; capital is becoming an a posteriori intervention in the realization of innovation, rather than a condition for its occurrence; more and more positive externalizations are created from the social field.

Quite eloquently argued here that open platforms create near infinite value which is difficult to be captured by business and money. I still remember companies such as Borland, HP, Sun, IBM making a fair amount of money through selling C/C++ compilers and IDEs (I remember it used to be almost $5000 per seat). However this was in the days when the community capability of creating similar competing software stacks was only in its infancy. No longer so today. Now communities, open standards, open processes and open source are fairly well established sources of delivering asymmetrically significant capabilities at a fraction of the cost, a fraction which is made even smaller when the same is incurred by a small motivated group of individuals or small highly agile companies. There in lies the difficulty. Companies are trained to and built to deliver linear and symmetric capabilities in the context of costs they incur. However they have a relatively poor handle on monetising in scenarios where small teams can deliver exponentially asymmetric capabilities.

The blog post I referred to above, identifies the right area to look at in this context – Identify what is scarce and Monetise the scarcity. So when any software has a sufficiently large utility and can be managed through open processes to be able to satisfy a globally substantial demand at a fraction of the cost (eg. apache httpd), that particular capability (eg. static and dynamic file serving over http) is no longer scarce enough for a commercially focused company to make money out off. Thats why the money shifts where the scarcity is – how to leverage such software and put it to good use. This is where the existing commercial successes around open source are based on eg. RedHat, JBoss, IBM, Oracle etc. They monetise the support, training, services and consulting around such software. In many ways Sun could be argued to be a victim of its own success. It not only went much further than any other large corporate in terms of creating open specifications, it also contributed substantially to sufficient knowledge dissemination and tooling to allow many other individual, SME and large corporates to be able to compete with Sun in these very areas that Sun could’ve monetised. Sun thus created its own competition for monetising the scarcity that got created around Java even as many others cried out hoarsely that Sun simply wasn’t open enough. Will there be an incentive for Oracle to reverse that somewhat ? I suspect that could be a logical option if it focuses on ensuring a good ROI for Sun’s Java investments and assets and more so to be able to make itself more capable in the space around Java than the competition (ie. IBM).

So is a large corporate making money off open source or open standards an oxymoron ? Viewed narrowly yes. Because while it is possible for a small team to make some money with adequate ROI off open source or open standards, it is unlikely to be feasible for large corporates. They really need to figure out what is the new scarcity that gets created off such initiatives, come to a judgment that the opportunity arising from such scarcity is large enough and worthy of their time, attention and investment, ensure that there is no current available capability of anyone else disruptively delivering exponentially asymmetric value in that space, and finally occupy that space before other large corporates do.

It does boil down to the fact that while individuals and small companies may find it easier to monetise open source and open standards, the same is going to be generally tough for large corporates. Or as Michel Bauwens said in the blog post I referred to (in my opinion perhaps a little exaggerated but not entirely off the mark)

For all of this, we will need new policies, major reforms and restructurations in our economy and society.

But one thing is sure: we will have markets, but the core logic of the emerging experience economy, operating as it does in the world of non-rival exchange, is unlikely to have capitalism as its core logic.

A brush with Functional Programming and Scala

Posted by – April 13, 2009

Initial Struggles

A few months ago I was working with using Python for complex data processing scenarios. This was stuff I was very well versed in Java, but was working hard at to make sure that when I wrote code in Python, it would at least pass as half decent pythonic code. My initial struggles with idiomatic Python, soon became a strong interest in functional programming. I initially considered it to probably have a niche applicability in traditional business transaction processing space and of primary importance in algorithmic code, and thats the state where I almost left it. Curiously even after reaching that conclusion I did persist in attempting to figure out how it could be used in a manner where it would be helpful to have functional programming constructs in typical transaction processing scenarios.

Is functional programming useful in traditional business transaction processing ?

One of the aspects of the struggle was that I didn’t want to have the functional programming code merely replace the existing code. I wanted to figure out if it would be possible to replace it in a manner where it would make a substantial positive difference, and that hunt proved a little elusive for a while. Another aspect was the fact that the traditional OO code depended so intensively on state, that to actually try to figure out how the same would work under Functional Programming was really requiring a substantial leap in terms of thought, a leap I had to really struggle with. Finally my language of choice was Python, a dynamically typed language, and some of the literature surrounding functional programming really assumed static typing, especially that related to patterns and constructs in Haskell (eg. Monads). Such patterns were difficult to apply since the underlying expected static typing support was simply at odds with a language like Python.

It was after persisting for a few weeks, that I started to see how one could apply these techniques effectively. I also started figuring out how to Pythonise constructs such as Monads. Once I was past the initial rather steep hurdles, the going became a lot easier, and the progress much much faster. In at least a subset of some typical financial transaction processing scenarios I did get some extremely encouraging (to myself I call them jaw dropping) results. The resultant code suddenly seemed smaller, far far more intuitive to understand and extremely easy to change.

Working with Object Oriented and Functional Programming simultaneously

I am not sure whether it was due to my abilities and experience or due to my inabilities to visualise, I decided that pure functional programming was simply not a choice in this context and thus one would need to combine traditional OO and FP constructs. Having reached that decision, it became a little easier to identify the areas which could be better leveraged by FP. However I believe I still have some ways to go before being able to be confident of writing OO and FP together the way each of them should be.

Scala

My focus on functional programming and fondness to Java also led me to investigate Scala. I made the mistake of actually reading the reference documentation on its web site first. My initial reactions were that it simply was a weird syntax language which was unlikely to be of help in reducing development time even when writing FP code. Thankfully, I did not give up at that stage and continued to read a lot more. I soon realised that type inferencing and other aspects (eg. no getter setters) really resulted in some substantial savings compared to traditional Java code (it probably cuts down the code size to half). And the syntax really wasn’t so cryptic after all, it just required a day to get used to. Moreover one had the entire arsenal of the the FP constructs along with traditional OO constructs to deploy. I did write some processing code and was very happy at the results I got. In this current state, I am excited about learning more about Scala as a language that would be an alternative to Java. It gives me some of the capabilities of more productive languages such as Python, while continuing to leverage the performance and stability of the JVM. Note that Scala’s compatibility with the JVM should be thought of differently from that of dynamic language implementations such as Jython on the JVM. Given Scala’s static typing capabilities, its runtime structures are far more similar to Java and thus are able to retain somewhat similar performance and memory footprint characteristics as traditional Java programs. I suspect the same is unlikely to be found for other dynamic languages on the JVM.

Concluding Thoughts

Prior to this entire exercise, in my mind there were two primary individual preferences to programming languages. For most scenarios, my preferred choice was Python especially due to the sheer speed with which one could write and maintain code, code that was really compact and readable. And if Python runtime performance was not going to be good enough, use Java. Coming out of this exercise, it does seem that there is a high potential that I might want to switch my preference to Scala instead of Java. I have already validated the positive benefits of functional programming and Scala coding in processing scenarios (those that do a lot of computation and processing in the background). I do need to validate the same for typical web based applications, but on the face of things, I believe I have already identified areas where FP can help in typical web application scenarios as well. And now I have two multi paradigm (OO and FP) languages to be able to leverage FP in alongside OO – Python and Scala.

This is something thats definitely going to be my interest area for the next few months and I intend to write many more specific posts on the topic. Should that be of any interest keep listening on this blog’s RSS feed and my twitter stream.

What is statelessness in REST ?

Posted by – April 7, 2009

In my post Fomenting unREST : Is RESTfulness a semantics game ? Why does REST require statelessness ?, I had commented on some of my thoughts on dealing with some of the constraints prescribed by REST one of them being statelessness. Some of those thoughts continued to churn in my mind, and I had a few helpful interactions along the way which led me to what I believe to be the “Aha moment” on statelessness in REST. That doesn’t mean I’m necessarily right, feel free to comment on my thoughts in case you believe any differently or have a nuanced opinion.

Background

Per section 3.4.3 of Roy Fielding’s dissertation

The client-stateless-server style derives from client-server with the additional constraint that no session state is allowed on the server component. Each request from client to server must contain all of the information necessary to understand the request, and cannot take advantage of any stored context on the server. Session state is kept entirely on the client.

The same thought is further commented upon in section 5.1.3.

We next add a constraint to the client-server interaction: communication must be stateless in nature, as in the client-stateless-server (CSS) style of Section 3.4.3 , such that each request from client to server must contain all of the information necessary to understand the request, and cannot take advantage of any stored context on the server. Session state is therefore kept entirely on the client.

Like most architectural choices, the stateless constraint reflects a design trade-off. The disadvantage is that it may decrease network performance by increasing the repetitive data (per-interaction overhead) sent in a series of requests, since that data cannot be left on the server in a shared context. In addition, placing the application state on the client-side reduces the server’s control over consistent application behavior, since the application becomes dependent on the correct implementation of semantics across multiple client versions.

The potential confusion areas

Given the clear prohibition of storing client state on server, there are some typical idioms which do get challenged as REST unfriendly. eg.

  • Logging in to obtain a session token, which is subsequently a validation of an authentication status
  • Binding additional attributes to such a session token on the server side with variables such as :
    • Computed and cached access privileges of the user
    • Conversational state eg. fields entered on page 1 of a two page form

The question there is are all he above scenarios inconsistent with REST ? Many articles seem to suggest that that would indeed be the case. as an example one of them is Ajax and REST, Part 1 Section: Violating the “stateless server” constraint

My thoughts

Clearly in the above example computed data such as access privileges, or for that matter user’s time zone information are available to the server even if these are not in the session so long as the server has the user id. So long as a user id is being supplied (or a proxy such as the session token id), such data being stored in the session is simply a server optimisation and thus does not violate REST guidelines since this is application state and not conversational state.

To the extent the data is storing conversational state such as the fact that the user has authenticated himself and the fields that the user may have entered in page 1 of a two page form, such is incompatible with REST guidelines, and if such practices are adopted the resultant design may not be termed fully REST compliant.

So the existence of a user token does not make the design REST “uncompliant”. Storing conversational state in a user session usually associated with such a token does.

One way to ask if a data is conversational or application, may be to ask oneself, that can the request be satisfied properly if it was accidentally routed to a different server in a situation where the cluster was configured for session affinity. In a REST compliant design, the other server will have a capability to still be able to service the request. However this requires the presumption that the cluster is configured for session affinity and session state is not available to other servers. Should one configure the cluster with shared and distributed sessions, the cluster will be able to service the requests even when the APIs were not REST compliant.

The one thing that does leave me a little uneasy is that to be fully REST compliant one will need to always supply a userid / password with each API call instead of a server token. That does have implications on security, implications that I am not entirely comfortable with.

Update :Surya in a comment in earlier post refers to the fact that authentication need not be done through the URI but through the headers or through protocols such as OAuth. So what that leaves me with is the assessment that existence of a token or a session object is not a violation of REST but storage of conversational state in the same is.

JVM CLR unification : IBM Sun merger held up to complete CLR licensing

Posted by – April 1, 2009

While the Sun acquisition by IBM has been doing rounds for so many days, it has been rather surprising why the deal even though being in news for so long has neither been denied nor completed. It apparently turns out the real value in the deal is based on a simultaneous perpetual licensing of the CLR. The goal is to then build a single unified runtime which will allow for simultaneous support for and complete interoperability between all the languages based on both the JVM and CLR environments. Thus Java / Scala / C# / VB.NET will be able to freely talk to each other. An interesting problem is going to be the fate of the languages which are simultaneously being implemented on both the platforms eg. Jython and IronPython, IronRuby and JRuby etc. These issues will be sorted out in due course. The spade work to talk to some of the developers and bloggers to get a good promotion as soon as the announcement happens has already begun. Some of the bloggers and publishers are worried that the unification of the two is likely to substantially reduce the eyeballs and interest in many of the technical writings since much of the ’spice’ would now be lost. I would imagine a huge PR campaign the moment the final issues in the perpetual CLR licensing issues are ironed out. Apparently some of the prepared press releases were published a bit too soon. These were subsequently not removed but have been put under temporary access control and result in a 401 error when accessed thus leading one to believe that their release is likely to be imminent.

Update : Further breaking news on the topic can be found here.