functional programming / programming / software — No comments
30
Aug 10

Monads in an Object Oriented context

The other day I was referred to a blog post jQuery is a Monad. That is an interesting post which if you have any interest in Monads should read. My first thought was that jQuery was another implementation of Fluent Interface and it just did not strike me as a monadic construct.

There are blog posts which take an essential thought or a message and by putting together various arguments bring it to some logical conclusion. This post is not one of them. It just describes the results of my thought process which started off contesting the assertion, to accepting it and then realising that either way, it really didn’t matter a whole lot. I don’t claim it to be sufficiently accurate except to state that it is accurate only to the best of my belief and understanding. Pointers to any inaccuracies or alternate interpretations in the comments shall be gratefully received. And yes, this post in the end reaches a somewhat boring conclusiion.

Having got that little expectation management out of the way, lets get back to monads, jQuery and Object Orientation. Writing a tutorial on Monads is a rite of passage for many. Thats something I’ve never done. And I don’t wish to, but I cannot escape the fact that I’m going to have to introduce monads to those who are as challenged in their understanding as I am.

What are Monads ?

I’ll give you the loose idea. Whenever a function completes any computation, the results of the computations are often returned as the return value of the function. But every once in a while you need a container around that return value. A container to store the state of a computation, sometimes with additional metadata or historical annotations for use later. At its very basis, a monad is a container for storing computations and being the container to be carried between functions. Think of it as a shipping container if you will, which moves materials between factories – the factories being various functions which operate on the contents of that container.

But it takes more for a monad to be a monad than just being a container. Given any content, the monad should be able to construct itself by wrapping itself around the content. Thats what the source factory invokes when it fills up the container with goods. Some prefer to call it a type constructor – the type here being the container, or the monad (which may internally contain goods of various types, but I get ahead of myself).

In addition once the container reaches the destination factory, unlike typical container, this one does not just allow its contents to be unloaded. Nay, that would make it too simple. In this case, the container has to be taken to a factory where it allows a robotic arm of the factory to plug into a receptacle it provides, which the factory can use to extract the underlying values, often one at a time. The factory further processes these goods, which again come out at the other end of the factory as a different set of containers. And it is quite likely that the type of the goods in the new containers could’ve changed.

That in essence is a monad. Easy, right ? I’m sure not and I’m now only going to make it a bit harder.

The wikipedia, definition of monad Monad

I’m just going to let you read the definition as provided by wikipedia.

A monad is a construction that, given an underlying type system, embeds a corresponding type system (called the monadic type system) into it (that is, each monadic type acts as the underlying type). This monadic type system preserves all significant aspects of the underlying type system, while adding features particular to the monad.
The usual formulation of a monad for programming is known as a Kleisli triple, and has the following components:

  1. A type construction that defines, for every underlying type, how to obtain a corresponding monadic type. In Haskell’s notation, the name of the monad represents the type constructor. If M is the name of the monad and t is a data type, then “M t” is the corresponding type in the monad.
  2. A unit function that maps a value in an underlying type to a value in the corresponding monadic type. The result is the “simplest” value in the corresponding type that completely preserves the original value (simplicity being understood appropriately to the monad). In Haskell, this function is called return due to the way it is used in the do-notation described later. The unit function has the polymorphic type t→M t.
  3. A binding operation of polymorphic type (M t)→(t→M u)→(M u), which Haskell represents by the infix operator >>=. Its first argument is a value in a monadic type, its second argument is a function that maps from the underlying type of the first argument to another monadic type, and its result is in that other monadic type. The binding operation can be understood as having four stages:
    1. The monad-related structure on the first argument is “pierced” to expose any number of values in the underlying type t.
    2. The given function is applied to all of those values to obtain values of type (M u).
    3. The monad-related structure on those values is also pierced, exposing values of type u.
    4. Finally, the monad-related structure is reassembled over all of the results, giving a single value of type (M u).

In object-oriented programming terms, the type construction would correspond to the declaration of the monadic type, the unit function takes the role of a constructor method, and the binding operation contains the logic necessary to execute its registered callbacks (the monadic functions).

In practical terms, a monad (seen as special result values carried throughout the pipeline) stores function results and side-effect representations. This allows side effects to be propagated through the return values of functions without breaking the pure functional model.

The bind operation

So coming back to the container analogy, the container is the monad, the goods are the underlying types, and the factory is a function which takes a underlying type and spews out another monad which contains the newly manufactured goods. There is a difference to be noted, the container doesn’t go into the factory, its the factory thats plugged into the monad through the bind operation, which triggers the processing. Also note that the output of each such factory is not just goods, but containers of such goods it manufactures.

There, you can now imagine a series of containers and factories, or monads and functions which are sequenced to assemble a pipeline to produce the desired goods / computations.

Before we go on to how this works in OO, I would like to spend some more time on the function thats provided to the binding operation. This function accepts the individual contents of the incoming container and returns a container with the newly manufactured goods. A somewhat similar function is used when using map operations on containers eg. Lists. So if I had a list of integers, and I wrote a function which doubled their values, then the map operation would be written in python as :

1
2
3
4
5
def twice(val): return str(2 * val)

print map(twice,(1,2,3,4,5))

#Expected output is :['2', '4', '6', '8', '10']

This is a good example where the container (tuple) is unbundled inside the map operation, each constituent value of the list is passed to the twice function, and all the results of the twice function are reassembled into yet another container (list). The type of the constituents also changed from ints to str (string) along the way. To be a strictly monadic construct, the twice function should’ve returned not just the double of the value, but a double contained in some other container (either a tuple or a list), and the map function should’ve extract the individual strings from each of the individual tuples/lists and constructed a final tuple / list out of the same.

While unlikely to be confused as so, let me restate for clarity, the map function is not the monad. The tuple and the lists are the monads here. The map function is an example of the contextual capabilities that monads expect from their environment (eg. via and around the bind operator in haskell). And twice is the operation thats performed on the monad. Again to restate, the monadic chain is like a set of factories that spew out containers, which allow other factories to plug into them and in turn extract the contents and then spew out even more containers.

Monads and Object Orientation

So how does this work in an object oriented context ?

Lets take rules 1 and 2 of a monad. If I was to declare a class which had a constructor which took in a value and then wrapped it, then I would satisfy these two rules for being then be able to suggest that the class is a monad. However the rule 3 gets a little bit more interesting. Object oriented languages have the “.” operator which is somewhat analogous to the do block which can chain operations. So if I was to write o.foo() that would be equivalent of suggesting that I invoke the method foo() on the object which being a member method of the class has access to all the object internals and thus is able to access the wrapped value and do the necessary computations. Now if foo() were to return any object again of a class which satisfies these very rules, then I would be able to say that this class along with its member function foo() is a reasonable object oriented analogy of the monad. And I would be able to start chaining the methods as in o.foo().bar().baz()

Whoa! let me restate that again. If a class, has a constructor which takes a value and wraps that value, and has a number of additional member functions which each operate on these underlying values and return an instance of either the same object or another object of a class which follows the same rules, then I can say that that particular class is a monad. Incidentally thats exactly what fluent interfaces do, except that they do not have any specific expectation of wrapping. And a large number of classes may incidentally fit this description. And if jQuery is a monad, so are all of them.

Well, we did relax a few constraints along the way. First of all the functions are not stand alone functions. They are member methods which have direct access to the underlying wrapped value. Secondly they themselves don’t return a monad around an individual item in a collection. They return a monad around the entire collection of values. Thus the complexities and capabilities of the do block and the bind function are substantially simplified when using the “.” operator. And finally going back to the container analogy, the class defines and consists of both the containers and the factories. Seems like the threshold for stating a particular class is a monad is actually quite low. Turns out we reached a boring end. And it seems we are not much wiser at least in terms of any specific conclusions or insights. But every once in a while sometime the journey is more exciting than its end. For me this did seem like one.

Why are objects simple and monads complex ?

At least for me the above was true. Turns out objects collate related data and functions together into one class. Also the do block and bind operator on an object is very simple. So for many especially coming from the OO school, objects are well understood. On the other hand understanding the requirements of monadic constructs takes quite some time. So there’s a lot of gray cells that need to be exercised to start understanding what a monad is and how the various requirements for a monadic construct can be satisfied. And when mapping between monads, and objects which can also be seen to be monads, a lot of that complexity is either partially waived (eg. functions returning monads, the infrastructure around bind operators unbundling and bundling monads) or just simplified (functions and underlying values being colocated in the same class, the “.” operator being a far more simplified version of do block). But after some substantial headaches, it does start to seem that perhaps, just perhaps, monads aren’t so complicated after all :) .

So is jQuery a monad ? I believe one could choose to be very pedantic and point out minor issues with that assertion. Or one could accept that the intent of monadic sequences are well represented in jQuery chains and accept it as one. I started with the former and ended at the latter position. Would I express that jQuery is a monad ? To the monadically challenged – No. That obfuscates far more than it enlightens. To them I would say jQuery is a fluent interface which allows continuous chaining of operations on an underlying set of dom objects. And is there a big “Aha moment” when one realises that jQuery is a monad ? I couldn’t find one. While the journey of trying to understand monads and correlate it with monads was very exciting, at least the OO practitioner in me is unlikely to have missed much had I not known that. But I got to understand monads better – and thats well worth the time, and all the headspinning.

architecture / software — 2 comments
26
Aug 10

A case for non leaky dual abstractions.

A long long time ago, I worked on a fairly complex piece of design. And like any well behaved designer, I broke it down into a number of abstractions that made it manageable. I gave the abstractions funny sounding names. And before long I found those abstractions finding their way into the user interface. Abstractions which made no sense to the end user.

That experience taught me a good lesson. In attempting to deal with complexity, I was attempting to come up with the appropriate abstractions in a very bottom up way. So, strategic closure (of the Open Closed Principle), dependency inversion principle, et. al. all found their way into these abstractions I was modeling. These abstractions represented themselves via the various programmatic interfaces in the software design and their roles. At the same time there was a different perspective of the software. The way the end users would’ve preferred to see that software. The abstractions the way the end users saw the system were sometimes different than the underlying abstractions I modeled.

In a good design, the two abstractions should line up, and if there is an inconsistency, there is probably an issue with the design. Certainly this could be an issue in some cases. However in many cases the underlying backend might be built to offer far more capabilities than what are being exposed through the early version of the user interfaces. Probably there are multiple intents for which the software is being designed, and the particular user interface on table is just one of them. Probably, the underlying complexity of the back end is way too high which requires a very different nature of bottom up abstractions, which are different from the top down ones. Frankly, the reason doesn’t matter. Even after so many years, I look back and am comfortable with the thought that there needed to be two abstractions – one top down and one bottom up, one front end and one backend.

Fast forward many years. Another example helped me further attain some insight into the matter. We had a bunch of mobile smartphones floating around with the traditional PC UI abstractions being carried over into their design. Along came an iPhone – which rethought the interface the way users would’ve preferred to see the interface. Now iPhone was built on a traditional operating system. So it had the same abstractions that these operating systems have in the backend. However it decided to change some of the front end abstractions – in tune with the target market and the device / form factor peculiarities. Did the iPhone change the backend abstractions baked into its software – most likely not. However it changed the frontend abstractions, just enough to make the experience really simple and easy for its users.

As an engineer, I have lived with the regret of allowing backend abstractions to leak into the front end. But I have learnt something along the way.

  • Don’t let the inapplicable frontend abstractions leak into the backend. This is especially true for most reasonably complex software. Strictly top down design can lead to a lot of brittleness in the long run, requiring very substantial surgery eventually.
  • Don’t let the inapplicable backend abstractions leak into the frontend. In most cases this is a usability nightmare. ’nuff said.
  • Realise that you have to simultaneously service independent expectations ie. robustness of usability and robustness of modeling. Work with both the abstractions to mould each of them well: Work your frontend abstractions well enough to mould them to reflect the use cases of the software in the most usable manner. Work with the backend abstractions to let them emerge from the problem space you are trying to resolve at the backend. Now make sure you work with both of them to map them into each in a reasonably smooth fashion. This is easier said than done. But doing it is whats necessary.

PS: I am not referring to what is conventionally referred to as leaky abstractions, where the implementation leaks into an abstraction. I am referring to a set of frontend and backend abstractions leaking into each other. However if all the backend abstractions were to be treated as an implementation, then this would be a case of leaky abstractions as well.

java / php / programming / python / ruby / scala / software — 8 comments
13
Aug 10

Programming Languages should be Simple (or My ideal programming language)

I am disappointed with many of the newer languages which I earlier thought showed great promise of making programming easier, quicker, and more robust. And it boils down to one thing. Simplicity in learning. Having gone through substantial amounts of programming in C, C++, Java and Python, my quest for the “next” programming language remains unfulfilled.

Why ?

Programming should be simple. And it should be accessible. And when I mean accessible I mean people with IQ of approximately 100 should be able to write programs. I am disappointed that many of the trends seem to raise either the minimum IQ or the training time required to gain competency. And while that helps a community of the super brilliant, it does not make a substantial difference to programming in general. It remains esoteric and does not stoop to touch everybody.

What ?

So what are the features of my preferred programming language :

  • JRE support: Should run on the JRE with Java interop. Thats the dominant well engineered platform that runs across all classes of desktops, servers and devices. Additional support for CLR is a bonus but not mandatory.
  • Simplicity: Should be fairly simple to read, learn and understand. Python is a good example. PHP is a great example (at a simpler class of problems). C++ and Scala not good examples.
  • Multi paradigm : Should support both OO and FP constructs. eg. Scala and Python. Half hearted support to functional programming as with python discouraged. Ditto with passionate support for objects with second class treatment for functions as in Java/Ruby.
  • Multi core compatible : Should have good constructs for leveraging multiple cores eg. erlang, scala, clojure and many others.
  • Type inferencing : (and I am a python programmer :) ). Good type inferencing coupled with on the fly non intrusive/disruptive compilations as with say the eclipse on the fly compiler or play framework. Three cheers for Scala. One reason I prefer type inferencing to dynamic typing is the much superior performance even while maintaining brevity and removing boilerplate.
  • Constructs that are natural to humans not mathematics : This is actually a sub point to Simplicity. The constructs should be consistent with the normal average non mathematically trained brains. 2 + 3 is much simpler to understand than (+ 2 3). Python rocks. Lisp / Clojure or for that matter brainfuck dont.
  • Closures and code blocks : Love ruby for this.

Is this a pipe dream ?

For the moment seems so. Do you know of a language which helps meet these requirements ?

And to be very clear (because there is a substantial risk of the same) – this is no flame bait or an opportunity to trigger language wars. It is meant to highlight two things

  1. There is no ideal language out there, and
  2. When designing languages – make them simple to learn and use. ie. for a given problem statement a good language is one which requires the minimum talent or training to solve the problem
architecture / java / programming / python / rest / software / web — 1 comment
13
Jul 10

Presentation : Recent trends in technology

This was a presentation I recently conducted to an audience of programmers / architects primarily in the financial services domain.

Trends in technology
management / software — 7 comments
19
Mar 10

Double whammy. The state and dilemma of Indian IT

Every now and then I come across a blog post which talks about outsourcing and soon enough a commentary or discussion on Indian IT and a whole host of associated parameters comes up. Soon enough some of them start attracting a number of views and comments. And more often than not the comment stream starts attracting far more extreme views which do sometimes leave me recoiled.

No, I am not going to list them, link to them or comment on their content specifically. After some thought I decided that simply would shift the focus unnecessarily. Instead I thought given my reasonably long experience both in and outside India, it would be helpful offer my perspective on Indian IT within the context of global IT and Indian economy.

Indian IT and the role of exports

Indian IT (along with ITES and Gems & Jewellery) is a one of the stranger segments of the Indian economy. Unlike the rest of Indian economy, it is heavily export focused. The various numbers reflecting the state of Indian IT agree on at least one factor – at least 2/3rd of its revenues are based on exports.

Let us first look at some of the perceptions about Indian IT that I would like to comment upon.

You get what you pay for : In general thats a very reasonable statement. But it breaks down when you realise one Indian engineer + 1 H1B visa + 1 long flight results in tripling (at least) of remuneration without either the H1B visa or the long flight adding anything to the capability or the quality of the output of the engineer (well it could add a chip on the shoulder). Part of the difference is due to the difference in nominal exchange rates (which are driven by demand and supply of goods to/from the respective countries) and the exchange rates based on purchasing power parity. Based on purchasing power parity Indian salaries in the IT space do not underperform the rest of the world substantially. In fact when compared on purchasing power parity Indian Economy GDP more than triples and it ranks as the fourth largest economy in the world next only to the US, China and Japan.

Indian engineers are offering their services too cheaply. : Quite to the contrary exactly the opposite is true. The Indian IT engineer is too handsomely remunerated compared to the non IT engineers in India. I believe the high salaries in Indian IT is a problem. Many Indian innovations are today happening in areas outside of IT – primarily in the areas of making products and services affordable to the millions surviving on the fringe of the poverty line. And if recent trends in automobiles and telecom sectors are indicative, India is actually proving, that it is the rest of the world which is way too expensive, by offering products and services at price points which are unfathomable elsewhere. My reading of the empirical evidence leads me to believe that Indian IT could underperform other sectors of the Indian economy in terms of both innovation and quality. And a lot of it is due to the fact that the export revenues offer cushy jobs without the really hard work it takes to compete within the Indian economy. Quite frankly this one stereotype must be deposited very quickly where it belongs. In the Trash Can. So let me repeat – Indian engineers are and Indian software is too expensive and containment in its cost growth is most urgently required. While containing salary growth might be useful, investing in ability to create high quality software upfront and eliminating the defect fixing cycles post the initial release will help bring the cost down.

Indian IT offers poor quality software (The alternative version is outsourcing leads to drop in quality) : This is too hard a statement to comment upon, given its utter and gross generalisation. I am not aware of specific quality benchmarks which could be used to assert or deny such claims, though there is a fair amount of empirical evidence which could be used to either assert or deny these claims. India like any other country has a range of software quality in different companies and products which span the entire spectrum from superb to utter crap. I tend to agree with the statement that in many cases Indian software output does leave a lot to be desired in terms of quality. I also believe that there is a curious dynamic at play here. It is well known that in software one can demand and expect any two of three parameters to be fulfilled – viz. Cost, Time and Quality. Any efforts to increase quality can in some contexts run into the business issue of the customer clearly prioritising cost and time to market. And remember two thirds of the revenues (and presumably the customers) of Indian IT are from overseas. I have often struggled hard on the quality aspects, and have generally found that it requires a very strong support not just from the engineers but from the business stakeholders to offer sustained high quality.

Indian engineers are not as skilled / Indian engineers are far superior to the rest of the world : This is an interesting stereotype which is obviously wrong at either of the two extremes at which it is observed. Some believe the Indian engineers are extremely competent and productive but fail to realise that these engineers are from the extreme top end of a very competitive educational system. Bring down the comparison to some reasonable way of comparing an average Indian and Non Indian IT programmer and perhaps the comparison may not be so rosy for India. I think India does need to work much harder to strengthen its capabilities of programmers at the middle and lower ends. And sometimes I blame the fact that too many projects being transferred to India has resulted into so much demand for programmers that a programmer with a 10 percentile performance can still make a wonderful living by just changing his jobs every two years. This needs to change. But I am afraid as long as India continues to bill the rest of the world by the hour and not by capability and quality, this shall continue to be an uphill battle even as we shall continue to see islands of excellence.

Double whammy

The double whammy I refer to in the title of this post stems from the fact that due to heavy reliance on exports, Indian IT has been substantially based on the priorities dictated by her customers. Thus Indian IT is largely today what its customers asked it to be. And most of the customers are non Indian. The double whammy is the fact that Indian IT gets criticised for becoming exactly what its customers asked it to be – fast, low cost and high quality so long as the high quality doesn’t interfere with the fast or the low cost :) . To be fair I am aware of the same criteria getting applied within many non Indian IT companies as well – so its not an Indian innovation. To be fair I am also aware of some situations where the lack of focus on quality is not driven by customer prioritisation. Is there a way out of the double whammy. Frankly I can’t see one easily. Yet this post would perhaps go some way in enriching the reader’s perspective on some of these factors.

The role of the population

There is one aspect about India which separates her from many other IT exporting countries. Her population. Not is it only really large (the second largest in the world), but it is growing fast and expected to continuously grow younger over the next two decades (dream demographics for an economist). Which means India stands unique in her ability to deploy masses. Which also means the problems which are easier to solve by deploying masses are more likely to find their way to the Indian shores. Combined with a high growth rate it also contributes to most Indian programmers getting promoted to management ranks (whatever that means) fairly soon. As we shall see in the next section this is an important factor in the Indian context.

Improving capability and quality

Independent of the validity of stereotypes which I questioned above I think it is a dead certainty that Indian IT can focus on improving both its capability and quality. It is not infrequent for me to feel demotivated after meeting some college students or fresh graduates and realising their priorities are really driven around what is the quickest way to make maximum money. This leads to unhealthy focus on building skills with high resume value. (Yeah that may sound funny to some of you, but its not uncommon). And given the high growth as I referred to in the earlier section – the role model for IT is – start as a programmer, change jobs every 2-3 years, become a team lead in 3 years, a manager in 6. And if you are particularly technically inclined you can become an architect guiding many projects and helping support many pre-sales efforts. What I haven’t seen Indian IT getting criticised for frequently enough, is the fact that so few of her members contribute to open source. While dzone and reddit may attract a large number of readers from India, a disproportionately small proportion of the people writing the posts stay in India. And most good programmers have moved on to becoming a Tech Lead or a Project Manager driving the average sustained technology experience lower and lower. So much for the “IT superpower” marketing that the hype manufacturing machinery creates internally. In a recent meeting with around 20 plus people in the room, I was one of the only 2-3 persons who believed India is not an IT superpower.

Change

I would like this to change. I would like this to change substantially. But I do not see the economic incentives in place for that to happen. Yet. However there is this strong feeling that something will indeed happen to make things change for the better. This is clearly something Indian IT will need to grapple with in the months and years to come. However there are at least a few factors which will lend themselves positively towards strengthening Indian IT.

Agile. Movement to agile requires a continuous focus on quality that cannot be wished away as easily. Projects that genuinely adopt agile methodologies will be implicitly driven toward being able to offer higher and sustained high quality.

Saas. As more software development shifts from intra enterprise development (where it is a little easier to contain impacts of poor quality) into Saas, there will be implicit pressures from the customers on the quality front. Another positive influence of Saas is that given the higher sharing of software across a much larger user base, the demand for number of developers would come down. Even as it may influence the demand for Indian IT itself negatively, I think it would help drive Indian IT into focusing more on capability than deploying masses.

Internal Growth. While much of the historic growth of Indian IT came from outsourcing, I anticipate the Indian industry to start driving its growth even more soon (given the really high growth engine it finds itself in). In such scenarios, the portfolio of software assignments will include a higher number of strategic and critical projects at extremely challenging cost parameters. This portfolio readjustment will help influence the quality positively.

I’ve attempted to highlight that Indian IT as exists today is (partially) a function of her customer’s expectations. And while there are some unfair stereotypes about it, there are clearly some things it can clearly improve upon. It is with some trepidation I write this since attempting to deal with perceptions in a generalised / stereotyped scenarios can be quite risky. So allow me to end with the disclaimer. This is a documentation of my understanding – YMMV.

functional programming / programming / python / software — Comments Off
01
Mar 10

Functional Programming with Python – Part 2 – Useful python constructs

In Functional Programming with Python – Part 1, I focused on providing an overview. In this post I shall focus on the core python language constructs supporting functional programming. If you are experienced pythonista, you may choose to skip this post (and wait for the next post in this series :) )

Sequences in python are not immutable :

When using sequences in python the thing to be noted is that sequences are not immutable. This provides you with the following options.

a) Use immutable sequence types : This is only possible by defining different types for sequences than the ones built into the language.
b) Ignore all methods on the sequences which modify them
c) Waive functional programming tenet of immutability

My preferred option when explicitly focusing on writing functional code is to use b)

Sequence types in python

The most commonly used sequence types in python are :

  • Tuple : Immutable, Ordered, Indexed collection represented as (val1, val2)
  • List : Ordered collection represented as [val1, val2]
  • Dictionary : A key indexed collection represented as { key1: val1, key2 : val2 }
  • Set : An unordered collection allowing no duplicates. Represented as set([val1,val2])
  • str and unicode : While primarily meant to serv as ascii and unicode strings, these data structures also act as sequences of characters.Represented as “abcd” or ‘abcd’

Of the above only tuples are immutable.

There are other sequences used less often as well. An example is frozenset which is an immutable set.

Simple iteration
The for construct allows you to loop through a sequence. eg.

1
2
3
one_to_five = [1,2,3,4,5]
for num in one_to_five :
    print num

Iterators on dictionaries

Unlike tuples, lists and sets where iterators essentially traverse through the sequence constituents, there are a number of different iterators on dictionaries. These are :

1
2
3
4
5
6
7
8
9
d = {1:"One", 2: "Two", 3:"Three"}
# keys
for val in d : print val
# an alternative for keys
for val in d.keys() : print val
# values
for val in d.values() : print val
# key, value tuples
for key,val in d.items() : print key, val

Slices

Slices allow creation of a subset on a sequence. They take the syntax [start:stop:step] with the caveat that a negative value for start or stop indicates an index from the end of the sequence measured backwards, whereas a negative step indicates a step in the reverse direction. The following should quickly indicate the use of slices

1
2
3
4
5
6
7
8
9
10
seq=[0,1,2,3,4,5,6,7]

#first 3
print seq[:3]
# last 3
print seq[-3:]
# 2nd to second last
print seq[1:-1]
# reverse the sequence
print seq[::-1]

The iterator protocol
Since python is an object oriented language as well, it provides strong support for allowing iteration over an object internals. Any object in python can behave like a sequence by providing the following :

a. It must implement the __iter__() method which in turn supports the iterator protocol
b. The iterator protocol requires the returned object, an iterator, to support the following ie. an __iter__() method returning self. and a next() method which returns the next element in the sequence, or raise a StopIteration in case the end of iteration is reached.

I’ve tested iterators which do not themselves have an __iter__() method and it still works, but I still do not give in to the temptation of not defining the next() method alone since that would be inconsistent with the documented python specifications

Let us examine a simple class indicating a range. Note that this is just for demonstration since python itself has better constructs for a range.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# An iterator object to sequentially offer the next number
# in a sequence. Raises StopIteration on reaching the end
class RangeIterator(object):
    def __init__(self,begin,end):
        self._next = begin - 1
        self.end = end
    def next(self):
        self._next += 1
        if self._next < self.end :
            return self._next
        else :
            raise StopIteration
           
# A class which allows sequential iteration over a range of
# values (begin inclusive, end exclusive)
class Range(object):
    def __init__(self,begin,end):
        self.begin = begin
        self.end = end
    def __iter__(self):
        return RangeIterator(self.begin, self.end)

oneToFive = Range(1,5)

In the above example, __iter__() on Range returns an instance of a RangeIterator.

The way this capability is used is as follows :

1
2
3
4
5
for number in oneToFive :
    print 'Next number is : %s' % number

print 'Is 2 in oneToFive', 2 in oneToFive
print 'Is 9 in oneToFive', 9 in oneToFive

The output you get with it is :

1
2
3
4
5
6
Next number is : 1
Next number is : 2
Next number is : 3
Next number is : 4
Is 2 in oneToFive True
Is 9 in oneToFive False

Generators

In the above situation, the state necessary for the iteration (ie.begin and end attributes) need to be stored. This was stored as a class member. Python also provides a function based construct called a generator. In case of a generator, the function should just do a yield instead of a return. Python implicitly provides a next() method which resumes where the last yield left off and implicitly raises a StopIteration when the function completes. So representing the above class as a function,

1
2
3
4
5
6
7
8
9
10
11
def my_range(begin, end):
    current = begin
    while current < end :
        yield current
        current += 1
       
for number in my_range(1,5) :
    print 'Next number is : %s' % number

print 'Is 2 in oneToFive', 2 in my_range(1,5)
print 'Is 9 in oneToFive', 9 in my_range(1,5)

I do not document the results since these are identical to the earlier code using a RangeIterator.

Note: I used my_range() and not range() since there already exists another range() already provided by python

An interesting point to realise is that in both the above situations, the next element to be returned in the sequence was being computed dynamically. To put it in the terminology better consistent with functional programming, it was being lazily evaluated. While python has no mechanism of lazy evaluation of functions, the ability of functions or objects to lazily evaluate the return values are sufficiently adequate to get the benefits of lazy evaluation at least from the perspective of cpu utilisation only on demand, and minimal memory utilisation.

List comprehensions

List comprehensions are one of the most powerful functional programming constructs in python. To quote from python documentation (even as I leave out a very important part of the full quote .. to be covered later),

Each list comprehension consists of an expression followed by a for clause, then zero or more for or if clauses. The result will be a list resulting from evaluating the expression in the context of the for and if clauses which follow it. If the expression would evaluate to a tuple, it must be parenthesized.

A sample list comprehension is one which returns a sequence of even numbers between 0 and 9 :

1
2
3
4
# A list comprehension that returns even numbers between 0 and 9
even_comprehension = (num for num in range(10) if num % 2 == 0)
print type(even_comprehension)
print tuple(even_comprehension)

The output one gets on running the code above is

1
2
type 'generator'
(0, 2, 4, 6, 8)

Note that the list comprehension returns a generator which then returns a sequence containing 0, 2, 4, 6 and 8.

The for and if statements can be deeply nested.

lambda

Lambdas are anonymous functions with a constraint. Their body can only be a single expression which is also the return value of the function. I will demonstrate their usage in the next section.

map, filter and reduce

Three of the functional programming constructs which probably have aroused substantial discussions within the python community are map, filter and reduce.

map takes a sequence, applies a function of each of its value and returns a sequence of the result of the function. Thus if one were to use map with a function which computes the square on a sequence, the result would be a sequence of the squares. Thus

1
2
def square(num): return num * num
print tuple(map(square,range(5)))

results into

1
(0, 1, 4, 9, 16)

I mentioned earlier I will demonstrate usage of a lambda. In this case I could use a lambda by defining the square function anonymously in place as follows :

1
print tuple(map(lambda num : num * num,range(5)))

filter takes a predicate and returns only the elements in the sequence for whom the predicate evaluates to true.

Thus

1
print filter(lambda x : x % 2 == 0, range(5))

results in the following output

1
[0, 2, 4]

Finally the most controversial of them all – reduce. Starting with an initial value, reduce reduces the sequence down to a single value by applying a function on each of the elements in the sequence along with the current manifestation of the reduced value. Thus if I wanted to compute the sum of squares of numbers 0 through 4, I could use the following

1
print reduce(lambda reduced,num : reduced + (num * num), range(5), 0)

In the above example, 0 as the right most argument is the initial value. range(5) is the sequence of numbers from 0 thru 4. Finally the anonymous lambda takes two parameters – the first is always the current value of the reduced value (or initial value in case it is being invoked for the very first time) and the second parameter is the element in the sequence. The return value of the function is the value which will get passed to the subsequent invocation of the reduce function with the next element in the sequence as the new reduced value. The reduced value as returned finally by the function is then the returned value from reduce

The functional programming folks are likely to find this an extremely natural expression. Yet reduce resulted in a substantial debate within the python community. With the result that reduce is now being removed from python 3.0. (Strictly speaking it is being removed from python core but will be just an import statement away as a part of the functools package). See The fate of reduce() in Python 3000. Why so controversial ? Simply because the usage of map, filter, reduce above could be rewritten as

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
#map
seq = []
for num in range(5) :
    seq = seq + [num * num]
print seq

#filter
seq = []
for num in range(5) :
    if num % 2 == 0 :
        seq = seq + [num]
print seq

#reduce
total = 0
for num in range(5) :
    total = total + (num * num)
print total

Just look at the two blocks of code and figure out which is easier to read (especially look at reduce). One of the areas where a large number of pythonistas and lispists are likely to disagree is the tradeoffs between brevity and easy readability. Python code is often english like (well, at least to the extent that a programming language can be) and some pythonistas do not like the terse syntax of lisp thats hard to follow.

I earlier mentioned that I left out a part of the description of list comprehensions from python documentation. Here’s that part of the quote.

List comprehensions provide a concise way to create lists without resorting to use of map(), filter() and/or lambda. The resulting list definition tends often to be clearer than lists built using those constructs.

Having said that I do believe many including myself will continue to use map, filter, reduce

Other helpful functions in python core that are helpful for functional programming are :

  • all : Returns True if all elements in a sequence are true
  • any : Returns True if any element in a sequence is true
  • enumerate : Returns a sequence containing tuples of element index and element
  • max : Returns the maximum value in a sequence
  • min : Returns the minimum value in a sequence
  • range : Return a sequence for given start, end and step values
  • sorted : Returns a sorted form of the sequence. It is possible to specify comparator functions, or key value for sorting, or change direction of sort
  • xrange : Same as range except it generates the next sequence element only on demand (lazy evaluation) thus helping conserve memory or work with infinite sequences.

We’ve seen many of the constructs that are typically useful for functional programming. I left out one big part – the itertools package. This is not a part of python core (its a package which is available with a default python installation). Its a large library and substantially helps functional programming. That along with some more sample python programs shall be the focus of my next blog post in the series. At this point in time I anticipate at least a few more parts after that to focus on a) Immutability b) Concurrency and c) Sample usage.

Hope you found this useful and keep the feedback coming so that I can factor it into the subsequent posts.

functional programming / programming / python / software — 14 comments
23
Feb 10

Functional Programming with Python – Part 1

Lately there has been a substantial increase in interest and activity in Functional Programming. Functional Programming is sufficiently different from the conventional mainstream programming style called Imperative Programming to warrant some discussion on what it is, before we delve into the specifics of how it can be used in Python.

What is Functional Programming?

To quote from Wikipedia,

In computer science, functional programming is a programming paradigm that treats computation as the evaluation of mathematical functions and avoids state and mutable data. It emphasizes the application of functions, in contrast to the imperative programming style, which emphasizes changes in state.

For beginners, one of the most fluent starter pages I would recommend for the history and specifics of functional programming is Functional Programming For The Rest of Us. This is a must read article which provides the reader with a good overview without getting too much into the nitty gritties of functional programming.

Since a detailed discussion on functional programming (henceforth referred to FP) is beyond the scope of this post, I will just briefly summarise the most critical elements of FP.

  • Functions as the basic building blocks : Unsurprisingly FP requires the construction and usage of functions as the basic building units. In the simplest terms “int add(int x, int y) { return x + y; }” is a simple addition function written in ‘C’. It takes two parameters x and y, adds them, and returns the result. This is a rather obvious and simple case but I stated it since I would like to refer back to it subsequently in this post.
  • Functional Programming prefers functions without side effects : The add function above is a good example of a function without side effects. A function is said to be without side effects if the only changes it makes are those that are manifested in the return values. In other words such a function cannot change any global variables, write to the console, update the database etc. A fairly related term is referential transparency. A function is said to be referentially transparent if its invocation can be substituted by the return value in a program without impacting the program in any other way.
  • Immutability : Pure functional programming often requires you to deal with immutable data structures. Thus the value of any variable is not open to modification (Thus they are called values and not variables). This aspect complements the functions without side effects. Thus the way most changes to state are implemented are not by modifying an object in place (which is how imperative programming deals with it) but by cloning the data structure with some of the values getting modified and the modified data structure being returned by the function. Java programmers are aware of the immutability of the String instances wherein any modifications to the string result in a new String instance being created. Imagine the same happening to all the datatypes across the program.

Benefits of Functional Programming :

Some of the nice benefits (I am tempted to say side effects) of functional programming are :

  • Superior ability to deal with concurrency (multi threading) : Threaded programs are nasty to write. And nastier to debug. In an imperative environment, you not only have to deal with data structures being modified in place by some other parts of the program, in a threaded environment such modifications can happen using peer threads, even as your current thread whose logic you are focusing on is attempting to exercise that logic. Its an extremely unpredictable environment which has resulted in a number of how-to’s for safe threaded programming using constructs such as locks, mutexes etc. Functional programming deals with the issue far more elegantly. Instead of controlling and managing unpredictability, it takes it out completely. Because a data structure once constructed will not be modified and because the source of the modifications can be clearly located to the function which instantiated the datastructure, the unpredictability of data changing right under you is gone. This can be a little expensive to manage and FP does sometimes come up with some compromises (or cool features depending on how you view it) such as Software Transactional Memory but a discussion on that is completely beyond the scope of this post.
  • Easier testing and debugging : Because modifications to data are contained and because a function communicates with the context outside it only via its return values, testing and debugging become far easier. You essentially need to focus on testing each function individually. Similarly during debugging you need to be able to quickly locate the function likely to have the problem, after which you can easily focus on the function to be able to quickly resolve the issue. Mocking out functions can also help testing each function in isolation. In general because of fewer side effects, testing under functional programming is often a lot easier, and the importance of having to do “integration” testing and “module” testing is lesser since testing functions in isolation is likely to identify most issues, far more than in typical imperative programming.

Why Python?

Python is not the best functional programming language. But it was not meant to be. Python is a multi paradigm language. Want to write good old ‘C’ style procedural code? Python will do it for you. C++/Java style object oriented code? Python is at your service as well. Functional Programming ? As this series of posts is about to demonstrate – Python can do a decent job at it as well. Python is probably the most productive language I have worked with (across a variety of different types of programming requirements). Add to that the fact that python is a language thats extremely easy to learn, suffers from excellent readability, has fairly good web frameworks such as django, has excellent mathematical and statistical libraries such as numpy, and cool network oriented frameworks such as twisted. Python may not be the right choice if you want to write 100% FP. But if you want to learn more of FP or use FP techniques along with other paradigms Python’s capabilities are screaming to be heard.

Sample Program :

I debated whether I should introduce various elements of functional programming using python in detail and then put it all together in a sample program all in future blog posts of this series, or whether I should start with a sample program which cover various aspects of function programming in this post and then explain various aspects in much more detail in future posts. For better or for worse, I have chosen the latter option. That means I shall be explaining one sample program and shall leave it to future posts in this series to get into greater details.

The sample program I have chosen is that of a simple calculator. A typical calculator supports simple unary or binary mathematical operators and performs floating point operations. Without much ado we now get into the sample program.

Immutable Data :

1
2
3
4
5
from collections import namedtuple

Context = namedtuple('Context','stack, current, op')
def default_context():
    return Context([],0.0,None)

Python is not particularly strong at immutable data. However one of the data structures, a tuple is immutable. A namedtuple is another data structure which supports both tuple like access through indices or through named elements in the tuple. For the calculator I shall need a Context which contains a stack for storing any incomplete operations, an attribute current reflecting the current value being shown on the screen and an op which might reflect a pending operation which is typically required for binary operators where the second value still needs to be provided. While namedtuple is a reasonable construct for simple tuple like objects, it would be helpful to have immutable objects as well – but thats to be covered in a future post.

Simple Functions

1
2
3
4
5
6
def add(x,y): return x + y
def sub(x,y): return x - y
def mult(x,y): return x * y
def div(x,y): return x/ y
def reverse_sign(x): return -1 * x
def pow(x,y): return x ** y

There’s not much to describe here. The functions should be self explanatory. For purpose of emphasis I would like to note that in the above code, “add” is now an entry in the namespace which is a reference to a function. This reference can be passed around, assigned to other entries. Thus the code below should work (though it does not form a part of the calculator program). This is to demonstrate how python treats attributes and functions virtually identically consistent with the Uniform access principle.

1
2
3
4
otheradd = add
add = sub
assert otheradd(7,3) == 10
assert add(7,3) == 4

Currying
Currying is a treatment afforded in functional programming which allows a function of n parameters to be treated as a sequence of n sequential functions each of one parameter.

1
2
3
from functools import partial

square = partial(pow,y=2)

Here partial is a function reference. square now refers to another function with its y parameter value being anchored to 2

Invoking functions dynamically

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
unary_functions = {'!' : reverse_sign, '@' : square }

def handle_unary_op(ctx,x):
    return ctx._replace(current = unary_functions[x](ctx.current), op = None)

binary_functions = {'+' : add, '-' : sub, '*' : mult, '/' : div}

def handle_binary_op(ctx,x):
    return ctx._replace(op = binary_functions[x])

def handle_float(ctx,x):
    if not ctx.op :
        return ctx._replace(current = x)
    else :
        return ctx._replace(current = ctx.op(ctx.current,x), op = None)

Note that I created a unary_functions dict (or dictionary or hashmap) where the key is the character which represents the function and the value is the reference to the function.

Also note that in the handle_unary_opfunction, I invoke ctx._replace method. On a named tuple it creates another tuple based on the existing namedtuple data, but with some of the values modified as specified in the keyword paramters passed to _replace. After looking up the appropriate unary function ie. unary_functions[x], I also invoke it on the current value ie. unary_functions[x](ctx.current). I also defined another dict for binary operators. The handle_binary_op method reflects how the op in the context is set to the appropriate binary function that should be triggered after the subsequent value is known.

Finally the handle_float function either sets the current value to the incoming value or in case the current operator is already set it applies the binary operator to the current value and the incoming value and replaces current with the computed value.

Additional Code
When I wrote the calculator program, I wrote the functionality to introduce braces. However that functionality is not particularly important in this explanation. So it is being listed here for completeness.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
def start_brace(ctx):
    newstack = ctx.stack
    newstack.append((ctx.current,ctx.op))
    return ctx._replace(
                stack = newstack,
                current = 0.0, op = None)

def end_brace(ctx):
    stack = ctx.stack
    current = ctx.current
    oldcurrent, oldop = stack.pop()
   
    oldctx = Context(stack,oldcurrent,oldop)
    return process_key(oldctx,current)

tokens = { '(': start_brace, ')' : end_brace}

def handle_tokens(ctx,x):
    return tokens[x](ctx)

Processing one key
I must confess I started off using key to represent the keystrokes, but along the way the key can also represent a complete floating point number (not just a single keystroke). Thus the key parameter can refer to a single character operator or a sequence of characters representing a floating point number

1
2
3
4
5
6
7
8
9
10
11
12
13
function_groups = {
    tuple(unary_functions.keys()) : handle_unary_op,
    tuple(binary_functions.keys()) : handle_binary_op,
    tuple(tokens.keys()) : handle_tokens
}

def process_key(ctx,key):
    if isinstance(key,(types.FloatType,types.IntType, types.LongType)) :
        return handle_float(ctx,key)
    elif isinstance(key,(types.StringType)) :
        for function_class in function_groups :
            if key in function_class : return function_groups[function_class](ctx,key)
    return ctx

In this case I set up a dictionary where the key is a tuple of all the keys representing a particular class of a function. Note that the .keys() method is a method which returns a list of all the keys in a dictionary. However since list is mutable, it cannot get used as a key into the overall hashmap, hence I convert it into a tuple.

The process_key function takes the incoming key, passes it handle_float if it is a number, or treats it as an operator. If it is the latter it searches for it in all the keys of each operator groups, and if it finds a match, it locates the corresponding handler function from the map and invokes it. Finally in case no match is found it ignores the key.

Processing a sequence of keys

1
2
def process_keys(keys):
    return reduce(lambda ctx,key : process_key(ctx,key), keys, default_context())

Here you see a reduce function being invoked. This belongs to the family of map and filter functions which are used extensively in functional programming. I shall attempt to briefly explain it here, but this family of functions in addition to a number of others will again be dealt with in a future blog post.

To interpret the usage read the above reduce statement right to left. Thus we start with a default context, and for each key in the sequence of keys, we invoke a lambda (thats like an anonymous function), which calls process key with the context and the key. Note that the first parameter to the lambda is either the initial value (the default context) or the return value of the last process_key (which is also a context) and the key is each key in the keys sequence injected sequentially.

To further make it easy I re-represent the same function below differently which is much more readable and easier to understand. This shows one more strength of python. Because of its focus on readability, it actually can be used to write functional programs are much more readable by a large mass of programmers than most of the functional programming languages themselves (readability being subjectively interpreted by me as what is most natural for english or similar language speaking people).

1
2
3
4
5
def process_keys(keys):
    ctx = default_context()
    for key in keys :
        ctx = process_key(ctx,key)
    return ctx

Usage
As a mechanism to conduct some rudimentary tests on the code written so far, the following code is introduced. Here you can get an overall feel of the program.

1
2
3
4
5
6
if __name__ == "__main__" :
    assert process_keys((2,'+', 3)).current == 5
    assert process_keys((2, '!', '+', 5)).current == 3
    assert process_keys((2, '@')).current == 4
    assert process_keys((2,'+',3,'*',5)).current == 25
    assert process_keys((2,'+','(',3,'*',5,')')).current == 17

Some more slightly advanced Functional Programming

Finally to tickle your interest even more, here’s a slightly more advanced usage of functional programming constructs. Here I shall add all numbers between 1 through 10.

1
2
3
4
5
6
7
8
9
10
from itertools import chain

def plus_num_seq(n):
    count = 1
    while count <= n :
        yield '+', count
        count += 1

keys = list(chain.from_iterable(plus_num_seq(10)))[1:]
assert process_keys(keys) == 55

The plus_num_seq is a generator. Note the usage of the yield statement. Thus it will continuously generate tuples with the first element of the tuple being the ‘+’ character and the second being the number with the number varying from values 1 through n. The chain.from_iterable flattens the generated list (it thus has 20 items for n = 10, each alternate one being the ‘+’ character starting with the first items). Since we do not need the very first ‘+’ character, I removed it using the [1:] slice operator.

Just like reduce this style of code is quite typical of functional programming. Thats something I shall detail upon much more in future posts.

Hope you enjoyed the post. Keep the feedback coming so I can better structure the subsequent posts based on the feedback.

Note: The full source for the calculator calculator.py can be accessed here.

Internet and Social Media / web — 1 comment
10
Feb 10

Google Buzz Test Drive Report

Feature Notes :

Note: Google Buzz had not yet been enabled for me through normal GMail Desktop access. These notes are based on my experience with the same by switching my Firefox user agent to iPhone and accessing the iPhone Google Buzz UI.

  • Has a very facebook / friendfeed like interface.
    • Comments : People can enter comments against a post which appear sequentially below the post.
    • Like : Has the Like feature also supported by facebook / friendfeed
    • Inline images and videos : Images and videos appear inline as a part of the stream.
    • No 140 character limitation : Or at least none that I observed
  • Each conversation becomes a web page. eg. the comment stream against my very first google buzz post.
    • The same conversation is also posted to your inbox. The conversation in the inbox gets dynamically updated as the conversation continues. A conversation that has been read earlier gets marked as unread when more comments are entered against it.
    • The conversation web page becomes an alternative location for participation. If say the conversation link is forwarded to others, they can immediately comment on (or choose to “Like”) the conversation from the web page itself.
  • Private Channeled Messaging. You can mark posts as private but directed to a group of people. These groups are maintained and managed using Google Contacts. To the best of my understanding the future conversation is also constrained to the same group. Such messages are obviously not visible publicly. This is an excellent feature for closed group interactions.
  • Directed messaging is supported using the familiar twitter ‘@’ style. However in this case you use ‘@’ followed by the entire email address. Any posts made with ‘@’ addressing also get delivered to the user’s inbox.
  • Geolocation integration is supported. Thus you can view the public posts being made by people nearby you. Since I used firefox, it used firefox for geolocation support. I imagine and presume it would be using alternative superior mechanisms for mobile based conversations.

Comparison to Twitter :

One of the reasons I test drove Google Buzz was since I was wondering if it was a better Twitter. In many ways it is. But I don’t think thats likely to be universally true. There is the 140 character simplicity to twitter which is its identifying feature. In many ways Google Buzz is a Friendfeed clone – not a twitter clone (with added inbox integration). I always found Friendfeed much superior to twitter for conversations and I suspect those who shared that perception will find Google Buzz far superior to twitter too. However it will run into the same difficulty that Friendfeed did. If the twitter user doesn’t correlate his twitter account to the friendfeed account, it is very laborious in friendfeed to add a user you follow as an imaginary user in order to have his tweets appear in your friends stream.

I think where Buzz will take on Twitter .. and big time is the really large number of people who sign on to twitter, maybe make a few tweets and then say I don’t get it and go away. I suspect most of them will “get it” on Google Buzz – especially if they have any Facebook experience (which is a large large set of people). In that sense while Twitter appealed to only a small set of people, Buzz at least has the potential to appeal to the mainstream .. and in that sense go for the jugular – Facebook’s that is.

Comparison to Facebook :

Since facebook over a period of time has cloned its update stream to be like that of Friendfeed, it should come as no surprise that the Buzz stream is likely to look and feel quite similar. However two big differences stand out. The first is the way the social graph is managed. The social graph unlike facebook is asymmetric. ie. two persons don’t have to agree to be friends to follow each other. In case of Buzz, the social graph is managed through google contacts. You can follow people who have public google profiles or follow anyone in your contacts. Thus if a person doesn’t have a public google profile or his email address is not known to you, you may not be able to follow him/her. You don’t need the other person’s permission to follow. (At least thats the impression I formed, though I could stand corrected). The other big difference is the email integration. The entire stream and conversations are integrated into the gmail account. For most knowledge workers, I think these differences are likely to make for a more positive experience. In that sense – buzz is extremely well suited to be the Facebook of knowledge workers. Whether it can compete with the trivial, frivolous, chatty social networking capabilities of Twitter or Facebook is something I couldn’t form a clear opinion on.

On carving a new market :

Based on my earlier statements, it should be obvious that I believe buzz will carve new markets which did not use Facebook / Twitter / Friendfeed earlier. These markets are :

  • People who tried twitter but didn’t get it
  • People who primarily use email for communications instead of facebook / twitter (for both Personal and Business Networking)
  • People who would like to use Facebook / Twitter like capabilities from a knowledge working / business perspective.
  • People who are just too nervous to use anything new on the net may still find the email integration a continuum of their email exchange rather than a ‘new web site’.

On Privacy :

I think Buzz definitely ranks superior on privacy, given its capability to conduct private directed closed group conversations. As an example, I can imagine many google apps installations which could use buzz for intra company conversations even as they clearly restrict even the public streams not to be visible outside the domain. While I haven’t seen any comment from Google on whether buzz would be available for google apps, I imagine its only a matter of time since it is so particularly well suited for that environment.

On google wave :

Google wave in many ways was a damp squib. It imposed a new usability paradigm even as some of the capabilities such as near real time updates were clearly superb. Google buzz reverses the wave approach by offering a different conversation model which dovetails not only into similar usability expectations as defined by the competition but goes one step beyond by integrating into the killer web application – web mail. I wouldn’t be surprised if it came to light later that Google buzz is a google wave backend with a completely new usability experience.

A note on integration :

While its only a statement of intent at this stage, google has setup a site for the Google Buzz API. However the promise is awesome in terms of multi standard support for variety of standards and protocols support including RSS, Atom, PubSubHubbub, Social Graph API, OAuth, WebFinger and Salmon. It does seem that google will want to support a variety of other alternative and mashup use cases around Google Buzz which might allow for more interesting uses to emerge (as it happened for Facebook and Twitter). Clearly a space to watch with some interest in months to come.

In summary :

Google buzz makes the strongest case for appealing to the power email users and those particularly focused on privacy (eg. knowledge working for businesses). Its strong differentiator is its mail integration. It does offer adequate features to compete with Facebook / Twitter for their existing markets as well. And joins the party with a large number of people – who already have a google account.

← Older Entries