management

management / software — 7 Comments
19
Mar 10

Double whammy. The state and dilemma of Indian IT

Every now and then I come across a blog post which talks about outsourcing and soon enough a commentary or discussion on Indian IT and a whole host of associated parameters comes up. Soon enough some of them start attracting a number of views and comments. And more often than not the comment stream starts attracting far more extreme views which do sometimes leave me recoiled.

No, I am not going to list them, link to them or comment on their content specifically. After some thought I decided that simply would shift the focus unnecessarily. Instead I thought given my reasonably long experience both in and outside India, it would be helpful offer my perspective on Indian IT within the context of global IT and Indian economy.

Indian IT and the role of exports

Indian IT (along with ITES and Gems & Jewellery) is a one of the stranger segments of the Indian economy. Unlike the rest of Indian economy, it is heavily export focused. The various numbers reflecting the state of Indian IT agree on at least one factor – at least 2/3rd of its revenues are based on exports.

Let us first look at some of the perceptions about Indian IT that I would like to comment upon.

You get what you pay for : In general thats a very reasonable statement. But it breaks down when you realise one Indian engineer + 1 H1B visa + 1 long flight results in tripling (at least) of remuneration without either the H1B visa or the long flight adding anything to the capability or the quality of the output of the engineer (well it could add a chip on the shoulder). Part of the difference is due to the difference in nominal exchange rates (which are driven by demand and supply of goods to/from the respective countries) and the exchange rates based on purchasing power parity. Based on purchasing power parity Indian salaries in the IT space do not underperform the rest of the world substantially. In fact when compared on purchasing power parity Indian Economy GDP more than triples and it ranks as the fourth largest economy in the world next only to the US, China and Japan.

Indian engineers are offering their services too cheaply. : Quite to the contrary exactly the opposite is true. The Indian IT engineer is too handsomely remunerated compared to the non IT engineers in India. I believe the high salaries in Indian IT is a problem. Many Indian innovations are today happening in areas outside of IT – primarily in the areas of making products and services affordable to the millions surviving on the fringe of the poverty line. And if recent trends in automobiles and telecom sectors are indicative, India is actually proving, that it is the rest of the world which is way too expensive, by offering products and services at price points which are unfathomable elsewhere. My reading of the empirical evidence leads me to believe that Indian IT could underperform other sectors of the Indian economy in terms of both innovation and quality. And a lot of it is due to the fact that the export revenues offer cushy jobs without the really hard work it takes to compete within the Indian economy. Quite frankly this one stereotype must be deposited very quickly where it belongs. In the Trash Can. So let me repeat – Indian engineers are and Indian software is too expensive and containment in its cost growth is most urgently required. While containing salary growth might be useful, investing in ability to create high quality software upfront and eliminating the defect fixing cycles post the initial release will help bring the cost down.

Indian IT offers poor quality software (The alternative version is outsourcing leads to drop in quality) : This is too hard a statement to comment upon, given its utter and gross generalisation. I am not aware of specific quality benchmarks which could be used to assert or deny such claims, though there is a fair amount of empirical evidence which could be used to either assert or deny these claims. India like any other country has a range of software quality in different companies and products which span the entire spectrum from superb to utter crap. I tend to agree with the statement that in many cases Indian software output does leave a lot to be desired in terms of quality. I also believe that there is a curious dynamic at play here. It is well known that in software one can demand and expect any two of three parameters to be fulfilled – viz. Cost, Time and Quality. Any efforts to increase quality can in some contexts run into the business issue of the customer clearly prioritising cost and time to market. And remember two thirds of the revenues (and presumably the customers) of Indian IT are from overseas. I have often struggled hard on the quality aspects, and have generally found that it requires a very strong support not just from the engineers but from the business stakeholders to offer sustained high quality.

Indian engineers are not as skilled / Indian engineers are far superior to the rest of the world : This is an interesting stereotype which is obviously wrong at either of the two extremes at which it is observed. Some believe the Indian engineers are extremely competent and productive but fail to realise that these engineers are from the extreme top end of a very competitive educational system. Bring down the comparison to some reasonable way of comparing an average Indian and Non Indian IT programmer and perhaps the comparison may not be so rosy for India. I think India does need to work much harder to strengthen its capabilities of programmers at the middle and lower ends. And sometimes I blame the fact that too many projects being transferred to India has resulted into so much demand for programmers that a programmer with a 10 percentile performance can still make a wonderful living by just changing his jobs every two years. This needs to change. But I am afraid as long as India continues to bill the rest of the world by the hour and not by capability and quality, this shall continue to be an uphill battle even as we shall continue to see islands of excellence.

Double whammy

The double whammy I refer to in the title of this post stems from the fact that due to heavy reliance on exports, Indian IT has been substantially based on the priorities dictated by her customers. Thus Indian IT is largely today what its customers asked it to be. And most of the customers are non Indian. The double whammy is the fact that Indian IT gets criticised for becoming exactly what its customers asked it to be – fast, low cost and high quality so long as the high quality doesn’t interfere with the fast or the low cost :) . To be fair I am aware of the same criteria getting applied within many non Indian IT companies as well – so its not an Indian innovation. To be fair I am also aware of some situations where the lack of focus on quality is not driven by customer prioritisation. Is there a way out of the double whammy. Frankly I can’t see one easily. Yet this post would perhaps go some way in enriching the reader’s perspective on some of these factors.

The role of the population

There is one aspect about India which separates her from many other IT exporting countries. Her population. Not is it only really large (the second largest in the world), but it is growing fast and expected to continuously grow younger over the next two decades (dream demographics for an economist). Which means India stands unique in her ability to deploy masses. Which also means the problems which are easier to solve by deploying masses are more likely to find their way to the Indian shores. Combined with a high growth rate it also contributes to most Indian programmers getting promoted to management ranks (whatever that means) fairly soon. As we shall see in the next section this is an important factor in the Indian context.

Improving capability and quality

Independent of the validity of stereotypes which I questioned above I think it is a dead certainty that Indian IT can focus on improving both its capability and quality. It is not infrequent for me to feel demotivated after meeting some college students or fresh graduates and realising their priorities are really driven around what is the quickest way to make maximum money. This leads to unhealthy focus on building skills with high resume value. (Yeah that may sound funny to some of you, but its not uncommon). And given the high growth as I referred to in the earlier section – the role model for IT is – start as a programmer, change jobs every 2-3 years, become a team lead in 3 years, a manager in 6. And if you are particularly technically inclined you can become an architect guiding many projects and helping support many pre-sales efforts. What I haven’t seen Indian IT getting criticised for frequently enough, is the fact that so few of her members contribute to open source. While dzone and reddit may attract a large number of readers from India, a disproportionately small proportion of the people writing the posts stay in India. And most good programmers have moved on to becoming a Tech Lead or a Project Manager driving the average sustained technology experience lower and lower. So much for the “IT superpower” marketing that the hype manufacturing machinery creates internally. In a recent meeting with around 20 plus people in the room, I was one of the only 2-3 persons who believed India is not an IT superpower.

Change

I would like this to change. I would like this to change substantially. But I do not see the economic incentives in place for that to happen. Yet. However there is this strong feeling that something will indeed happen to make things change for the better. This is clearly something Indian IT will need to grapple with in the months and years to come. However there are at least a few factors which will lend themselves positively towards strengthening Indian IT.

Agile. Movement to agile requires a continuous focus on quality that cannot be wished away as easily. Projects that genuinely adopt agile methodologies will be implicitly driven toward being able to offer higher and sustained high quality.

Saas. As more software development shifts from intra enterprise development (where it is a little easier to contain impacts of poor quality) into Saas, there will be implicit pressures from the customers on the quality front. Another positive influence of Saas is that given the higher sharing of software across a much larger user base, the demand for number of developers would come down. Even as it may influence the demand for Indian IT itself negatively, I think it would help drive Indian IT into focusing more on capability than deploying masses.

Internal Growth. While much of the historic growth of Indian IT came from outsourcing, I anticipate the Indian industry to start driving its growth even more soon (given the really high growth engine it finds itself in). In such scenarios, the portfolio of software assignments will include a higher number of strategic and critical projects at extremely challenging cost parameters. This portfolio readjustment will help influence the quality positively.

I’ve attempted to highlight that Indian IT as exists today is (partially) a function of her customer’s expectations. And while there are some unfair stereotypes about it, there are clearly some things it can clearly improve upon. It is with some trepidation I write this since attempting to deal with perceptions in a generalised / stereotyped scenarios can be quite risky. So allow me to end with the disclaimer. This is a documentation of my understanding – YMMV.

management / programming / software — 1 Comment
14
Jan 10

Software development is about the middle and not just the end

Software development often gets similar treatment as other disciplines when it comes to matters of management. Often at a cost. And the cost comes from the fact that, the learnings and best practices gleaned from a number of disciplines are hard to apply in the field of software development, without understanding and dealing with some of the essential differences. And those differences and their implications are what this post explores.

There was an interesting post recently Why program with continuous time?.

Today I read a confusion that I’ve heard many times before about continuous time, which I’d like to bring up, in part because I love continuous time, and in part because there’s a much broader underlying principle of software design.

I don’t see why the lack of continuous streams leaves a “gap”. In the end all streams are discrete

“In the end”, yes. Just as in the end, numbers are displayed as ascii numerals. However, programming is more about the middle than the end, i.e., more about composition than about output. … Similarly, continuity in space and in time is better for composition/modularity, leaving discreteness to the output step.

The post above is completely unrelated to management. Its talking about a programming style called Functional Reactive Programming. But an idea it eloquently brings out is programming is more about the middle than the end, i.e., more about composition than about output. And thats a thought that I would like to expand upon.

While arguably in all disciplines the middle is also important, I would emphasise that it is particularly more so in software development. And that stems from a number of uncertainties that abound in software. The requirements are often not available in adequate detail, and to the extent available, are not stable. Moreover solutions to these problems are not all thought through upfront. Not because people don’t do their homework – its simply because it is actually extremely difficult to visualise all the moving parts, and then anticipate and resolve all the challenges. In fact many of the challenges don’t show up until you start programming. Thus programming often is an exercise in identifying surprises, resolving them, working out algorithms, discovering flaws with them, fixing these flaws only to find more surprises. Rinse and repeat. Basically its a significant exercise in uncertainty management. In such a situation, estimation results in underestimates since these are based on visible not actual challenges and surprises. The solution that does get applied with some success is to use historic estimation errors to compensate current estimates. (Note that thats not always obvious since the estimate could have the compensation already built in).

To add to the problem there is usually no Bridge 2.0, Lathe 2.0 or a Marketing Campaign 2.0 (and to the extent there might be some, these are far less dependent upon the quality of the work that went into the 1.0). Some programmers are continuously focused on 1.0, while many are thinking about the 10.0 even as they work on the 1.0. Optimising for 1.0 alone may not be the right way to approach an initiative. In the good old design methodology one attempted to anticipate and build for a lot of future changes. This requires a lot of extra complexity and development in terms of layering, parameterisation, open extensibility etc. In conventional agile methodologies the focus is much more on the 1.0, but there is a deliberate and definite cost introduced in terms of creation of a large number of automated test cases. These automated test cases is the assurance premium to be paid, in order to provide the comfort to undertake the necessary refactoring challenges when moving from 1.0 to 2.0. Thus whichever way one looks at it, the 1.0 is always suboptimised to allow for a more optimal 2.0, 3.0 etc. (note: I am referring to 1.0 figuratively as the current initiative and higher version numbers to future initiatives building on the current one). But selection of the necessary suboptimisations, with a view to optimise the overall sequence, requires a substantial understanding of the software development processes, the code, the tools etc.

Finally programming has a far far higher human element involved than other disciplines. After all there’s hardly any raw material dependency and the tooling is often a desktop, the cloud and set of open source libraries. Thus it is subject to the vagaries and frailties of human behaviour. Coordination, Communication, Productivity, are all held hostage by this fickle human element. While programming is not unique in this respect, achieving high team productivity is as much a function of morale, motivation, and communication as perhaps any other discipline. Thus progress is often non linearly proportional to such soft factors.

I believe there’s some benefit in evaluating software development management styles with how sports teams are managed. In sports, team performances are rarely consistent. Moreover even though team playing skills are critical, raw talent is also very important. But the analogy is most appropriate when one evaluates the game. The game is often a series of surprises reacted to by immediate on the fly action. Thats where adaptability and ability to counter attack surprises, can be effectively brought to bear. And in sports as in software development, player morale and motivation needs to be enhanced even as the player arrogance or dysfunctional behaviours are contained.

Since different sports are structured differently let me clarify the roles here. In some cases the manager and coach are different. In others they are vested in the same person. In my mind the manager’s role is resource management, broader strategy planning, impediment removal and stakeholder communication. The role of the coach is to enhance team skills, ensure fitness and training, and in case of many sports actively guide the team in refining the on field strategy when the game is in progress. Now lets consider the players. In most cases the players are living the game. They are in the middle. A substantial portion of their interesting professional time is spent in the middle. And when they are doing that, they are continuously dealing with a set of constant surprises, even as they broadly attempt to achieve the deliverables required for the end. A slight difference between sports and software development is that in case of sports the time is fixed and the score is variable. In case of software development at least in the non agile world, the score (feature set) is fixed and the time therefore becomes the variable. But the interesting takeaway from this analogy is that once the game starts, the ends become the background and the game dominates. The players and the developers are all living in the middle.

Why is this particularly relevant ? It is so because it helps us understand the implication of management styles in software development teams performance. If one wants to optimise – one needs to very well understand the middle and not just the ends. And (to come to the point) thats where I have a big issue with software developers, who end up imagining that as they rise up in the hierarchy, they somehow can stop worrying about coding and quality assurance, and instead start focusing only on managing people and deadlines. Thats a mistake. Since the game is played in the middle, you need to be able to help your team play the game. And you need to do it as a coach or a captain. For that, so long as you want to manage an ongoing game, you need to keep on playing and keep on being either sufficiently conversant about whats happening in the middle or be the captain and play the game with the team. Thinking that somehow you’ve grown in stature enough to not worry about the tactics, and focus on things increasingly strategic might actually make you unhelpful. The game requires skill and continuous adaptation – and you will need to be able to play, until you one day decide, you are no longer interested in managing the game but only the periods between.

The non playing managers who attempt to manage the game on the other hand display some interesting smells. They stop understanding the sport. They start believing that since they had played some particular sport many years ago, its easy for them to now manage any other sport. They are unable to appreciate the actual game play that happens. So when required to score a particular number of goals in 90 minutes (say in soccer), they start pushing in more team members (alas, this is a luxury not available to sports teams). Now on field coordination starts getting more difficult. The newer members being relatively junior are the ones who start getting intercepted more often (thus the team keeps on losing the ball more frequently). If the team luck is all run out, the same managers also attempt to run the tactics and gameplay selection. This is based on some fairly old set of experiences. The spectators (customers) start seeing far more confusion on the field. And the objectives change from playing the game well to just scoring the required number of goals. And in the end the spectators may get to see the number of goals promised, but only through a confused dissatisfying game, and that too sometimes only after many bouts of extra time. And soon enough once the non playing managers realise they are unable to effectively control the game play, they try to manage other variables. But really whats starting to happen is that they are hurting the gameplay, and sometimes instead of managing the game, they start managing the spectators with the game on autoplay. Suboptimisation maximised. Alas, sometimes the spectators also get used to this and start forgetting what a good game used to be like.

To summarise, ends are good to have, ends are important, ends provide the background urgency. But software development is not so much about managing the end goals as much as managing the middle. If you want to build good software and help your team build good software, make sure you continue to focus on how software is built not just the quantitative targets. Make sure you live and enjoy the middle. More often than not, the ends you achieve eventually could be amongst the best possible ends that were practical – but more importantly the customers will also go home satisfied. So long as you keep your eye on the ball and don’t lose sight of the middle. And if you want to view it a bit philosophically, in the end we are all dead. Only the middle matters.

Authors Note : This post has been substantially revised from its earlier version from a syntax, grammar and punctuation perspective. And the philosophical note (the last line) was added later.

architecture / management — 5 Comments
26
Oct 09

Service Oriented Architecture is primarily about business and not technology. Bollocks!

There’s quite a few times I’ve heard / read a gross oversimplification of architecture in reference to business and technology. And while I believe I understand the ‘essential cause’ which drives such a simplification, I’ve often felt quite frustrated at the resultant impression thats provided by such a simplification. In many ways and forms, it boils down to the statement (not exactly the same since I’m not quoting directly), quite similar to the one below :

Service Oriented Architecture is primarily about business and not technology

This also reflected in the recent SOA Manifesto which states as its very first described value :

Business value over technical strategy

Allow me to straight away start picking some holes into this :

  1. Anything that a business does – whether it is soa, software architecture, building architecture or simple plant and machinery design, to the extent (which is exactly 100%), technology serves the business goals, all technology activities (and non-technology as well) are at the end of the day about achieving business objectives and therefore about business. So why single out architecture? And even more so why single out SOA?
  2. Architecture is also about business. But its not the same as saying its primarily about business and not so much about technology. For a moment lets step away from Software/Hardware Architecture and look at Building Construction Architecture. The legendary creation of Ayn Rand – Howard Roark, for all his eccentricities and seemingly portrayed egocentric and egotistic behaviour did meet the test of business objectives to the extent of making the residents of his creations extremely satisfied. And at no point would you gather the impression that he in any manner put construction technology to any secondary position to his business context and objectives. At the end of the day thats what architecture is. It is not about making one of business or technology more important than or subservient to other. Its about effectively mapping the two to provide a strong technology solution appropriate to the business needs.

I suspect one of the important causes here is that people have forgotten that the A in SOA stands for architecture, and therefore shorn of architecture, business and technology can be seen to be competing in a non win-win form.

So if SOA stands for Service Oriented Architecture, then I must submit that architecture is the art of getting the two working together. And I am of the opinion that an exercise suggesting one is more important than other is an exercise in a field unrelated to architecture. I suspect many practicing software architects will agree with this. I suspect Ayn Rand wouldn’t disagree as well.

management / programming — Comments Off
28
Sep 09

The best amount of polyglotism is that you can manage successfully

Polyglotism was one topic I got into some discussions on twitter earlier this week. So this is to just pen down some of my thoughts on the same. Polyglotism is increasingly being talked about, ever more so since Java started moving from the realm of being a language into a runtime environment to host a number of languages. Yet polyglotism is not new. Most programmers have been mixing their favourite languages with SQL for ages. The advent of the web introduced HTML and javascript as additional languages that web programmers needed to know to be able to successfully write web programs. At the same time other tools started appearing which started to reduce your requirement to know these languages. Object Relationship Mappers could help you get out of writing SQLs (at least so they claimed :) ) in many cases. And one of the features of GWT is to ensure that you work primarily on only one language.

But the goal of this post is to neither encourage nor discourage the use of polyglotism. It is to suggest that at the end of the day even if polyglotism really stimulates the technogeek sensors in us, it needs to be evaluated not in technology terms, but in business and economic terms. A point that is also partially supported by The plague of polyglotism. Unless we use tools like GWT and hibernate to keep us monoglots, (which is likely to be a very small proportion), polyglotism is the default state that we shall operate in. So its not worthwhile to fight that trend per se. The important aspect is to ensure that we understand the costs of supporting each new language in the mix. And the reason to do so is to evaulate whether the benefits of adding the language outweigh the costs in the medium to long term. So here are some of the costs (in no specific order)

  • Syntax and libraries : This is probably one of the lowest costs even though it is the most apparent. For every language we add to the mix, generally for a long running project, at least three developers (if you have reasonable risk management objectives) need to learn the new language syntax and the libraries they work with. Along with this is the cost of books, training courses and materials etc. (yes for many enterprises, many developers but those purely self motivated will need to be formally provided all of these).
  • Idioms and Philosophies : Knowing the language is not adequate. You need to understand its philosophy, its design intent and its typical idioms. Being able to use a language to write code is not the same as being able to use it idiomatically. And while that may not constrain you in over small code bases in the short term, over a period of time for reasons such as performance, consistency and ability to understand other libraries that you may reuse, you will be forced to deal with these aspects of the language as well. The book Dreaming in Code talks about how Java programmers who started writing python could not write idiomatic python code, and thus a fair degree of code had to be redesigned or rewritten. And that is by no means the only such instance. I suspect similar trends are likely to be found if java programmers move to writing ruby or scala code. And its a trend you might have likely observed when people comfortable with writing relatively simpler languages such as Visual Basic or PL/SQL or even more complex languages like C++ moved to writing Java. Knowing the language is not the same as knowing it idiomatically – and that takes time. Which means – do not take on high risk activities when you absorb a new language, it will take a few iterations for your team to absorb the new language – far far more time than it will take to read a book or attend a training course.
  • Skills to manage multiple idioms in one head : Unless you are going to have different programmers for different languages, the programmers working on multiple languages at a time will not only have to deal with the multiple idioms, but will need to concurrently apply them. In my experience, this is not a trait we are born with, and it takes some time and experience with it. It will take some time for them to learn that the best practices are not identical across various languages. It will take even more time to figure out why they are different. So there is a good likelihood, you will not be in a situation to have all your developers work on all the languages simultaneously – especially some of the Junior ones.
  • Skill portfolio management will become more difficult : For typical enterprise or ISV teams, the managers need to ensure adequate skill availability across all teams. As the number of languages in use increase, this management process will become harder. Which means there is an automatic economic disincentive which rises is in magnitude, each time you add yet another language to the mix.

So is polyglotism good or bad ? I submit such a value attribute cannot be assigned to polyglotism per se. It needs to be really evaluated in your context, preferably beyond a project window into the medium or a long term. What is indeed obvious is that each new language will bring in some very strong capabilities and aspects where it is superior to the other languages in the mix. Equally obvious is that each new language will bring in associated costs. Just like there is no perfect language, there is no perfect number of languages that make sense in every context. Just like you do your homework in terms of language selection, make sure you do the same for your language portfolio not just in technology but in management and economic terms as well.

java / management / programming / scala — 9 Comments
17
Aug 09

Why should I switch to Scala ?

This post is a role-play and does not reflect my individual opinion about scala accurately. I am convinced about the capabilities and features of Scala along with the fact that it deserves the mantle of a long term replacement for Java. However language adoption goes beyond technical capabilities, and this post is a speculation on what a typical manager might be dealing with when attempting to decide whether to switch to Scala.

So I have been reading a lot about Scala lately and even opinions about how it will be a long term replacement for Java. I’ve also read some interesting writeups about Scala adoption such as On Scala’s Future and A Tipping Point for Scala. While I used to code a lot, my responsibilities today require me to interact with and address a lot of issues including those faced by our customers, our development teams and also engage with my peers and superiors on many other difficulties bedeviling our organisation. This gives me little time to try out Scala. I know I should be doing that, but sincerely I do not have the time. So I rely on the feedback of my team, the trade journals and other influential architects within and outside my organisation.

I have heard about many developers switching from Java to Python / Ruby. However I have heard of relatively only a smaller number of large Java shops which have done the shift – most of the switch stories I’ve heard reflect a smaller sized teams. I can feel the excitement Scala has generated amongst the development teams – the brevity, the functional programming model introduction, the exciting stuff being done concurrently et al. I have no doubt that, given so much excitement it must really be a good language.

To introduce my organisation – it is one of those shops which service many projects concurrently. Given the tremendous business and growth, I must confess we do not always have the luxury of being able to hire the most top notch talent. We do have a lot of projects we use Java for, and thats a language our customers are comfortable with. I’ve had some of the senior people check out Scala to gain some feedback into the language. But at this stage I must say I am inclined to evaluate the shift but not convinced enough to do so. I am sure that I could if convinced drive the change to Scala incrementally. However my fear stems from the fact that if things don’t turn out well, despite all the great advice I’ve received – its going to be my rear end on the line. So here’s some of my concerns regarding evaluating the shift to Scala and there are many of them, so some of you might be able to help me through this thought process.

  • Functional Programming : I’m sure in many ways it rocks. But my guys tell me they are not sure how to use it in the typical bread and butter applications which read from database, do some processing and write back to the database. Does Functional Programming help me in this context ? Will my team scale into being able to write functions with no side effects assuming thats a desirable goal ? What if they tie themselves up in knots and my release to the customer is risked ? I can’t afford that. Is functional programming even desirable in such contexts ? So I am not sure if in these contexts I should just ditch functional programming and work with just normal imperative programming capabilities of Scala. I am so confused, and afraid.
  • Different Syntax : While Scala runs on the JRE, its syntax is very different from Java. From what I could gather, it is much easier for a Java programmer to read (make sense of) simple Python code than to read Scala code. Is it true ? So even if I do get compatibility in terms of the runtime environment, would I be picking up a language that is syntactically so different a language that it would involve a substantial relearning curve ? I remember when we had to learn Java and Javascript. For better or for worse these were indeed relatively minor modifications of the C/C++ syntax, compared to what I sense as the syntactic shift between Java and Scala. Am I wrong ? If so, could you help point me to resources which help me understand that Scala code is not much different than Java ?
  • Sample code : Guys, I need your help. I need to see some good sample code. Some code which reflects how a typical application is architected, designed and programmed in Scala. And I don’t need it for a complex multi threaded actor based processing – I just need to see simple J2EE server based departmental applications maybe a simple recruitment tracking or library maintenance application. If I find a good one, I’ll just take it and give it to my team and say – there, thats how we’re largely going to build it, and even if we make a few changes along the way we at least have a reasonable template that we can build from.
  • Dumbed down environment : I remember my great adventures with C and vi and make. But my team today is very different. They want great IDEs. They must have syntax highlighting, autocompletion and nice refactoring capabilities. If I ask them to move, some of them might be excited about the change and be willing to overcome these short term hurdles. But there are some of them who will not be keen to do so and may be disinclined to support such a shift. And at the end of the day my ability to conduct this shift is a function of my ability to carry a large proportion of them along with me. Even when I considered a shift from svn to git, the IDE support was a big issue even though quite obviously git capabilities were really exciting. I couldn’t push along that change, and in this case we are talking of changing the language.
  • Is this a good time to shift to Scala ? I remember the early adopters of Java from 1996 thru 2001. While they gained a lot of experience, JRE and J2EE really matured only post JRE 1.3. Scala seems to be coming out with so many enhancements so fast, I am not sure if it has stabilised. I am told there is a 2.8 coming out in a few months. So if I train my team and Scala continues to change rapidly will I have to keep on retraining my team regularly ? And what about the customers I take to production. Will the frequent upgrades mean I end up supporting multiple customers on multiple versions of Scala ? Maybe Scala is stable but it would be helpful for someone important enough to make a clear statement that there are no new major shifts anticipated anytime soon and that these version shifts are likely to be no faster than the JRE version upgrades (which were fast enough).
  • Support from peers and superiors : I remember the day I decided to shift to Java. What made the move easy for me was the sheer fact that Java was a big paradigm leap away from the then dominant C++. Not only was it cross platform with binary compatibility thrown in for good measure, Sun ensured that it made all the right noises to appeal to the enterprise architects and all the business managers. I see the senior developers in my team clamouring for the shift to Scala, but my peer managers and my superiors don’t display even the fraction of the enthusiasm they displayed during the Java shift. The implication for me is that the risk cover I get when I order the shift is far lesser than what I had when I made the move to Java. Which means if things don’t quite work out well, I’m really going to be screwed.
  • Business friendliness : I understand all the nice talk about the technical excellence of Scala. But I really need to translate all these great language features into a projected ROI that I can use to convince others about. So I would like to see actual case studies of applications that were moved to Scala and what impact it had on the time and cost so that I can use it to compute my ROI. And what scares me is that learning curve may risk the initial applications long enough to push my breakeven point of shifting to Scala well beyond a 12 month and perhaps even a 24 month period. I fear things might not be as difficult but in absence of known studies, I am likely to lean towards projecting a worst case scenario rather than an optimistic one.

So folks, I am asking for your help. And while a lot of you may think that people like us who balk at the thought of limited IDE support are wimps, please remember that 80% of us don’t fit into the top 20%. And if you would like Scala to be popular, you need us as much as we need you. And if you are not too sure, please remember Lisp and Smalltalk are great languages as well.

management / software — 3 Comments
23
Jun 09

Most american graduates are unemployable because …

Came across this post Top Indian CEO: Most American Grads Are ‘Unemployable’ leading to some substantial discussion (not all of it impassionate). I would first like to clarify that I am assuming the content in the post to be accurate based on the fact that other Indian sites are reporting that “No response or clarification from Nayar has as yet been issued”. Moreover it is not targeted against Mr. Nayar or HCL but to the suggested thought process which attempts to make process orientation so critically important overriding many other qualities.

What perhaps hasn’t been clearly stated but is important is that that the standards for employability here are set by the employing organisation based on its context and preferences which may be different from standards required by many other organisations and I would submit that the statement ought to be looked at in that context as well.

To quote from the article :

The official wanted to know why HCL, a $2.5 billion (revenue) company with more than 3,000 people across 21 offices in 15 states, wasn’t hiring more people in his state. Vineet’s short answer: because most American college grads are “unemployable.” (In fairness to HCL, the company recently announced plans to open a delivery center in another state, North Carolina, and invest $3.2 million and hire more than 500 employees there over the next five years under a Job Development Investment Grant.)

Many American grads looking to enter the tech field are preoccupied with getting rich, Vineet said. They’re far less inclined than students from developing countries like India, China, Brazil, South Africa, and Ireland to spend their time learning the “boring” details of tech process, methodology, and tools–ITIL, Six Sigma, and the like.

As a result, Vineet said, most Americans are just too expensive to train–despite the Indian IT industry’s reputation for having the most exhaustive boot camps in the world. To some extent, he said, students from other highly developed countries fall into the same rut.

So why are some graduates are unemployable ?

Some of these are folks who are working on creating tons of new languages (Java, C#, Scala, Closure, Erlang, Python etc.), Operating Systems (Linux, Mac and Windows), frameworks for web applications, clustering, fault tolerance and scalability, schemaless and distributed databases to ensure availability and fault tolerance at a wide scale, competing messaging architectures that each service a particular problem differently etc. etc. Would you believe it some have set up clusters of  hundreds of thousands of machines and service search requests very rapidly. And some others are working on creating innovative and disruptive models in social networking, application integration, peer to peer networking etc. Many are working out next generation mobile technologies including building (not building on) android, iphone OS and the palm (whatever OS it has). And so many in their spare time are spending a whole bunch of time creating open source software, blogging and micro blogging about all the work they have done and sharing it with the wide world so that they can learn off it. And if these are consumer and technology stories, there are a whole bunch of people building critical business infrastructure architectures and frameworks and solutions as well. Even after you complete reading this post and come back to it a month from now, I shall still be busy trying to figure out what exactly India has been able to deliver that can challenge this. Of course let’s not miss out on the fact that most of these activities are conducted by fairly small sized teams. (I am aware some of the examples I quote could have non-american heritage, but that per se is not likely to detract from the issue)

Let’s call a spade a spade. When you have large or very large software construction and maintenance contracts, there are multiple ways to deal with it. In my experience, many Indian companiess have honed to a fine art the process of recruiting, deploying and juggling large armies of programmers to service such expectations (not all Indian companies are based on the same model). Many companies have indeed managed to acquire, retain and expand customer engagements through a combination of technology and business innovation. And they have done a good job of it in the context they’ve defined for themselves. In my limited understanding and experience it is not technology innovation, creativity or extraordinary technical prowess that is at the top of the list of skills that get deployed (though these are indeed found in sufficient levels across the board and some of the prowess can be impressive) – it’s the clockwork project management and business methodologies, techniques and innovation (and even these do result in projects with delays) with their reliance on “another brick in the wall” that gets the customer serviced.

So, if true, the “employable” was perhaps in that context.  If so based on the earlier two paragraphs, I submit it reflects positively and negatively on the employing organisation. Perhaps even more so than the graduates themselves. I do wonder if “incompatible” would have been a better choice of word. And I wonder what it reflected more poorly upon, as well who should be feeling more sad about it.

Disclaimer : These views are mine, mine alone and should not be puported to be shared by any other people or companies I’ve been associated with in the past, present or the future.

java / management / software / web — 2 Comments
22
Apr 09

Is a large corporate making money off open source or open standards an oxymoron ? In a Sun / Java Context

The recent acquisition of Sun by Oracle reignited a thought that had been going through my mind for a long time. It simply boils down to whether a large corporate can make money off open source or open platforms. Now quite obviously it in itself is not a truism. But the point remains that large corporates which have become large in the conventional economy do find the going a little difficult when trying to make money off open software.

The way I perceived it, Sun was going through similar difficulties. The hardware business was delivering dwindling margins post the dot com bust, and the software business was under threat from upstarts such as Linux which offered a substantially similar software stack at near zero licensing costs. One of the crown jewels asset Sun had in its stable was Java. And while it was (and continues to be) a wonderful asset, it just was incredibly difficult to make money off it. Now java hasn’t been open source exactly for most of its life, but it was a sufficiently open stack nevertheless to cause the same difficulties that open source software would in terms of monetisation.

There’s an excellent blog post written a year and half ago by Michel Bauwens, Can the experience economy be capitalist? which does refer to some of the underlying issues. I suggest you do read it, but would like to quote a few points from it below.

First of all, in the field of the immaterial, we are no longer dealing with scarce goods, but with marginal reproduction costs and non-rival goods. With such goods, sharing does not diminish the enjoyment of the good, since all parties retain their ability to use them. The emergence of peer production shows a new form of creating value, that is in fundamental aspects “outside the market”. Typically, in commons-based production we have a common pool, accessible to everyone (Linux, Wikipedia), around which an ecology of business can form to create and sell scarcities (usually services and experiences). In sharing-oriented production (YouTube, Google documents), we have proprietary platforms that enable and empower the sharing, but at the same time, sell the aggregated attention (a scarcity), to the advertising market. Finally, in the third crowdsourcing mode, companies try to integrate participation in their own value chain and framework.

So the good news is that indeed business is possible. But I would like the readers to entertain the following proposition, nl. That:

1) The creation of non-monetary value is exponential

2) The monetization of such value is linear

In other words, we have a growing discrepancy between the direct creation of use value through social relationships and collective intelligence (open platforms create near infinite value through the operations of the laws of Metcalfe and Reed), but only a fraction of that value can actually be captured by business and money. Innovation is becoming social and diffuse, an emergent property of the networks rather than an internal R & D affair within corporations; capital is becoming an a posteriori intervention in the realization of innovation, rather than a condition for its occurrence; more and more positive externalizations are created from the social field.

Quite eloquently argued here that open platforms create near infinite value which is difficult to be captured by business and money. I still remember companies such as Borland, HP, Sun, IBM making a fair amount of money through selling C/C++ compilers and IDEs (I remember it used to be almost $5000 per seat). However this was in the days when the community capability of creating similar competing software stacks was only in its infancy. No longer so today. Now communities, open standards, open processes and open source are fairly well established sources of delivering asymmetrically significant capabilities at a fraction of the cost, a fraction which is made even smaller when the same is incurred by a small motivated group of individuals or small highly agile companies. There in lies the difficulty. Companies are trained to and built to deliver linear and symmetric capabilities in the context of costs they incur. However they have a relatively poor handle on monetising in scenarios where small teams can deliver exponentially asymmetric capabilities.

The blog post I referred to above, identifies the right area to look at in this context – Identify what is scarce and Monetise the scarcity. So when any software has a sufficiently large utility and can be managed through open processes to be able to satisfy a globally substantial demand at a fraction of the cost (eg. apache httpd), that particular capability (eg. static and dynamic file serving over http) is no longer scarce enough for a commercially focused company to make money out off. Thats why the money shifts where the scarcity is – how to leverage such software and put it to good use. This is where the existing commercial successes around open source are based on eg. RedHat, JBoss, IBM, Oracle etc. They monetise the support, training, services and consulting around such software. In many ways Sun could be argued to be a victim of its own success. It not only went much further than any other large corporate in terms of creating open specifications, it also contributed substantially to sufficient knowledge dissemination and tooling to allow many other individual, SME and large corporates to be able to compete with Sun in these very areas that Sun could’ve monetised. Sun thus created its own competition for monetising the scarcity that got created around Java even as many others cried out hoarsely that Sun simply wasn’t open enough. Will there be an incentive for Oracle to reverse that somewhat ? I suspect that could be a logical option if it focuses on ensuring a good ROI for Sun’s Java investments and assets and more so to be able to make itself more capable in the space around Java than the competition (ie. IBM).

So is a large corporate making money off open source or open standards an oxymoron ? Viewed narrowly yes. Because while it is possible for a small team to make some money with adequate ROI off open source or open standards, it is unlikely to be feasible for large corporates. They really need to figure out what is the new scarcity that gets created off such initiatives, come to a judgment that the opportunity arising from such scarcity is large enough and worthy of their time, attention and investment, ensure that there is no current available capability of anyone else disruptively delivering exponentially asymmetric value in that space, and finally occupy that space before other large corporates do.

It does boil down to the fact that while individuals and small companies may find it easier to monetise open source and open standards, the same is going to be generally tough for large corporates. Or as Michel Bauwens said in the blog post I referred to (in my opinion perhaps a little exaggerated but not entirely off the mark)

For all of this, we will need new policies, major reforms and restructurations in our economy and society.

But one thing is sure: we will have markets, but the core logic of the emerging experience economy, operating as it does in the world of non-rival exchange, is unlikely to have capitalism as its core logic.

java / management / php / programming / python / ruby / software — 4 Comments
29
Mar 09

Talk Slides : Programming Language Selection

Slides of my talk yesterday on programming language selection (from a technical and a business perspective)

Programming Language Selection

Background article (not exactly same but on a similar topic) : Improve your web based software development and maintenance ROI with dynamic programming languages

Talk Announcement : Seminar: Strengths and weaknesses of various programming languages – 28th March

← Older Entries