Terrace (noun) : each of a series of flat areas made on a slope, used for cultivation, or a flight of wide, shallow steps providing standing room for spectators in a stadium
Recently there was a useful discussion triggered off by the post True Scala complexity by Yang Zhang. Much has been debated about it in the comments on the post and other forums, most of it both articulate and constructive.
Martin Odersky proposed an idea on the ycombinator news thread as follows (next paragraph)
So, I believe here is what we need do: Truly advanced, and dangerously powerful, features such as implicit conversions and higher-kinded types will in the future be enabled only under a special compiler flag. The flag will come with documentation that if you enable it, you take the responsibility. Even with the flag disabled, Scala will be a more powerful language than any of the alternatives I can think of. And enabling the flag to do advanced stuff is in any case much easier than hacking your own compiler.
The basis of the idea wasn’t new. Martin himself had earlier suggested Scala Levels, though the suggested enforcement of two levels via a compiler flag was. While I thought it was a useful idea, strangely enough I found many (especially in the twitter streams I follow) less than enthusiastic. I think there exists a perspective where one can look at this idea with more enthusiasm, and this post details that.
Learning scala is non trivial. For a person who is well versed with Java programming and with little else, this will require learning at least the following (I am listing only a few items)
- Learning traits and objects (and unlearning statics). There’s a lot of stuff here including multiple inheritance, member overriding, whether the members should be defs or vals etc. which takes some times to get used to.
- Learning to deal with immutability and occasionally lazy evaluation
- Learning collections constructs such as for comprehensions or, map, flatMap, filter etc.
- Understanding the capabilities that scala offers in terms of generics including being able to define co/contravariant collections, and then learning to use these capabilities effectively
- Being able to understand and leverage many aspects of type theory
Typically when a person makes a sustained effort driven by passion, he can cover a lot of ground very quickly. Thus some might be able to deal with the scala learning curve quickly. While this ability to learn could be achievable for many especially smaller companies, it is likely to be quite difficult for many others. Especially when the team sizes are large and/or developer capabilities might be modest, this is a real issue. One could form an analogy with a Cheetah and an Elephant. Some animals will be able to run fast individually, and others will trod slowly as a herd. One could wish elephants could run as fast as cheetahs but thats unhelpful, since the jungle is defined, and one best deal with it the way one finds it.
An ideal way to introduce scala would be to start with small groups of 3-5. But thats not practical in all cases. Besides for large teams, that means, most will need to wait for a much longer time before being able to use scala and profit from its substantial benefits. So lets say an organisation attempts to introduce scala in a modest team of say 20. Of which 2 are passionate, self driven souls, who tend to work much harder the rest to be able to progress much faster. Because learning to use scala the way scala wished to be used, is not a sprint but is a marathon (or at least a long distance jog), this creates issues. Soon enough the 2 are likely to run far ahead of the rest of the team and will start writing code which the others can’t grok. Even as some of the remaining may actually be struggling hard but are likely to be losing enthusiasm simply because they can’t find themselves being able to keep pace. An even bigger risk is the 2 continuing to make rapid progress even as they realise so much more remains to be learnt, even as the remainder are unable to start leveraging scala, since they still think there’s a lot of distance yet to cover and they become far more focused on catching up rather than leveraging what they have learnt. So while learning proceeds, actual development with direct economic benefit stalls.
Lets say hypothetically this team decided that they would only focus on reducing java boiler plate (which is a big deal) and continue to use scala in a primarily OO paradigm (since that is what they are used to) and defer learning remainder of scala by at least six months. There’s likely tons of benefits to be had by shifting to scala even with this limited scope. Yet all it takes to destroy harmony is one very enthusiastic developer. Who starts inserting (as yet) alien notions such as writing highly functional code, or implementing advanced type theory concepts (eg. scalaz) or starting to use a largely un-understood symbol soup (scalaz or the dispatch periodic table). This would make a mockery of the phased learning and force every one else to catch up (even if just to be able to read and maintain somebody else’s code).
We’ve long known hills are not conducive to agriculture. If these hills are our learning curves, fields could be considered to be directly impactful deployment of our skills. Agriculture generally happens flatlands and plateaus but not hillslopes. And yet we’ve learnt to tame the tall hills using terraces. Instead of climbing for a long time, climb a little & grow a little. If teams can learn for a month, and deploy these skills for the next 6-9 months with the cycle repeating itself, it could help in the following ways :
- Organisations do not need to invest in long training times where the opportunity cost of lost development time is one large fixed cost.
- Intra team disparity is likely to be contained since the deployment period will allow many to catch up with those in the lead
- Instead of scala being viewed as one big leap, it could be viewed as a incremental leaps with managed effort. I saw a remark which said people anyways deal with complexity – its called OO. And Java programmers deal with OO. So they should be able to come to terms with scala quickly. But there is a difference. Java wasn’t as big a learning curve leap over C++ as Scala is over Java. And as importantly, a learning curve one has triumphed over already, somehow seems far less onerous than an identically sized one, one still needs to deal with.
In short learning scala needs terraces. Terraces, where some developers will not run far ahead of the pack during the learning stages. Terraces where developers won’t have to feel too scared of alien’ish code suddenly appearing on their monitors. Terraces where developers will learn a few things and then deploy these skills over the next few months. These terraces could be made to work in different ways. It could just be a honour system using a set of levels identical or similar to the levels Martin Odersky proposed (though it would be extremely hard to ensure the same without tooling support). Or they could implemented using (as yet unwritten) tool like a scala-lint which will flag off advanced usages as warnings. Or as compiler switches the way Martin proposed.
There could be other ways to look at it too. Scala could be considered as a sharp weapon. Lethal in the hands of the trained. Self damaging in the hands of the untrained. So compiler switches could simply be a way to graduate from using sticks, to wooden swords, to blunt steel swords to the really sharp ones.
I saw a remark which said a compiler switch would divide the programmers. I seem to think such steps which help create terraces will divide the learning challenges. I also saw another remark which said, implementing compiler switches would play into the hands of languages like Kotlin or Ceylon or Fantom. I believe exactly to the contrary. The lower terraces can compete with Kotlin or Ceylon, and the upper terraces can provide a growth path that many other languages will not have. Not being able to plan a phased and an economically practical learning curve for large teams will make the case for other languages stronger.
Related posts: (Automatically Generated)
Thanks for this blog! I could not agree more.
I don’t like the idea of a compiler flag. I would prefere something which is more local. Maybe import the “complex” feature, like i would import strange scalaz syntax.
Something like Haskell’s LANGUAGE annotations, perhaps:
{-# LANGUAGE FlexibleInstances #-}
I really like this sort of file-by-file à la carte system for enabling special features.
Interesting thoughts.
However, I think devs will find natural terraces, which will vary from team to team and be agreed upon by experience, convention and just a natural sense of what people can understand.
Compiler flags make a clear statement about where the ‘language designer’ thinks the terraces are, which might not match the local situation. Its a kind of artificial barrier.
Its even possible that orgs will have conventions like ‘we don’t use the xx flag here’, which limits what people can do unnecessarily.
Channing
Fair Point. Compiler switches would tend to absolutely classify important language characteristics, whereas terrace identification could be driven by evolving needs.
+1 to @Channing
I am very sceptical about the practicability and advantage of this approach.
In my experience bigger than small enterprise projects tend to put very restrict rules on when to use higher levels or feature flags on.
So instead of being able to solve issues devs will have go through permission processes, being confronted with denials due to limited understanding, build process checks to confirm compliance a.s.o, in the end resulting in a code base with more and more subversive workarounds .
The result is not better understandable and maintainable code but more complexity, as the feature’s implementation use explicitly visible proprietary patterns now.
Beside that there will follow some sort of elitarianism. “He works for framework team. They *are* allowed to put flag x on!”
Result: An additional organisational hell beside the one opened by the discussion about Scala vs. Java, which exists per se.
There are organisations which control and teams who self manage. I believe self managing teams could be better served by some tooling support. Organisations which tend to control, are more likely to issue diktat’s anyways (I’ve seen a few coding standards in the past which suggested “thou shalt not …” even when there was no tooling support). So yes, what you say is a valid point, but I suspect that we’ll see good applications and bad applications of the feature, in some cases even if tooling support was not offered.
Organisations that control will control regardless of compiler switches. Self-managing teams shouldn’t find it too hard to agree amongst themselves which features they want to use. I don’t see the utility of a compiler flag here. The large and small Java teams I’ve worked in have had coding standards agreed to suit the team and/or the organisation and I don’t really see why Scala needs different treatment in this regard.
If a particular team would find tooling support helpful, then something like Checkstyle for Scala seems like a much more flexible and less heavy-handed approach than compiler switches.
I strongly disagree. This basically creates new language which is not compatible with original Scala. And it creates even more complexity. Imagine beginner wondering why ‘this example I found on the net’ does not compile.
Perhaps these are two different ways of looking at it. But I never perceived these as different languages. Just a “compilation profile” if I may use such a word, which allows a subset of language to be leveraged. Yes, a beginner may get an error/warning with regards to some code constructs, but compared to the amount of cognitive effort a scala programmer would normally incur during programming, this seems like a pittance.
Having said that I of course do not mean to contest your disagreement. I’m sure there are good reasons each one of us might feel differently.