This post is a role-play and does not reflect my individual opinion about scala accurately. I am convinced about the capabilities and features of Scala along with the fact that it deserves the mantle of a long term replacement for Java. However language adoption goes beyond technical capabilities, and this post is a speculation on what a typical manager might be dealing with when attempting to decide whether to switch to Scala.
So I have been reading a lot about Scala lately and even opinions about how it will be a long term replacement for Java. I’ve also read some interesting writeups about Scala adoption such as On Scala’s Future and A Tipping Point for Scala. While I used to code a lot, my responsibilities today require me to interact with and address a lot of issues including those faced by our customers, our development teams and also engage with my peers and superiors on many other difficulties bedeviling our organisation. This gives me little time to try out Scala. I know I should be doing that, but sincerely I do not have the time. So I rely on the feedback of my team, the trade journals and other influential architects within and outside my organisation.
I have heard about many developers switching from Java to Python / Ruby. However I have heard of relatively only a smaller number of large Java shops which have done the shift – most of the switch stories I’ve heard reflect a smaller sized teams. I can feel the excitement Scala has generated amongst the development teams – the brevity, the functional programming model introduction, the exciting stuff being done concurrently et al. I have no doubt that, given so much excitement it must really be a good language.
To introduce my organisation – it is one of those shops which service many projects concurrently. Given the tremendous business and growth, I must confess we do not always have the luxury of being able to hire the most top notch talent. We do have a lot of projects we use Java for, and thats a language our customers are comfortable with. I’ve had some of the senior people check out Scala to gain some feedback into the language. But at this stage I must say I am inclined to evaluate the shift but not convinced enough to do so. I am sure that I could if convinced drive the change to Scala incrementally. However my fear stems from the fact that if things don’t turn out well, despite all the great advice I’ve received – its going to be my rear end on the line. So here’s some of my concerns regarding evaluating the shift to Scala and there are many of them, so some of you might be able to help me through this thought process.
- Functional Programming : I’m sure in many ways it rocks. But my guys tell me they are not sure how to use it in the typical bread and butter applications which read from database, do some processing and write back to the database. Does Functional Programming help me in this context ? Will my team scale into being able to write functions with no side effects assuming thats a desirable goal ? What if they tie themselves up in knots and my release to the customer is risked ? I can’t afford that. Is functional programming even desirable in such contexts ? So I am not sure if in these contexts I should just ditch functional programming and work with just normal imperative programming capabilities of Scala. I am so confused, and afraid.
- Different Syntax : While Scala runs on the JRE, its syntax is very different from Java. From what I could gather, it is much easier for a Java programmer to read (make sense of) simple Python code than to read Scala code. Is it true ? So even if I do get compatibility in terms of the runtime environment, would I be picking up a language that is syntactically so different a language that it would involve a substantial relearning curve ? I remember when we had to learn Java and Javascript. For better or for worse these were indeed relatively minor modifications of the C/C++ syntax, compared to what I sense as the syntactic shift between Java and Scala. Am I wrong ? If so, could you help point me to resources which help me understand that Scala code is not much different than Java ?
- Sample code : Guys, I need your help. I need to see some good sample code. Some code which reflects how a typical application is architected, designed and programmed in Scala. And I don’t need it for a complex multi threaded actor based processing – I just need to see simple J2EE server based departmental applications maybe a simple recruitment tracking or library maintenance application. If I find a good one, I’ll just take it and give it to my team and say – there, thats how we’re largely going to build it, and even if we make a few changes along the way we at least have a reasonable template that we can build from.
- Dumbed down environment : I remember my great adventures with C and vi and make. But my team today is very different. They want great IDEs. They must have syntax highlighting, autocompletion and nice refactoring capabilities. If I ask them to move, some of them might be excited about the change and be willing to overcome these short term hurdles. But there are some of them who will not be keen to do so and may be disinclined to support such a shift. And at the end of the day my ability to conduct this shift is a function of my ability to carry a large proportion of them along with me. Even when I considered a shift from svn to git, the IDE support was a big issue even though quite obviously git capabilities were really exciting. I couldn’t push along that change, and in this case we are talking of changing the language.
- Is this a good time to shift to Scala ? I remember the early adopters of Java from 1996 thru 2001. While they gained a lot of experience, JRE and J2EE really matured only post JRE 1.3. Scala seems to be coming out with so many enhancements so fast, I am not sure if it has stabilised. I am told there is a 2.8 coming out in a few months. So if I train my team and Scala continues to change rapidly will I have to keep on retraining my team regularly ? And what about the customers I take to production. Will the frequent upgrades mean I end up supporting multiple customers on multiple versions of Scala ? Maybe Scala is stable but it would be helpful for someone important enough to make a clear statement that there are no new major shifts anticipated anytime soon and that these version shifts are likely to be no faster than the JRE version upgrades (which were fast enough).
- Support from peers and superiors : I remember the day I decided to shift to Java. What made the move easy for me was the sheer fact that Java was a big paradigm leap away from the then dominant C++. Not only was it cross platform with binary compatibility thrown in for good measure, Sun ensured that it made all the right noises to appeal to the enterprise architects and all the business managers. I see the senior developers in my team clamouring for the shift to Scala, but my peer managers and my superiors don’t display even the fraction of the enthusiasm they displayed during the Java shift. The implication for me is that the risk cover I get when I order the shift is far lesser than what I had when I made the move to Java. Which means if things don’t quite work out well, I’m really going to be screwed.
- Business friendliness : I understand all the nice talk about the technical excellence of Scala. But I really need to translate all these great language features into a projected ROI that I can use to convince others about. So I would like to see actual case studies of applications that were moved to Scala and what impact it had on the time and cost so that I can use it to compute my ROI. And what scares me is that learning curve may risk the initial applications long enough to push my breakeven point of shifting to Scala well beyond a 12 month and perhaps even a 24 month period. I fear things might not be as difficult but in absence of known studies, I am likely to lean towards projecting a worst case scenario rather than an optimistic one.
So folks, I am asking for your help. And while a lot of you may think that people like us who balk at the thought of limited IDE support are wimps, please remember that 80% of us don’t fit into the top 20%. And if you would like Scala to be popular, you need us as much as we need you. And if you are not too sure, please remember Lisp and Smalltalk are great languages as well.
Related posts: (Automatically Generated)
If you plan to move to Scala big bang, it may not be possible with the kind of setup that u have described. The best way, I think, is to move incrementally. The biggest advantage of Scala is that it runs on the JVM and has a very good integration with Java. One of the ways u can adopt is to start building smarter APIs on top of your Java objects. That way, some of your clients can use the smarter APIs, while others can continue to use the legacy Java APIs. This way you can start slowly injecting the goodness of Scala within your team. I had adopted a similar approach, which I documented at http://debasishg.blogspot.com/2008/01/scala-can-make-your-java-objects-look.html. And regarding using Scala at the Java EE level, JPA, JMX etc., you will find lots of information and implementation in Jonas Boner’s blog and github repository.
Debasish,
I (and the role playing manager states as well) agree that incremental adoption is the best way to go. The essential point I was attempting to bring out is that a lot of Scala discussion is a the level of APIs, algorithms, concurrency etc – imo there is a paucity of examples that are directly relevant to the level of an average enterprise programmer who has modest capabilities and works with web and form based applications, an audience that needs to be catered to for mass popularity. I shall review Jonas Boner’s blog as you suggest to see if I can find information that might be useful to such an audience.
And yes, I do remember reading your post some time back – it was one of the few early ones which did get me excited about the power of scala.
First, I agree with Debasish on moving slowly, and that there are many benefits of the move. I work with a small team of pretty senior people and have been using Scala on an existing Wicket/Hibernate/Spring/JMS app. Here is how I address your points.
Functional Programming: though our app is not functional, the use of closures, list operations and pattern matching, makes our business logic more concise and easier to read. You don’t have to go totally functional to get benefits. Code like people.filter(_age < 30) is very clear and has much less noise than equivalent java code.
Different Syntax: this is probably the biggest issue we have had with the transition. The flexibility of the language can cause issues for new people that are hard to track down. For example, val x = new Object and def x = new Object are called the same way (x.foo), but the method returns a new instance each time the val does not.
Sample code: our app was a java app first and is architected as such. I don't think we would change anything if we had used scala from the beginning. One thing to know is that until 2.8 Scala doesn't support nested annotations so we use java for all JPA beans.
Dumbed down environment : We use IntelliJ and it supports syntax highlighting but not much refactoring. It is getting better with every release though. I feel more productive in Scala with lesser refactoring.
Is this a good time to shift to Scala?: I don't know enough about your situation to comment intelligently.
Support from peers and superiors: Once again I don't know much about your situation, but the switch to Scala is nowhere near as risky as the switch from C++ to Java. Scala and Java are both JVM languages and play nicely together. If a developer doesn't like Scala, let him/her use Scala
Business friendliness: I can't quantify it, but I certainly feel more productive in Scala.
I'd say do what your team wants. If they want to use Scala they will be more productive with it. If they are scared of it, they might fail. Finally, I'd say you need a competent champion: someone to change Ant (or whatever) to compile the Scala files, someone to help junior guys with compiler errors they haven't encountered before, someone to teach the cool features that can really save lots of time.
Good luck.
Hi Dhananjay,
I have felt that as a hobbyist early adoption is fun, but as a business late adoption is often more wiser.
I totally agree with the learning curve you mention. Learning and doing new things is imperative, but with so many new things happening at such a fast pace, I feel that it is also important to weight in on which new thing you want to experiment with and adopt.
Functional programming does have benefits, but I suspect that the shift to functional programming will take a while, just as the shift to OO from procedural took a while. Maybe this shift will happen faster because of the need to write safe programs for multicore processors, but at the same time I remember reading that some libraries are coming up for Java using which we can get closer to writing multi-threaded code which does not use the shared memory model. If this is true then do you think people prefer to continue using Java with these new libraries?
I am not quite sure how this will play out. There are a lot of people screaming the demise of Java, but change especially when so much is invested in code and skills does not come that fast.
Excellent blog, I really admire the honesty. “I must confess we do not always have the luxury of being able to hire the most top notch talent.” You should know that you are not alone here. In fact, I will readily call BS on most arrogant programmers who claim that they only work with a team of rockstars. The truth is that hiring is a crap-shoot. What most people do is wind up hiring other people who are similar to them, and since we all think that we are smart, we think we hire smart people. Steve Yegge had an excellent blog post on this a while back. Anyways, here are a couple of suggestions…
Yes, it is true that Scala is a hybrid object-oriented / functional programming language. I would recommend just forgetting about the functional part of it. Treat it as a Java++. With Scala you can code in a manner very similar to Java. Only because of things like type inference, case classes, and traits, you’ll only need half the lines of code compared to Java. I would guess that people will start using closures over time as well. This is what Ruby/Python programmers do. Nobody would describe those as functional programming languages, but developers take advantage of functional features without even realizing it. You can do this with Scala and you will get a similar LOC as you would with Ruby/Python, but you will get IDE support, better performance, and of course good interop with legacy Java code.
You won’t see the Scala momentum coming from business people. There is no corporation behind Scala, like there is with Java. Ruby got some hype because of the quick time-to-market promised by Rails, but it is really just a niche language at this point. PHP has major adoption, but it rode in as part of the LAMP stack that promised big savings courtesy of free open source software. There is no angle like this with Scala, so its momentum will come from technical thought leaders who aren’t going to put up billboards or make mass mailings. The main non-technical hype generator around Scala has been Twitter’s high-profile switch to it.
I see Scala (with good IDE support) being a common way that large development organizations increase their productivity. I suspect that many such orgs struggle with productivity and wind up looking to things like agile/scrum to address this.
As for the stability of Scala, 2.8 is a big release. There is some extra sugar in the syntax, particularly named parameters and specialized annotations. However, the language is relatively unchanged. I don’t think you have to worry about people writing code today that they would have to change in the future. The major thing that I could see changing in the next few years with Scala is with regards to concurrency. Martin Odersky has stated that he wants Scala to be the best language for concurrent programming. So I would expect Scala to offer more options (software transactional memory for sure, IMO) and to improve its existing options, i.e. improve actors. It does not sound like this would affect you. I would only worry about this if your main reason to move to Scala was its actor library.
Finally, on the topic of J2EE-ish apps, I would recommend looking at Lift. It works inside Java web containers. It has its own ORM if you like, or you can use JPA (i.e. entity beans 3.0.) If you write a lot of web applications that are either CMS-ish (i.e. you are constantly grabbing data from a database to display on a web page), e-commerce, etc. then Lift will give you rapid development similar to Rails. Ajax is also a given in Lift, not an afterthought, and it can really make your life easier. Here are a couple of Lift articles I wrote http://www.ibm.com/developerworks/opensource/library/os-ag-lift/ and http://www.ibm.com/developerworks/ajax/tutorials/wa-aj-comet/
Very constructive response. A similar tone from the Scala community towards the programming masses is something I think will be essential for Scala to achieve mainstream popularity.
Thats an advice that should be more visible. Debasish offers a similar one in an earlier comment as well. If potential adopters can realise they can actually phase out their adoption, and really defer functional programming, it would certainly make the adoption seem a little less onerous.
I really don’t know how, but I do believe mainstream popularity will be greatly eased by a conducive business environment. Otherwise Scala might need far more than just a Twitter’s high profile switch to help it get there. Sure technology community driven adoption is feasible, but witness the degree of difficulties through which REST adoption is only now starting to compete with SOA in the enterprise – and that happened only after a large number of internet sites started offering REST APIs. I would suggest if there are ways to get the business community onboard on the Scala momentum, these should be examined and some collective attempts could be made in that direction. These could include but need not be restricted to a welcoming tone (no to “why do you need IDE support to switch to Scala ?”), a site with examples that businesses could readily use (a la Java BluePrints) and probably a Learn Scala in 21 days / Scala for dummies tutorial explicitly targeted to Java programmers.
This is a personal view point – but I think there’s work to be done here. And while many believe that people shouldn’t gripe about whats missing and pitch in to help instead, let me step around that argument for a moment. When I decided to switch from Java to Python, a very large factor that really helped me was the availability of comprehensive examples and reference I could readily use and reuse. eg. The Definitive Guide to Pylons and The Django Book. I find myself today having an ability to contribute, back but back then I was just looking for help, and such wonderful help made it easy for me to switch. On a separate note, I would very much like to build a strong skill set in Scala and do continue to read up eg. on Code Commit by Daniel Spiewak, but I really don’t get the time to actually delve into the code of various open source projects despite having received some pointers by some helpful folks. Have I seen anything that comes anywhere close to the two Python resources I referred to, something that doesn’t show off the power of Scala but instead focuses on making it easy ? Not yet. But maybe I haven’t searched for it hard enough. And yes, I know specific cases where people backed off introducing Scala into a Java shop, choosing to defer it instead.
None of this is meant to be viewed in a negative light on the language or its technical capabilities or the community around it. It is instead meant to highlight a set of possibilities which if exercised could ease and hasten the process of Scala gaining mainstream populartiy.
Functional Programming – I would NOT put this first on a list of scala features that motivate switching; Scala allows functions as objects and they support Scala’s richer API, but selling Scala as FP is only a plus to CS PhD majors and Reddit; sell it on its other strengths.
Syntax – It IS different, but once you learn it, slowly, it is not bad. I find it certainly more comprehensible than python; Scala has braces for scoping, e.g.
Sample Code – I have found this a serious lacking; most sample code is math-based and very hard to follow or extremely abstract (and therefore not as useful as it could be).
Environment – I don’t use IDEs, and I certainly won’t put my ability to learn and expand my knowledge in the hand sof others (in this case, the IDE plugin writers). They can catch up to me, not vice versa.
Time to shift – was 2001 really Java early adoption? Wow. No time like the present. Lead or bleed. You want you and your team to be leaders or followers?
Support – Again, do you want you and your team’s abilities to be in the hands of others? Most of your developers (and probably you) will be doing Ruby or Scala professionally one day. Might as well get started.
Business Friendliness – I think Scala can easily support enhanced developer productivity, but it’s hard to know that for sure.
You should also consider developer retention. Assuming your team isn’t comprished entirely of scared clock-punchers who can’t get too far away for their F5 key, you probably have some senior developers who like learning new things and want to apply them to their work. Such people are highly valuable and are also likely (and able) to land jobs elsewhere, doing Scala, Ruby, or whatever. Losing them is a blow to your team. Far more of a blow than not having refactoring support in an IDE.
[...] the latest post on his blog (which you must subscribe to), he is pretending to be a manager who is resisting exhortations from [...]
> I must confess we do not always have the luxury of being able to hire the most top notch talent.
One thought I have is that using a language like Scala may help a situation like this. If you put “Scala” (or Clojure or F# or any other sophisticated non-mainstream language) on the job description do you a) attract more top-notch programmers and b) weed out more dunderheads? I honestly don’t know the answer and I don’t think anything like a proper study has even been attempted, but some anecdotal evidence suggests that this might be the case.