I have 8+ years experience on C++ and Java each and at least consider myself an expert on the latter and used to consider myself one on the former a long time back. For any programmer it is a difficult choice to move away from the platform and the environment in which one has both a substantial investment into and into another one where you essentially throw away years of experience and start as a novice (at least in pure programming terms).
This is not a post which is either to be interpreted as pro Python or anti Java or a pro/anti “name any language you wish”. I will gladly and gleefully go back to C++ / Java when using them makes sense under a context. I am sharing my thoughts and not promoting or denigrating any languages or frameworks here.
Context :
Here’s the shift of context (not all aspects are necessarily relevant to the shift in language). There are many details I have deliberately not got into for obvious protection required for any commercial activity.
- From a relatively large programming team that I used to manage into a small one (yours truly only to begin with)
- From building a commercially owned closed software into open source software development (what is at this stage intended to be though it will take a few months to get there)
- From customers always being large corporates who could often afford substantial hardware into customers who can range from individuals to large corporates
- From mostly internal (intranet) facing to internet facing
- From performance requirements of upto a few hundred thousands of transactions per hour into a completely wide range of requirements based on each customer
- From a very high percentage of writes to a much smaller percentage (ie. read percentages are now much higher)
Initial Choice Set
Given the fact that this application is intended to be hosted on the internet and is primarily a browser based application my choices quickly narrowed down to Java / JEE (something I had a long experience on), PHP (I had developed one web based application with it), Ruby and Python (I only had academic exposure to these). C/C++ did not figure on the choice set for their obvious development overheads in the context of web applications.
I went through a fair degree of thought and creation of dummy applications and the mental to and fro and the the process was not nearly as linear as I will describe below. The process is simplified below simply so that the reader can get at least some insight into my mind and my mind is made to seem a lot less confused than it actually is.
1st Elimination :
Java was the first one to get knocked off. One reason was that Java based applications typically require either a dedicated host or have to work under memory constraints in case of shared hosts. Simply put Java scales exceptionally well but it has a minimum hardware / investment requirement which was not acceptable within this context. The application should be able to worked on shared hosting environments. Another equally important reason was that the productivity of initial development and that of making changes to Java applications is much lesser than the other languages. I was only too acutely aware of the performance implications of this choice, but I believe I the appropriate choice here shall be to scale out especially where the read activity is especially large compared to the writes. Scaling out does require more complex architectures (compared to simply scaling up) but thats the way that is appropriate in this context.
Subsequent thought process
This one was much tougher. Let me delve for a moment into what I believed the key strengths of the various languages were.
Ruby : I really really loved the syntax. Compact, cute ‘n’ thoroughly OO. Strong metaprogramming capabilities
PHP : Massive developer base (especially important when the intention is to eventually open source the application). It is amongst the easiest languages to use. Another advantage in its favour is the ‘C’ness of the syntax which makes it easier for anyone coming in from the C/C++/Java world.
Python : The metaprogramming and OO were almost but not as good as Ruby. I really love the indentation driven syntax. I know many might differ but I really like it and the neat paragraphs and lack of block braces make the code a lot more readable. Current production interpreters seem to be the best performant compared to Ruby and PHP. I know Ruby 1.9 is getting much faster but I suspect it is unlikely to be enough to make it much faster than Python already is.
As I considered the languages, it was important to look at the frameworks. I looked at CodeIgniter, CakePHP and Zend for PHP, Rails for Ruby and Pylons and Django for Python.
I believe one of the important aspects of architecture decision making today is that you bring in the available toolsets / frameworks into the decision making process even when you are attempting to select a language. You are in effect evaluating a package involving both the language and the framework. A specific issue to be noted in my context was that regardless of what framework I chose I was completely sure I would need to change it / extend it due to the fact that some of the basic requirements of the application I am about to build – no well known existing framework supports them. I am certain and convinced this is not a “Not Built Here” syndrome that I am suffering from but simply a necessity of the domain I am attempting to work with.
In terms of ease of use for simpler applications I would rate Rails very highly. Not only does it make the actual programming simple, it gives you a nice set of tools around it to make a lot of typical activities really easy.
2nd elimination
Ruby and Rails went out. The reasons were as follows.
- Ruby is just so well designed from a syntax and OO perspective, that coming from so many years of C++/Java background with a substantial grounding in Object Orientation as implemented by these languages, I really did not get the confidence that I could do it sufficient justice. My fear was that I would keep on finding ways of doing things in a better way in this language (and given my inherent compulsions would feel inclined to refactor code rather than focus on newer development).
- I perceived that Rails was really focused on typical much simpler use cases. What I intend to do requires getting into an immense amount of complexity and I felt fairly certain Rails wasn’t designed for that and that I would spend a very very substantial time reinventing or hacking through Rails.
- One of the strong features of Rails was something I couldn’t really completely come to terms with was “Convention over Configuration”. I still can’t get over some of the implicitness in the environment.
3rd Elimination
Clearly PHP is such a widely used language with so many developers who are already trained on using it, that using it for an as yet intended to be open source application should be a no brainer – Right ? Not in this case. Two reasons why PHP went out.
- I intend to pull off some really complex programming. Given the better OO and metaprogramming capabilities of python – I thought I would be able to keep my code much better concise, structured and readable if I was to use Python (this would’ve been true with Ruby as well!).
- Django – This framework simply came closest to being the framework I would really like to end up with. Thus the gap between what it has vs. what I would like to have was the smallest, and the general design principles were those that I was extremely comfortable with.
In summary : Why these got eliminated
- JEE : The difficulty of using JEE in shared hosting environments and the long development times required made it tough to use it.
- Ruby : I like the language. I was a little intimidated by it. Rails however seemed limited to simpler use cases
- PHP : Wonderful developer base. However didn’t give me the same comfort in terms of its OO capabilities and metaprogramming and did not believe the resultant code would be the most compact, concise, and readable.
In summary : Python and Django : What do I hope to get from them
Now that I have made the choice – what do I expect from these choices at this stage :
- Excellent framework to start off with – provide the maximum initial boost
- Ability to write concise, structured and readable code
- Ability to make changes rapidly
- Reasonably performant application
Other choices that I did not spend too much time on
During this process I also evaluated other languages such as C# and Groovy/Grails. I actually was quite impressed with the recent architecture trends coming from Microsoft and was actually tempted to spend more time on them. However the same reasons that eliminated JEE quickly made these choices unviable as well. I wish I had the luxury to consider other languages as well, but these were what were at the top of my mind, and I did not consider any other languages in the process given the time constraints.
I am missing JEE
I really really will miss JEE. I would love to have used it for the simple fact that I know it so well and this one will require me to move from an expert to a novice category. I will also miss it for the fact that with JEE I knew what it took to deliver exceptionally high performance. Even though I am feeling the JEE separation pangs, I believe Python is the right choice since it will allow for the fastest development, will allow for some really rapid changes (agility in coding), and will allow me to get the developers who shall eventually be joining us come up the curve much faster and be able to deliver more features much faster.
Final thoughts
One thing I realised through the whole process was that there are two strong influencing factors to any architecture choices. The first one obviously is the context. There is no way to compare or contrast any architecture or design choices without putting a context around it. Secondly and this was a little bit more of a surprise to me was that you simply cannot remove personal proclivities from such a choice making process. Since I as an individual am more comfortable with some styles of design than others, and since I am likely to be substantially involved with the initial development, it only seems to make sense that the choice set gets evaluated in this subjective context of individual comfort and therefore the projected individual productivity and effectiveness as well. Note that while in this case I was evaluating it strictly from my own perspective, one could evaluate the choices in the context of Team styles, culture and comfort as well.
No related posts.
thanks for sharing! Very interesting post, I like how you present factual data and your personal thought. Very inspirational.
This is a very good demonstration that for any project, one should take some time to think of the best tool to use (not necessarily the one we’re used to).
PLEASE look at Merb, Ruby will make you so happy!
I can tell you have done your homeworks and I came to the same conclusions one year ego. Nevertheless I would love to hear your opinion about web2py.
@Massimo Given the broad range of languages I covered, it simply was not feasible to look at every single framework. web2py was not something I looked at so would be unable to formulate an opinion. However what I did like about django was its modular and layered design, fairly broad feature coverage, performance metrics, community size, the various independently developed contrib modules / djangosnippets.org and the substantial yet efficient use of metaprogramming in its db.model and admin modules. If I do get around to checking out web2py I shall most certainly ping you on the same.
PLEASE look at Merb, Ruby will make you so happy!
I can tell you have done your homeworks and I came to the same conclusions one year ego. Nevertheless I would love to hear your opinion about web2py.
@Massimo Given the broad range of languages I covered, it simply was not feasible to look at every single framework. web2py was not something I looked at so would be unable to formulate an opinion. However what I did like about django was its modular and layered design, fairly broad feature coverage, performance metrics, community size, the various independently developed contrib modules / djangosnippets.org and the substantial yet efficient use of metaprogramming in its db.model and admin modules. If I do get around to checking out web2py I shall most certainly ping you on the same.
I do understand when you say:” Rails however seemed limited to simpler use cases” and that is ActiveRecord fault, maybe you shod try using Datamapper or Sequel for your model layer, that will give you some more flexibility specialty in legacy data models, the normal way to evaluate them is using Merb frame work.
I do understand when you say:” Rails however seemed limited to simpler use cases” and that is ActiveRecord fault, maybe you shod try using Datamapper or Sequel for your model layer, that will give you some more flexibility specialty in legacy data models, the normal way to evaluate them is using Merb frame work.
I can understand one of your concerns about Ruby and doing justice to the language, getting caught in re-factoring, etc. Actually, I think we all go through that with Ruby by its very nature. However, its expressiveness and meta strengths keep us. We all need to exercise a little self-discipline…
Rails may “seem” limited, but I don’t think it is and coupled with Ruby, the sky’s the limit. You can mix-out pretty much anything and there are many other libraries which meet needs and “roll right in”.
Nice post. Thanks!
Whatever happened to Pylons?
I’m going through a similar language/platform selection process now – with many of the same constraints as you had. I too was leaning toward Python. One very significant hurdle with Python is that my application will be heavily text search driven and I haven’t yet found a search toolkit with even close to the power that Java Lucene has. Using PyLucene would introduce the same resource problems as using Java for the web framework. In my case, the cost of more expensive hosting may be less than the cost of using a non-Lucene search option.
Whatever happened to Pylons?
smacko, Xapian might be an option. It’s C, so you still might have problems on a hosted machine, but it should be comparable to Lucene and has a *very* nice Python wrapper with xappy.
I gave a hard look at Ruby on Rails for a latest project. It sounded cool but then there is a huge learning curve and finally had to settle with my well known territory, .NET
This comment is from cies. Unable to extract it from its resting place at my is_human wordpress plugin. Hence reposting on original author’s behalf. (Just enabled the new plugin – need to figure out what to do in such cases).
By cies on June 9th, 2008
i\’d like to second the \’dude\’ poster.. maybe you want to look at merb + datamapper. it is addmittedly not as tested as django but the setup of this framework makes a lot of sence (re-using ruby\’s gem framework for it\’s components and plugins, very loosly coupled, you can choose stuff like HAML and fly, many deployment options (including mod_passenger now more to come (jruby?)), ruby syntax all the way, super small mem footprint, eventbased, thread safe, ok i stop now) anyway, django looks nice as well nice write up.
I gave a hard look at Ruby on Rails for a latest project. It sounded cool but then there is a huge learning curve and finally had to settle with my well known territory, .NET
Good write up. Yeah I unfortunately have come to the same conclusions. With Python + Django you’ll be writing an app with potential for high speed AND high productivity. Not as cute as ruby, but without the deathly magic of rails, and faster. So it wins, as much as I hate to say it [and I love Ruby and work on it all day!]
Maybe maglev will change all this. Hope so
-R
@jt
{I can understand one of your concerns about Ruby and doing justice to the language, getting caught in re-factoring, etc. Actually, I think we all go through that with Ruby by its very nature. However, its expressiveness and meta strengths keep us. We all need to exercise a little self-discipline…}
I just might be able to get the self-discipline … if only I had the time to get there
. As I move towards dynamic languages – python seemed more manageable a shift in the given time frames than Ruby. I loved Ruby syntax and especially all its nifty features but found Python easier to read.
{Rails may “seem” limited, but I don’t think it is and coupled with Ruby, the sky’s the limit. You can mix-out pretty much anything and there are many other libraries which meet needs and “roll right in”.}
My assessment was that everything was doable .. but the cost factor here was much higher compared to hacking through django.
{Nice post. Thanks!}
All the encouragement helps. Thanks.
@elhumano
I preferred the everything in one box capability of django. I would like to keep framework changes to the minimum, and rather than having to study multiple frameworks and figure out how to change these in an appropriately interdependent way – made sense to have a single framework with all capabilities. Just makes it a little easier I guess.
@Rich
Nice post. The contrasting between mod_python and fastcgi in the context of hosting environments is quite helpful. Glad to know you liked the post.
Hi there,
I recently went through a very similar process, coming from a Java background. I also settled on Python and Django for similar reasons.
If you are interested, I wrote it up here:
http://jetfar.com/django-python-frameworks-and-vps-hosting/
Although perhaps not as succinctly as yourself.
BTW: I’m very happy with the outcome.
@jt
{I can understand one of your concerns about Ruby and doing justice to the language, getting caught in re-factoring, etc. Actually, I think we all go through that with Ruby by its very nature. However, its expressiveness and meta strengths keep us. We all need to exercise a little self-discipline…}
I just might be able to get the self-discipline … if only I had the time to get there
. As I move towards dynamic languages – python seemed more manageable a shift in the given time frames than Ruby. I loved Ruby syntax and especially all its nifty features but found Python easier to read.
{Rails may “seem” limited, but I don’t think it is and coupled with Ruby, the sky’s the limit. You can mix-out pretty much anything and there are many other libraries which meet needs and “roll right in”.}
My assessment was that everything was doable .. but the cost factor here was much higher compared to hacking through django.
{Nice post. Thanks!}
All the encouragement helps. Thanks.
@Finebanana
Ahh ! I skipped that bit about Pylons – didn’t I ? I didn’t find anything worrisome about Pylons – but what I knew for certain was that I would need an extended admin interface and would need to hack through the models. Pylons does not really get into those spaces much. Django offered a more comprehensive set of capabilities all nicely interdependent on each other (especially admin and models).
@elhumano
I preferred the everything in one box capability of django. I would like to keep framework changes to the minimum, and rather than having to study multiple frameworks and figure out how to change these in an appropriately interdependent way – made sense to have a single framework with all capabilities. Just makes it a little easier I guess.