March 2nd, 2010
What is the maximum number of developers you can ever imagine working _efficiently_ on a project ? 5 ? 7 ? 10 ? 20 ?
Who wrote 2.6.33 reminds us how much the open source world (and in particular the linux kernel community) excels in this area. For the single 2.6.33 release that was developed in about 3 months :
“As of this writing, 10,500 non-merge commits have found their way into 2.6.33 – fairly normal by recent standards. These changes added almost 900,000 lines while deleting almost 520,000 others; as a result, the kernel grew by a mere 380,000 lines this time around.”
So, if you happen to struggle to scale your team past the 10-people mark using your usual development habits, then there are maybe a few things you could learn from the open source world.
My personal understanding of why it works so well :
- good elite developers
- top-notch, distributed, super-fast and merge-friendly version control tools (e.g. git)
- Fault-proof and compromise-free (though sometimes not politically-correct) ways of enforcing software quality and architecture. Examples showing the disagreement-proof nature of the kernel development process include last summer’s Alan Cox vs Linus Torvalds dispute regarding the tty subsystem, or Linus Torvalds vs Hans Reiser argument regarding Reiser4’s plugin system that does not fit well into linux architecture
- result-oriented and meritocracy-driven way of managing the project
- decentralized development (made possible thanks to distributed SCM tools). To quote Linus Torvalds : “Centralized _works_. It’s just *inferior*.“
- modular architecture supporting the collaboration of many developers. “The large number of developers and the fact that they are volunteers has an impact on how the system should be architected. With such a large number of geographically dispersed developers, a tightly coupled system would be quite difficult to develop — developers would be constantly treading on each others code.“
You might disagree on the reasons why it works so well (after all, that’s just my analysis based on my understanding of the situation), but the success is a reality, a fact.
Also, if you are tempted to think that it could not work in the corporate world, please think about that twice by taking another look at who wrote 2.6.33, where contributing companies are listed.
Posted in Software Development best practices | No Comments »
March 1st, 2010
Looks like go is attracting some attention.
“”Open source does not mean anarchy. Somebody has to have a vision and the perseverance to see that through. The open source community can then create their own versions if they wish, but it is best if there is a main line, stable version with a consistent architecture with a guiding force behind it,” Gordon said.”
Posted in Uncategorized | No Comments »
February 27th, 2010
The Scala 2.8 beta 1 announcement gives hope regarding the availability of a decent IDE for editing Scala code. We will see what Scala 2.8 final looks like, but if the eclipse IDE support features basic Class and Method renaming, I will most likely make Scala my main programming language for writing open source code that targets the JVM. Two projects that I would most likely convert to Scala would be :
- Gisgraphy Java client : a Java library that gives access to gisgraphy City and GIS features search engine.
- Pymager Java client : a simple Java wrapper on top of the RESTful interface provided by pymager, an image service that provides simple conversion and thumbnailing / resizing features.
Posted in Uncategorized | 2 Comments »
February 27th, 2010
In many situations, open Source software developers need to deal with the maintenance of patches. Examples include :
- Unofficial versions of the linux kernel, where specific patches are applied (e.g. Xen kernel, openvz kernel, ..) and need to be constantly forward-ported to the latest kernel when it is released
- Distribution-specific changes (e.g. Ubuntu-specific changes to debian packages).
Maintaining one big diff file for all changes would clearly quickly become unmaintanable, so it looks like different approaches are now widely used instead :
- Maintaining stacks of patches, using specialized tools such as quilt.
- Using distributed VCS tools such as git.
How to survive with many patches describes the use of quilt. Here is some background :
“Andrew Morton originally developed a set of scripts for
maintaining kernel patches outside of any SCM tool. Others extended these into a suite called quilt.
It looks like distributed VCS tools have now superceded quilt, as far as pure software development is concerned (linux kernel, etc.). But quilt remains very popular for maintaining distribution-specific changes to packages. Indeed, as distribution packages live outside a SCM tree, it is important to have mechanisms to apply distribution-specific changes to the upstream source packages. And this is where quilt comes to the rescue.
Ubuntu packaging guide describes the use of quilt in debian’s packaging system. Such a simple system is clearly awesome, and the more I understand how the Open Source communities organize themselves, the more it makes me realize how technically advanced the Open Source world is compared to the corporate world !
No matter how much you might have heard that tooling is unimportant, the reality is that tools are important. Tools enable complex collaboration, and this is clearly an area where Open Source excels.
http://www.suse.de/~agruen/quilt.pdf
Posted in Uncategorized | No Comments »
February 27th, 2010
Cumulative Flow Chart in Kanban attracted my attention as I consider it a nice example of using branching efficiently.
IMHO, it is simply wrong to assume that every single task can be split into small fragments that are then iteratively incorporated into the mainline. The author calls this kind of task a “technically complex story”, and I have yet to see a successful example of migrating frameworks or doing technical migrations like that without resorting to branching. This is what the whole open source community does, and it is high time the “enterprise” world catches with these practices.
Branching then comes to the rescue ! Let the “technically complex story” evolve in its own branch, and make sure to conduct in-depth QA BEFORE the merge. Same thing for code reviews and going through the DONE checklist : make sure to do it BEFORE the merge so that you do not end up with non production-quality code in the mainline, which is then pretty hard to get rid of.
BTW, Version Control Tools gives an overview of the differences between git and mercurial, which are two wonderful SCM tools that are very branch-friendly.
Posted in Uncategorized | 1 Comment »
February 26th, 2010
why can’t programmers…. program is definitely worth reading… Yes, this is disturbing….
Posted in Uncategorized | No Comments »
February 26th, 2010
The worst question interview ever describes… well, just read the post and you will quickly figure out what it is about. I simply LOVE Gavin King’s comment :
Heh, and I’ll continue to be an ass in all future responses to “John Smith”s who tell talented guys who worked on my projects for years that they aren’t “team player”s, “have an attitude” and are “prima donna”s. I’ll be the judge of that, not some asshole anonymous blog comment poster who has never met or worked with Norm. I’m protective of my team. That’s not going to change. Sorry if you don’t like it.
YES, Gavin King is an ass, but most of the time, this category of ass is right, and people should just listen to them instead of complaining.. ah ah ah
love it !
PS: for those who don’t already know it, Gavin King is the founder of Hibernate…
Posted in Uncategorized | 2 Comments »
February 26th, 2010
EJB 3.1, a compelling evolution describes the new features available in EJB 3.1. It looks like EJBs are finally getting the features they miss..
However, there are still a few things bugging me :
- Why insist on keeping the neat features (IoC, ..) server-side only ? Why can’t I just use the same mechanisms for in-container server-side code and other kind of code ? Do I still need to revert to using Spring for everything that is not running inside the JEE6 container ? What about integrated tests ?
- How come we still don’t have any equivalent to Spring templates, that take care of creating standardized, runtime exceptions and handling opening/closing resources automatically ?
So for now, when working in non-spring environments, I need to create my own Templates to avoid creating clumsy code…
I agree that Spring Framework is a patch, and should eventually disappear. But for it to disappear, the underlying technologies need to start being half-decent…
Posted in Uncategorized | No Comments »
February 26th, 2010
These days, it looks like there is a lot of hype around the NoSQL movement.
These data storage systems have a number of features in common:
• a call level interface (in contrast to a SQL binding)
• fast indexes on large amounts of data,
• ability to horizontally scale throughput over many servers, and
• ability to dynamically define attributes or data schema.
NoSQL explained correctly gives an idea of what these datastores are useful for, and how they complement the current RDBMS offering.
High performance scalable datastores compares the technical characteristics, maturity and licenses of the NoSQL offering.
Other links on the subject that might be of interest :
Posted in Uncategorized | No Comments »
February 24th, 2010
It looks like that the future of OpenSolaris is unclear, now that Oracle has acquired Sun..
I take the official Oracle website to be rather … official ?
Lets recap, shall we ?
a) Almost every trace of OpenSolaris Support subscriptions vanished from
the official website within the last 14 days.
b) An Oracle sales rep informed me personally last week that I could no
longer purchase support subscriptions for OpenSolaris.
More information available on this thread.
Posted in Uncategorized | No Comments »