Sami Dalouche

GeoTools developers have released the 2.6.2  version. GeoTools contains an incredible amount of utilities related to GIS and I am totally impressed by the feature set.

To give an example of its use, here is some sample code from gisgraphy-java-client ( a simple Java client I am writing for the open source GISgraphy project).  It calculates the orthodromic distance between two coordinates :

    public double distance(GisFeatureGeography o, Unit unit) {
           Unit targetUnit = (unit != null) ? unit : SI.METER;
           com.vividsolutions.jts.geom.Geometry me = location;
           com.vividsolutions.jts.geom.Geometry other = o.getLocation();
           try {
                   return  SI.METER
                          .getConverterTo(targetUnit)
                          .convert(JTS.orthodromicDistance(
                                 me.getCoordinate(),
                                 other.getCoordinate(),
                                 DefaultGeographicCRS.WGS84));
           } catch (TransformException e) {
                   throw new RuntimeException(e);
           }
   }

Please note that the code makes use of two excellent libraries : JTS for geographical types, and JScience for units. And for your information, WGS84 is a friendly name to refer to the GPS coordinate system (x,y,z).

There is a constant about Software Developers : they love debating and arguing about every single aspect of the development process. Moreover, they will most likely debate forever, because there is usually nothing that can serve as a reference to tell good and bad practices apart. Want to know why ? Well.. everyone is making his own opinion based on his own vision of the truth. There is no axiom that is taken for granted and that serves as the basis for further discussion.

“In traditional logic, an axiom or postulate is a proposition that is not proved or demonstrated but considered to be either self-evident, or subject to necessary decision. Therefore, its truth is taken for granted, and serves as a starting point for deducing and inferring other (theory dependent) truths.”

The rest of the mathematical logic is based on these axioms, and given these axioms, everything else can either be proven right or wrong. While it is certainly impossible to create universal axioms and reasoning principles that cover the foundation of software development, I believe we should at least mimic the approach : decide of which design, architectural and coding principles we believe on, and then use these axiomatic principles as the foundation for decision making. Of course, the outcome of your next project will depend on the  quality of these axioms, but at least you will be able to move forward and take consistent decisions throughout the lifecycle of the application.

As far as I am concerned, I tend to base my reasoning on a set of axiomatic principles that are based on the opinion of respected and talented people in the software industry. Even though nothing is perfect, I believe that listening to these experienced people will more likely lead to successes than listening to any lambda developer’s opinion. This is my bet, and the rest of this post is a first draft of the main design and architectural principles that I consider my axioms.

• Domain Driven Design, Specifications pattern, Layered design.
• Principles of OOD, and in particular, the S.O.L.I.D. principles
• Clean code: A Handbook of Agile Software craftmanship.
• Implementation Patterns
• Implementation of the right Enterprise Integration Patterns that best describes a given integration issue
• Defensive Programming (also see Defensive programming in Java) and its derived principles : Design by contract, Assertions,  Immutability, Copy-On-Write, Immutable collections.
• Inversion of Control
• Declarative Programming, and in particular Functional Programming-style collection manipulation and Command/Query separation, side-effect free methods.
• No premature optimization.
• Lessons learnt from UNIX programming.
• Tell don’t ask.
• Robustness principle : Be conservative in what you do, be liberal in what you accept from others.
• Do not reinvent the wheel.

The people behind these principles are smart, experienced, and potentially more intelligent than you and me together. So let’s just follow these principles for now, and once we master every single aspect of them, we will be able to help create the next generation of design and coding principles. In the meanwhile, I take these principles for granted.

It is a pleasure to hear that some open source projects are conducting usability reviews :

• Wikipedia : Usability and Experience Study
• OpenStreetMap: UX Review
• any other ?

Usability reviews are of uttermost importance if you want to learn how your end users use your product. I initially thought that conducting usability reviews was complex, and involved lots of steps, but Jean-Francois Proulx definitely convinced me of the approach when I attended one of his usability talks a few months ago.

To me, this is clearly a better investment than having your market-team meet during long hours in front of screenshots trying to shape the next version of your website

So…. you’ve been developing serious Java applications for quite a few years now, and while it was fun and enjoyable to discover the best practices, the misc. tools, how the messy fragmented ecosystem of frameworks and libraries  hardly wonderfully integrates thanks to amazing JEE-whatever integration stacks (Spring, no pun intended), you now feel that the platform has become pretty much boring, and you want to try something else..

Of course, you still love Java (it pays more) and think it is still the best way to write serious applications (currently waiting for Scala to get decent IDE support ?). You do not want to hear about this over-hyped language that is supposedly perfect because you simply don’t like languages that use half of the keyboard’s non-letter keys as metacharacters.  But you definitely want to hack a little bit using some dynamic language (maybe because you feel like an idiot when these dynamic-language lovers tell you they get a 10x productivity boost by using XXX instead of Java – replace XXX by whatever trendy, over-hyped popular language of the moment).

Anyways.. for some reason, your choice is Python (if not, then the rest of this post is of no interest to you). You have read a book or two online, and since you’re not an idiot, you already know how to code basic stuff (still need to lookup some stuff here and there, not sure of what is idiomatic yet, but you have definitely grasped the basic concepts). However, you feel a little bit alone in this new world, wondering what the best practices are, which tools are generally used.. etc.  And nobody on the internet really helps you because when it comes to giving technological advice, people are either of the “mine is bigger than yours” type, or the “everything depends on your needs/preference/[...]” BS.

So, here are a few things I learned while developping pymager, a RESTful image conversion/rescaling service (hopefully, it will help you to find your way):

• The equivalent of jars is eggs.
• The equivalent of maven dependency resolution/download system is easy_install or the newer pip that is even better.
• The equivalent of ibiblio main repository is Pypi.
• the equivalent of maven is either distutils or setuptools. distutils is the default tool shipped with python, and setuptools is an alternative, that is simply superior. This is what runs your unit tests and creates source/egg packages for you.
• Installing dependencies can be done using several ways : using easy_install / pip (pollutes your system), using your system package manager (e.g. debian/ubuntu apt-get :  super-clean, but does not install the most up to date packages), and easy_install / pip inside a virtualenv sandbox . See Tools for the modern Python hacker for some help regarding virtualenv.
• There are mainly 2 decent stacks for creating web applications : Django and TurboGears 2. Django is for people who like monolithic frameworks that reinvent the wheel, and TurboGears 2.0 is for people who favor integrating best-of-the-breed components. (TG2 gives you this integration for free, so you can see it as an equivalent of Spring ROO ).
• The equivalent of Hibernate is SQLAlchemy. In addition to what hibernate gives you, SQLAlchemy provides you with some lower-level utilities (such as SQL manipulation, DB-agnostic way to create a connection, ..). However, the transaction management is clearly inferior to what you get with Spring/Hibernate and their ThreadLocal implementation is just a hack that is clearly not suited to anything else than using from a web framework.  If you need to do anything more serious, you will need to reinvent the wheel (See pymager’s reimplementation of ThreadLocal’s transaction management ).
• There are a dozen ways to expose a python webapp including nasty CGI-related techniques. All of them are either hacks or legacy stuff except the newer WSGI approach. Most websites use fancy names for describing what WSGi does, but it is mostly an equivalent of the servlet API.
• Cherrypy is a wonderful embedded web server that supports WSGi (and that can be used as a WSGi application itself behind apache, pretty much like tomcat can serve applications behind mod_jk).  It used to be the one shipped with TurboGears, but they switched to Paste for some political reasons. (that was necessary for the merge with pylons)
• Naming conventions are a joke in python, as nobody seems to follow the same rules. Even some modules in the python standard library (e.g.: unittest module) seem to adopt different conventions than what looks like the python coding standard. I guess that too many ex-java developers program in python without being able to let away their java naming conventions..
• As a Java developer, there is some OO purity that you will need to forget about. It seems to be the “python way” to use module-wide variables, and you feel like you are fighting the platform / frameworks if you insist on applying your IoC best practices…
• nose is the way to go for running tests. (it integrates well with setuptools)
• there is no really manipulable classpath, and the default python mechanisms for discovering / handling files/data the equivalent of the classpath is pkg_resources.  (an additional reason to use setuptools)

Looks  like go is attracting some attention.

“”Open source does not mean anarchy. Somebody has to have a vision and the perseverance to see that through. The open source community can then create their own versions if they wish, but it is best if there is a main line, stable version with a consistent architecture with a guiding force behind it,” Gordon said.”

The Scala 2.8 beta 1 announcement gives hope regarding the availability of a decent IDE for editing Scala code. We will see what Scala 2.8 final looks like, but if the eclipse IDE support features basic Class and Method renaming, I will most likely make Scala my main programming language for writing open source code that targets the JVM. Two projects that I would most likely convert to Scala would be :

• Gisgraphy Java client : a Java library that gives access to gisgraphy City and GIS features search engine.
• Pymager Java client : a simple Java wrapper on top of the RESTful interface provided by pymager, an image service that provides simple conversion and thumbnailing / resizing features.

In many situations, open Source software developers need to deal with the maintenance of patches. Examples include :

• Unofficial versions of the linux kernel, where specific patches are applied (e.g. Xen kernel, openvz kernel, ..) and need to be constantly forward-ported to the latest kernel when it is released
• Distribution-specific changes (e.g. Ubuntu-specific changes to debian packages).

Maintaining one big diff file for all changes would clearly quickly become unmaintanable, so it looks like different approaches are now widely used instead :

• Maintaining stacks of patches, using specialized tools such as quilt.
• Using distributed VCS tools such as git.

How to survive with many patches describes the use of quilt. Here is some background :

“Andrew Morton originally developed a set of scripts for
maintaining kernel patches outside of any SCM tool. Others extended these into a suite called quilt.

It looks like distributed VCS tools have now superceded quilt, as far as pure software development is concerned (linux kernel, etc.). But quilt remains very popular for maintaining distribution-specific changes to packages. Indeed, as distribution packages live outside a SCM tree, it is important to have mechanisms to apply distribution-specific changes to the upstream source packages. And this is where quilt comes to the rescue.

Ubuntu packaging guide describes the use of quilt in debian’s packaging system. Such a simple system is clearly awesome, and the more I understand how the Open Source communities organize themselves, the more it makes me realize how technically advanced the Open Source world  is compared to the corporate world !

No matter how much you might have heard that tooling is unimportant, the reality is that tools are important. Tools enable complex collaboration, and this is clearly an area where Open Source excels.

Cumulative Flow Chart in Kanban attracted my attention as I consider it a nice example of using branching efficiently.

IMHO, it is simply wrong to assume that every single task can be split into small fragments that are then iteratively incorporated into the mainline. The author calls this kind of task a “technically complex story”, and I have yet to see a successful example of migrating frameworks or doing technical migrations like that without resorting to branching. This is what the whole open source community does, and it is high time the “enterprise” world catches with these practices.

Branching then comes to the rescue ! Let the “technically complex story” evolve in its own branch, and make sure to conduct in-depth QA BEFORE the merge. Same thing for code reviews and going through the DONE checklist : make sure to do it BEFORE the merge so that you do not end up with non production-quality code in the mainline, which is then pretty hard to get rid of.

BTW, Version Control Tools gives an overview of the differences between git and mercurial, which are two wonderful SCM tools that are very branch-friendly.

why can’t programmers…. program is definitely worth reading… Yes, this is disturbing….