as a shit umbrella, the product managers protect the engineers from getting distracted. It’s not enough to be a “shit funnel” where they would pass some of the junk down to engineers, they need to fully protect the engineers.

No wonder google engineers accomplish great things : they just focus on their work !

Even though it might be a good idea to always eat your own dog food, the simple fact that your employees prefer the available alternatives is worth taking into consideration. Indeed, the motivation behind eating your own dogfood is to get feedback ; and the simple fact that even your employees do not want to use the product is exactly the kind of feedback you are looking for.

So, if you are a software editor for a product, and for some reason, your own employees (or worse, the developers of the solution itself) want to use the competitor’s product, then the first thing you should do is welcome them to, and get back to them once the product becomes at least as good as the alternative. No matter whether the reason for not using the product is good or not, it just means there is something to address.

Grails is another example of a framework that fails to follow this rule (at least until v1.1, I cannot speak for v1.2).  I have way too often seen situations where the input was clearly incorrect (e.g. spring builder configuration) and grails was not reporting any error, and was injecting null values instead.

Note: this rule is part of the Art of unix programming, and thus part of the design principles I believe in (previous post).

If you have a (possibly good) product that you have open sourced, but without being able to gather a community around it, then I urge you to listen to the episode.

Tarus Balog, keep up the good work, you clearly have understood what a number of supposedly open-source companies totally miss  ! Good work pays in the long run !

BTW, if you want some help convincing people that git is better, do not hesitate to take a look at why git is better than X.

On a related note, you might want to listen to FLOSS weekly 111 podcast (direct link here), where CMake lead developer talks about his wonder build tool (I’ll never ever use autoconf again if I need to write some C any day) that is now developed using git.

With more and more evidence of systems/companies switching to NoSQL for scalability reasons, creating a rich  domain model becomes less and less of an option if you don’t want to shoot yourself in the foot. Indeed, while traditional applications sometimes often rely on the database to enforce integrity and referential constraints, this is no longer an option with NoSQL because of the CAP theorem.

So, this means enforcing constraints becomes the sole application’s responsibility, which is, IMHO, a good thing. Validation naturally belongs to the domain layer, and once you go through the trouble of transforming your POJOs/anaemic domain model into a rich domain model, you will certainly start adopting more and more DDD principles.

Responsibilities thus become clear : the storage layer handles the (possibly distributed) persistence, and the domain layer handles the domain-specific business rules and validation.

To give an example of its use, here is some sample code from gisgraphy-java-client ( a simple Java client I am writing for the open source GISgraphy project).  It calculates the orthodromic distance between two coordinates :

    public double distance(GisFeatureGeography o, Unit unit) {
           Unit targetUnit = (unit != null) ? unit : SI.METER;
           com.vividsolutions.jts.geom.Geometry me = location;
           com.vividsolutions.jts.geom.Geometry other = o.getLocation();
           try {
                   return  SI.METER
                          .getConverterTo(targetUnit)
                          .convert(JTS.orthodromicDistance(
                                 me.getCoordinate(),
                                 other.getCoordinate(),
                                 DefaultGeographicCRS.WGS84));
           } catch (TransformException e) {
                   throw new RuntimeException(e);
           }
   }

Please note that the code makes use of two excellent libraries : JTS for geographical types, and JScience for units. And for your information, WGS84 is a friendly name to refer to the GPS coordinate system (x,y,z).

“In traditional logic, an axiom or postulate is a proposition that is not proved or demonstrated but considered to be either self-evident, or subject to necessary decision. Therefore, its truth is taken for granted, and serves as a starting point for deducing and inferring other (theory dependent) truths.”

The rest of the mathematical logic is based on these axioms, and given these axioms, everything else can either be proven right or wrong. While it is certainly impossible to create universal axioms and reasoning principles that cover the foundation of software development, I believe we should at least mimic the approach : decide of which design, architectural and coding principles we believe on, and then use these axiomatic principles as the foundation for decision making. Of course, the outcome of your next project will depend on the  quality of these axioms, but at least you will be able to move forward and take consistent decisions throughout the lifecycle of the application.

As far as I am concerned, I tend to base my reasoning on a set of axiomatic principles that are based on the opinion of respected and talented people in the software industry. Even though nothing is perfect, I believe that listening to these experienced people will more likely lead to successes than listening to any lambda developer’s opinion. This is my bet, and the rest of this post is a first draft of the main design and architectural principles that I consider my axioms.

• Domain Driven Design, Specifications pattern, Layered design.
• Principles of OOD, and in particular, the S.O.L.I.D. principles
• Clean code: A Handbook of Agile Software craftmanship.
• Implementation Patterns
• Implementation of the right Enterprise Integration Patterns that best describes a given integration issue
• Defensive Programming (also see Defensive programming in Java) and its derived principles : Design by contract, Assertions,  Immutability, Copy-On-Write, Immutable collections.
• Inversion of Control
• Declarative Programming, and in particular Functional Programming-style collection manipulation and Command/Query separation, side-effect free methods.
• No premature optimization.
• Lessons learnt from UNIX programming.
• Tell don’t ask.
• Robustness principle : Be conservative in what you do, be liberal in what you accept from others.
• Do not reinvent the wheel.

The people behind these principles are smart, experienced, and potentially more intelligent than you and me together. So let’s just follow these principles for now, and once we master every single aspect of them, we will be able to help create the next generation of design and coding principles. In the meanwhile, I take these principles for granted.

• Wikipedia : Usability and Experience Study
• OpenStreetMap: UX Review
• any other ?

Usability reviews are of uttermost importance if you want to learn how your end users use your product. I initially thought that conducting usability reviews was complex, and involved lots of steps, but Jean-Francois Proulx definitely convinced me of the approach when I attended one of his usability talks a few months ago.

To me, this is clearly a better investment than having your market-team meet during long hours in front of screenshots trying to shape the next version of your website

Of course, you still love Java (it pays more) and think it is still the best way to write serious applications (currently waiting for Scala to get decent IDE support ?). You do not want to hear about this over-hyped language that is supposedly perfect because you simply don’t like languages that use half of the keyboard’s non-letter keys as metacharacters.  But you definitely want to hack a little bit using some dynamic language (maybe because you feel like an idiot when these dynamic-language lovers tell you they get a 10x productivity boost by using XXX instead of Java – replace XXX by whatever trendy, over-hyped popular language of the moment).

Anyways.. for some reason, your choice is Python (if not, then the rest of this post is of no interest to you). You have read a book or two online, and since you’re not an idiot, you already know how to code basic stuff (still need to lookup some stuff here and there, not sure of what is idiomatic yet, but you have definitely grasped the basic concepts). However, you feel a little bit alone in this new world, wondering what the best practices are, which tools are generally used.. etc.  And nobody on the internet really helps you because when it comes to giving technological advice, people are either of the “mine is bigger than yours” type, or the “everything depends on your needs/preference/[...]” BS.

So, here are a few things I learned while developping pymager, a RESTful image conversion/rescaling service (hopefully, it will help you to find your way):

• The equivalent of jars is eggs.
• The equivalent of maven dependency resolution/download system is easy_install or the newer pip that is even better.
• The equivalent of ibiblio main repository is Pypi.
• the equivalent of maven is either distutils or setuptools. distutils is the default tool shipped with python, and setuptools is an alternative, that is simply superior. This is what runs your unit tests and creates source/egg packages for you.
• Installing dependencies can be done using several ways : using easy_install / pip (pollutes your system), using your system package manager (e.g. debian/ubuntu apt-get :  super-clean, but does not install the most up to date packages), and easy_install / pip inside a virtualenv sandbox . See Tools for the modern Python hacker for some help regarding virtualenv.
• There are mainly 2 decent stacks for creating web applications : Django and TurboGears 2. Django is for people who like monolithic frameworks that reinvent the wheel, and TurboGears 2.0 is for people who favor integrating best-of-the-breed components. (TG2 gives you this integration for free, so you can see it as an equivalent of Spring ROO ).
• The equivalent of Hibernate is SQLAlchemy. In addition to what hibernate gives you, SQLAlchemy provides you with some lower-level utilities (such as SQL manipulation, DB-agnostic way to create a connection, ..). However, the transaction management is clearly inferior to what you get with Spring/Hibernate and their ThreadLocal implementation is just a hack that is clearly not suited to anything else than using from a web framework.  If you need to do anything more serious, you will need to reinvent the wheel (See pymager’s reimplementation of ThreadLocal’s transaction management ).
• There are a dozen ways to expose a python webapp including nasty CGI-related techniques. All of them are either hacks or legacy stuff except the newer WSGI approach. Most websites use fancy names for describing what WSGi does, but it is mostly an equivalent of the servlet API.
• Cherrypy is a wonderful embedded web server that supports WSGi (and that can be used as a WSGi application itself behind apache, pretty much like tomcat can serve applications behind mod_jk).  It used to be the one shipped with TurboGears, but they switched to Paste for some political reasons. (that was necessary for the merge with pylons)
• Naming conventions are a joke in python, as nobody seems to follow the same rules. Even some modules in the python standard library (e.g.: unittest module) seem to adopt different conventions than what looks like the python coding standard. I guess that too many ex-java developers program in python without being able to let away their java naming conventions..
• As a Java developer, there is some OO purity that you will need to forget about. It seems to be the “python way” to use module-wide variables, and you feel like you are fighting the platform / frameworks if you insist on applying your IoC best practices…
• nose is the way to go for running tests. (it integrates well with setuptools)
• there is no really manipulable classpath, and the default python mechanisms for discovering / handling files/data the equivalent of the classpath is pkg_resources.  (an additional reason to use setuptools)

Who wrote 2.6.33 reminds us how much the open source world (and in particular the linux kernel community) excels in this area. For the single 2.6.33 release that was developed in about 3 months :

“As of this writing, 10,500 non-merge commits have found their way into 2.6.33 – fairly normal by recent standards. These changes added almost 900,000 lines while deleting almost 520,000 others; as a result, the kernel grew by a mere 380,000 lines this time around.”

So, if you happen to struggle to scale your team past the 10-people mark using your usual development habits, then there are maybe a few things you could learn from the open source world.

My personal understanding of why it works so well :

• good elite developers
• top-notch, distributed, super-fast and merge-friendly version control tools (e.g. git)
• Fault-proof and compromise-free (though sometimes not politically-correct) ways of enforcing software quality and architecture. Examples showing the disagreement-proof nature of the kernel development process include last summer’s Alan Cox vs Linus Torvalds dispute regarding the tty subsystem, or Linus Torvalds vs Hans Reiser argument regarding Reiser4’s plugin system that does not fit well into linux architecture
• result-oriented and meritocracy-driven way of managing the project
• decentralized development (made possible thanks to distributed SCM tools). To quote Linus Torvalds : “Centralized _works_. It’s just *inferior*.“
• modular architecture supporting the collaboration of many developers.  “The large number of developers and the fact that they are volunteers has an impact on how the system should be architected. With such a large number of geographically dispersed developers, a tightly coupled system would be quite difficult to develop — developers would be constantly treading on each others code.“

You might disagree on the reasons why it works so well (after all, that’s just my analysis based on my understanding of the situation), but the success is a reality, a fact.

Also, if you are tempted to think that it could not work in the corporate world, please think about that twice by taking another look at who wrote 2.6.33, where contributing companies are listed.

“”Open source does not mean anarchy. Somebody has to have a vision and the perseverance to see that through. The open source community can then create their own versions if they wish, but it is best if there is a main line, stable version with a consistent architecture with a guiding force behind it,” Gordon said.”

• Gisgraphy Java client : a Java library that gives access to gisgraphy City and GIS features search engine.
• Pymager Java client : a simple Java wrapper on top of the RESTful interface provided by pymager, an image service that provides simple conversion and thumbnailing / resizing features.

• Unofficial versions of the linux kernel, where specific patches are applied (e.g. Xen kernel, openvz kernel, ..) and need to be constantly forward-ported to the latest kernel when it is released
• Distribution-specific changes (e.g. Ubuntu-specific changes to debian packages).

Maintaining one big diff file for all changes would clearly quickly become unmaintanable, so it looks like different approaches are now widely used instead :

• Maintaining stacks of patches, using specialized tools such as quilt.
• Using distributed VCS tools such as git.

How to survive with many patches describes the use of quilt. Here is some background :

“Andrew Morton originally developed a set of scripts for
maintaining kernel patches outside of any SCM tool. Others extended these into a suite called quilt.

It looks like distributed VCS tools have now superceded quilt, as far as pure software development is concerned (linux kernel, etc.). But quilt remains very popular for maintaining distribution-specific changes to packages. Indeed, as distribution packages live outside a SCM tree, it is important to have mechanisms to apply distribution-specific changes to the upstream source packages. And this is where quilt comes to the rescue.

Ubuntu packaging guide describes the use of quilt in debian’s packaging system. Such a simple system is clearly awesome, and the more I understand how the Open Source communities organize themselves, the more it makes me realize how technically advanced the Open Source world  is compared to the corporate world !

No matter how much you might have heard that tooling is unimportant, the reality is that tools are important. Tools enable complex collaboration, and this is clearly an area where Open Source excels.

IMHO, it is simply wrong to assume that every single task can be split into small fragments that are then iteratively incorporated into the mainline. The author calls this kind of task a “technically complex story”, and I have yet to see a successful example of migrating frameworks or doing technical migrations like that without resorting to branching. This is what the whole open source community does, and it is high time the “enterprise” world catches with these practices.

Branching then comes to the rescue ! Let the “technically complex story” evolve in its own branch, and make sure to conduct in-depth QA BEFORE the merge. Same thing for code reviews and going through the DONE checklist : make sure to do it BEFORE the merge so that you do not end up with non production-quality code in the mainline, which is then pretty hard to get rid of.

BTW, Version Control Tools gives an overview of the differences between git and mercurial, which are two wonderful SCM tools that are very branch-friendly.

Heh, and I’ll continue to be an ass in all future responses to “John Smith”s who tell talented guys who worked on my projects for years that they aren’t “team player”s, “have an attitude” and are “prima donna”s. I’ll be the judge of that, not some asshole anonymous blog comment poster who has never met or worked with Norm. I’m protective of my team. That’s not going to change. Sorry if you don’t like it.

YES, Gavin King is an ass, but most of the time, this category of ass is right, and people should just listen to them instead of complaining.. ah ah ah love it !

PS: for those who don’t already know it, Gavin King is the founder of Hibernate…

However, there are still a few things bugging me :

• Why insist on keeping the neat features (IoC, ..) server-side only ? Why can’t I just use the same mechanisms for in-container server-side code  and other kind of code ? Do I still need to revert to using Spring for everything that is not running inside the JEE6 container ? What about integrated tests ?
• How come we still don’t have any equivalent to Spring templates, that take care of creating standardized, runtime exceptions and handling opening/closing resources automatically ?

So for now, when working in non-spring environments, I need to create my own Templates to avoid creating clumsy code…

I agree that Spring Framework is a patch, and should eventually disappear. But for it to disappear, the underlying technologies need to start being half-decent…

These data storage systems have a number of features in common:
•   a call level interface (in contrast to a SQL binding)
•   fast indexes on large amounts of data,
•   ability to horizontally scale throughput over many servers, and
•   ability to dynamically define attributes or data schema.

NoSQL explained correctly gives an idea of what these datastores are useful for, and how they complement the current RDBMS offering.

High performance scalable datastores compares the technical  characteristics, maturity and licenses of the  NoSQL offering.

Other links on the subject that might be of interest :

• Twitter is switching to Cassandra
• Common principles and patterns for building scalable systems
• Seeking a database that does not suck

I take the official Oracle website to be rather … official ?

Lets recap, shall we ?

a) Almost every trace of OpenSolaris Support subscriptions vanished from
the official website within the last 14 days.

b) An Oracle sales rep informed me personally last week that I could no
longer purchase support subscriptions for OpenSolaris.

More information available on this thread.

Why is that important? Because each time you fix something in OSM your brain gives you a little drop of seratonin, or something, that makes you feel good. And that comes from a quick feedback loop. With traditional companies your feedback loop is far, far too long. Anything more than a couple of minutes is too long because there’s no association of the act of fixing something with the good feeling. More often than not, you help fix something and it drops in to a black hole.

[...]

The logistical costs of producing maps in a top-down fashion pretty much levelled out at having fleets of cars driving around North America and Europe. Increasing the detail with footpaths would cost far more than the benefit any company could leverage if they tried to do it a similar way. Crowd sourcing this from customers and others is simply inevitable – there is no other way to do it. But many will likely try to add quality assurance in between their contributors and the map and thus limit the scope and depth of the contributions.

[...]

Those maps will, at times and in certain areas, be better than OSM. But the growth of OSM is inexorable and like a freight train will ultimately roll over everything in its path. It may even happen before traditional companies figure this out and try to start seriously crowd sourcing data.

Is OSM the next wikipedia of mapping data ?

• Failure to Preserve Web Page Structure (‘Cross-site Scripting‘)
• Improper Sanitization of Special Elements used in an SQL Command (‘SQL Injection’)
• Cross-Site Request Forgery (CSRF)
• Reliance on untrusted input in a security decision
• …

What he says on git :

Git certainly seems to be liked for its power. Folks go ga-ga over it’s near-magical ability to do textual merges automatically and correctly, even in the face of file renames. I haven’t seen any objective tests comparing merge capabilities, but the subjective opinion favors git.

[...]

For many git’s biggest downside was its oft-cryptic commands and mental model. Ben Butler-Cole phrased it beautifully: “there is this amazingly powerful thing writhing around in there that will basically do everything I could possibly ask of it if only I knew how.” To its detractors, git lacks discoverability – the ability to gradual infer what it does from it’s apparent design. Git’s advocates say that much of this is because it uses a different mental model to other VCSs, so you have to do more unlearn your knowledge of VCS to appreciate git. Whatever the reason git seems to be attractive more to those who enjoy learning the internals while mercurial seems to appeal more to those who just want to do version control.

[..]

People generally find that git handles branching better than Mercurial, particular for short-lived branches for experimentation and check-pointing. Mercurial encourages other mechanisms, such as fast cloning of separate repository directories and queue patching, but git’s branching is a simpler and better model.

I especially love the following part :

right now a feature is considered not leaking if there is average increase of less then ~5KB per iteration

By continually getting feedback and improving pointed weaknesses, the end result is that you get better over the time

Simple, isn’t it ? So simple that it is clearly understandable and unchallengeable. Of course it is true, and those who do not think so should just burn in hell ! Or.. shouldn’t they ?

360 evaluations are not tainted by political issues and personal preferences !

Let’s take a similar different example: politics. Theoretically, thanks to the power of voting, parties should continually get better to be (re-)elected. Hum ! Who feels like political parties focus on the real issues ? Does it have to do with the fact that real issues are unpopular and unlikely to get them re-elected ? Here is a short excerpt from Les Cowboys Fringuants – 8 secondes :

Mais l’monde oublie vite c’est pas grave
Suffit de faire un bon budget
D’parler d’santé pour que les caves
Vous réélisent l’année d’après

But 360 evaluations do not have anything to do with politics, really !

The same way parties shift their focus from the real,  unpopular issues, I believe that listening too much to 360 feedback will just make you focus on the wrong things. Either you are too X, or not enough Y.  Either you’re too concrete or too abstract. Either you have too much self-confidence, or need to higher your self-esteem. Either you do not show enough leadership, or your leadership does not leave enough room for others. Oh, and sometimes, something is both in the “strengths” and “weaknesses” columns. wao !

It truly makes me laugh when I realize that my collegue’s evaluation strengths include both :

• “Less jokes in the messages” [understand: you got better since last 360 evaluation by reducing the amount of jokes in your messages]
• “Great sense of humour”

Hum ! hum ! So.. what is going on exactly ? Should he continue to do jokes and develop his sense of humor, or should he stop doing so ? .. It might simply be that different people perceive things differently, so the only true way to accomplish a perfect, no-fault, 360 evaluation is to be nothing more than a moderate lemming that has no real strengths and no weaknesses. 101 guide to having a good 360 evaluation :

• Take a few un-risky initiatives to show that you do have some leadership, but make sure you do not address the risky, unpopular issues !
• Criticize things moderately and constructively, but make sure you only criticize things that others would criticize too.
• …  [hint: do not address the real issues, as those are the most risky, address the popular ones)

If you have real strengths, then they usually come with their set of weaknesses ! Not everybody might be happy about them, but in my humble opinion, erasing them would definitely not make a better company in the end.

What if the problem was the unability of people to understand the situation correctly ?

All truth passes through three stages. First, it is ridiculed. Second, it is violently opposed. Third, it is accepted as being self-evident.

Arthur Schopenhauer
German philosopher (1788 – 1860)

So, what do we do ?

Always make sure to bring this question when you raise an issue. Even if you do not have any answer, this shows your willingness to contribute positively instead of just openly criticizing

• If you are an absolute TDD-lover, think that Ruby is awesome in every aspect, that RSpec is THE WAY to go, agree with every single principle of TDD, never write a single line of untested code and believe that  sex cannot be good unless you have written a test case to explicit the conditions of success before going to bed, then jMock is the tool for you.
• If you’re a duct-tape programmer, think that Joel Spolsky rocks, and favor pragmatism over purism, then Mockito is the right tool for you. Mockito is very similar to EasyMock but is just newer and simpler to use.
• For everything else, there is Mastercard.

Anyone interested in wage transparency and paying value for value ?

delete unused local branches

git branch -a

git branch -D unused_local_branch

delete unused remote branches

git push origin :unused_remote_branch

delete unused remote-tracking branches

If you use a shared-repository approach (several people have commit access to the same remote git repository), it will happen that someone else deletes a branch on the remote repository. Git does not automatically delete your local remote-tracking branches. To do so, it is necessary to :

git remote prune origin

clean-up unreachable commit objects

git prune

efficiently pack the repository objects

git gc –aggressive

That’s it

• The Specification pattern. This pattern can be used in a wide range of situations, from abstracting domain constraints to making complex data driven tests more readable thanks to the “outsourcing” of the test data creational logic out of the testcase.
• Constraint Oriented Programming
• DSLs can be used to declaratively specify what you’re interested in, rather than how you want to achieve it
• SQL, XPath, XSLT are all declarative programming languages that focus on the intent

As a result, he covers hiring, hardware, salaries, skills, etc… really, really interesting  !

A little excerpt :

This reason is related to #5 in the sense that most software shops don’t have enough good people to take responsibility for the truckloads of work that they’re expected to do. You can’t put idiots in charge of important projects, therefore pair programming requires double the amount of “good” people as not pair programming. 

So… come on with your metrics, guys! they’re probably never going to match the “WTF/min” metric of a human-being senior developer.

The conclusion of the post is that if you insist too much and try to convince others that your way of doing something [in this case, agile] is superior to their way of doing the same thing, you end up being seen as a freak. So his conclusion is to let people “learn from themselves”. So, in this scenario, I guess darwinism will then help determine the best alternative, as the inferior solutions will eventually die. awesome !

Now, let’s say you have some kind of interest in the success of the projects (involved in the project, somehow)  using an “inferior” solution that you consider threatening for the project.  What are your options ? The ones I see are :

• Debate forever in the hope that the others will rally in favor of your solution. In his article, Grant Joung seems to think it’s just making things worse, and I can understand that religious debates do not lead anywhere, as nobody is likely to rally (have you ever seen a muslim convert to Judaism just because someone made a point against  islam during a debate ??)
• If a few others also support your solution, and depending on whether it’s worth it, you can always attempt a putch on the project and just let the unsatisfied people go away, and end up dealing with those that agree with you
• Quit (which ends up being the same solution as Grant Joung suggests). You definitely do not want to be part of something that you know will fail

Hmm.. It looks like none of the options I see is ideal… Does anyone have a suggestion ?

This is the first article I read on this subject that is actually giving strong, valid points. I totally agree!

Relying on the technical merits of whatever platform is mostly religion. However, considering the surrounding community and environment totally makes sense. If you pick PHP as a language, no matter what you think of it from a technical point of view, you’re going to have to deal with (or even hire) PHP developers, who have been deeply affected by the hacker syndrome. If you pick the .NET platform, you’re going to work with religious people who do not consider anything which is not an official Microsoft BestPractice. If you pick the Java platform, you’re going to deal with framework-ill people who focus more on the infrastructure than the target application. and so on….
Everything is about choice!  So, just consider this when you start your next project, I feel like it’s as important as other technical considerations..

• Be simple to implement (or at least, be simple for simple cases)
• Be machine-parseable (so that intelligent REST clients can be built)
• Be human-readable (so you don’t have to fire up your HTTP sniffer to understand what went wront, when you’re developping a client)
• Follows HTTP semantics, so proxies, caching, etc continues to work correctly

None of the ideas suggested in the article match all of my criteria, so here is the solution I would suggest :

• ALWAYS return the most appropriate HTTP error code whenever an application error happens. Sure, it might not map perfectly, but choose the one that expresses the concept correctly. It is totally absurd to always use 200 error codes, even the SOAP guys have not made that mistake ! Also make sure to use a server-side REST framework that will automatically handle common errors for you (405 for unsupported methods, etc..)
• When the same HTTP error code maps to several application errors, add an ADDITIONAL X-Application-Error-Code : header that provides a simple string allowing to remove ambiguity (choose a simple all-ASCII exception name). This will allow creating simple clients that can easily parse exceptions without too much burden for simple cases
• Add a body that allows a human being (for instance, the developer of a REST client for your service) to instantly understand what went wrong. This can take the form of either a free-form response, or a fully-parseable (XML, JSON, whatever) response body that contains a message field.
• Only implement the fully-parseable response body when your application has matured. Do it smartly, avoid working weeks on your service before even displaying the first feature to your user

I wish I would explain my super-great-idea about achieving this goal, but as usual, Martin Fowler & Eric Evans have thought about it before me, so here’s just the link to the wonderful pattern called Specification.  You could also buy the Domain Driven Design book, as Evans speaks about it in his book.

First of all, dispatching calls to different services based on runtime conditions is something that is needed all the time. The example explained on the code reef might not speak to everybody, though. Example scenarios of every-day use include :

• You need to change the behavior of a payment system depending on the environment. Obviously, you cannot use the real bank service for development, so you need to switch between a DevelopmentPaymentFakeSystem and a RealWorldBankPaymentSystem implementation depending on a runtime condition (the environment on which the application is deployed)
• You need to activate debug features (IE browser check pass-through in development mode, etc)

Obviously, every programming problem can be solved by adding spaghetti code that adds a few conditions here and there in your real services. But the rest of this post assumes that for testability and maintainability reasons, you are implementing the features as different services : the fake payment system, and the real world payment system

The idea suggested by the aforementionned blog post is to use a front-service that will deleguate to the right implementation (real vs fake) based on a runtime condition (deployment). Even though the idea of doing that is brilliant,  I would tend to think that the provided implementation is assuming a broken, half-implemented IoC container.

Indeed, by using a framework such as Spring :

• The Spring container can directly be used as a service locator (just implement ApplicationContextAware, and spring will inject you the service locator). So, no need to create your own implementation of the container (you could always implement ApplicationContextAware later if you need to switch to a different container, as this is just an interface)
• There is actually no need to use the IoC container as a service locator: the dispatcher service can be declared in the IoC container and one can inject it the different implementations using the IoC mechanisms. This can be accomplished using several techniques : name-based wiring (or the @Autowired annotation coupled with @Qualifier), tag-based wiring (you can now tag implementations based on some logical semantic, and then use @Autowired complemented with @Qualifier to get the right instance), or you can just have all implementations of a given interface directly injected to you so you can do your runtime logic to determine the right object (once again, the magical  @Autowired annotation is your friend, just declare an array of the given type and @Autowired it! )

Sure, there are advantages of abstracting the container, but sping provides so much goodness that it’s pretty sad to avoid making use of it, or mirroring its features in custom code !

For the sake of explaining my point, let’s consider two types of developers :

• Joe the programmer, who doesn’t care about code quality
• Scott, the more advanced developer, who works as hard as possible to produce good code quality

It’s pretty easy to realize that Joe doesn’t care about agility, scrum, and whatever, as he just works on whatever his hierarchy wants him to work on. Give him scrum, give him waterfall, he just won’t see a difference anyways. Code is about copy-pasting, and google is your friend !

Now, let’s see the case of Scott, the “elite” developer. He wants to focus on code. He likes code, he dreams about code, he wants to refactor, and he will do anything that allows him to produce better code. And usually, he just _hates_ politics. When Scott first heard of agility, what he thought was “yeahhhhhhh great!!! FINALLY something that allows me to just focus on my code”. yeah.. that was a few years ago.

That was true, at least for some time, because agility used to be about good developers trying to find a way to just work.

But now, things have changed. Agile is nearly mainstream, and most developers who claim they do agile development are just instances of Joe() who learnt how to create crappy, unmaintainable tests. The good developers have already moved to the latest bleeding edge methodology which is called “Common Sense Politic-Less Programming (CSPLP)“.  That’s what allows them to get away from the religious debates brought by the scrum and agile masses. (they prefer technical religious debates.. but that’s another problem.. )
Once again, what they want is to produce code, not  debate forever about “how long should a daily scrum last”, “what kind of clown tricks could we do to build a better team”, or “how to teach the monkeys how to produce good code without affecting their ego “. Have you heard of google developers applying Scrum ? Have you heard of Linus torvalds bringing agile to improve Linux ? All of them just don’t care, because they already produce good, working software, that they release iteratively. And they did not need agile for that, they had their skills, and their common sense. It’s the developers who are producing the code, not the methodology ! And the successful companies have understood one thing : “if you hire the good guys, you’re gonna make great things”. It’s as simple as that.
To put it in a nutshell, nobody cares about agile and scrum. While the average joe doesn’t care or hasn’t even heard about it, the elite scott is skilled-enough to apply the CSPLP methodology, which stands for Common Sense Politics-Less Programming.

Teams can take advantage of having small iterations followed by retrospectives in order to progressively get better, smarter, and learn from their errors.

And theoretically eventually, the team will get to a point where its members can efficiently work together, where every team member can produce high quality and maintainable code, where the sky is perfectly blue, and the sun is shining.

Yeah.. eventually, we will live in a perfect world…

However, there is something that bugs me with this approach: by letting the team “auto-organize itself to discover its own solutions to its highly specific issues”, we actually end up having the team re-invent the wheel to solve the always-same problems that every development team faces. Yes, guess what ? At some point, sometime in the future, the team will discover that it needs to write decent OO code, that it needs to have a decent coverage of carefully written tests or executable specifications (TDD, BDD), that the domain needs to be properly externalized from the technical code (DDD), that it needs decent modelling skills, that IoC enhances the testability of their code,that it needs to decouple the code with small modules and services, that junior developers need to be coached by senior ones, and so on…. Or.. the team won’t discover it, because the project will lack money at some point, and everything will need to start from scratch again.. Wao! so productive !

So, the question is.. What can we do to reduce the duration of the learning curve ? How can we make sure new projects can benefit from the experience of previous ones ? That’s exactly the debate we have had with a few guys from Pyxis Technologies (http://www.pyxis-tech.ca).

When dealing with organizational coaching (Scrum, Agile), the idea of imposing engineering practices (XP, DDD, ..) was totally rejected by a few people that had concrete examples of failures. Most developers have a great ego and still haven’t learned enough of programming to just admit they can learn more from others.

So, what are the other ideas that were thrown ? Everyone seemed to agree that most of the problem lies in the beginning of the contract: every Scrum/development contract should start with an initial diagnostic, that states the technical debt accumulated by the team. This can then serve as the basis of an execution plan. Most of the challenge lies in linking the technical recommendations with business objectives, and clearly explaining the value of these engineering practices to the product owner and stake holders.
Once the product owner (PO) agrees with the values and the vision, most of the job is done. The team is supposed to work on the PO’s priorities, and the technical coaching will be seen as a wonderful help to meet the business objectives.

Of course, in order to achieve this, a few things will need to be sorted out such as compiling the list of the best practices that we embrace. But nothing impossible here, I guess.

Is that enough to learn faster ? probabably. Is that enough to learn wayy faster ? probably not ! We will still see crappy code being produced for months (years ?), and projects failing because of that. So, what can we do to DRASTICALLY improve the quality of code that is produced by those teams ?

ACEGI

1. If you use composition, your domain objects don’t have to implement the UserDetails interface, so you don’t have to “pollute” them.

1′. Even if you do not use composition, implementing an interface doesn’t  mean your domain objects are not POJOs anymore. However, having to inherit some base class makes them not POJO, which is not the case with Acegi.

2. There is no concrete provider, but once again, if you wrap your AccountDao (or whatever JPA/Hibernate Dao ), implementing the provider takes 2 or 3 lines of code….

3.  Yes, indeed the documentation is not easy to deal with…

APPFUSE

1. Annotations vs XML debate… I guess it’s a matter of taste.. XML is definitely more powerful, etc, but you don’t generally need this power, and ease of maintenance can be considered as more important that theoretical flexibility…

2. I agree.

For instance… Let’s say you want to display AJAX-pages that will query the server asynchronously for some stuff, and display the result to the user. One of the best alternatives is DWR, so let’s say you use version 2, because it supports annotations, etc..

Now, you want to create AJAX-validation of your forms. You don’t want to reinvent the wheel, so you use something like Struts2, with the ajax theme. Bad luck ! Struts2 seems to work fine with DWR1, but one will have problems with DWR2… How do you solve the issue ? Patching, patching, patching, if you have the time.

The problem is that this kind of headaches constantly happens whenever you use a few frameworks and libraries, which completly kills the productivity… Another real headache  would be the use of Spring 2 + AspectJ + Hibernate to Dependency-inject POJOs… The problem + solution is described on this post…

Another example I came accross would be the incompatibility between some prior ActiveMQ  version (before 4.x was released) and Spring 2.0, whereas everything was working fine in 2.0 RC4… The problem came from Xbean 2.6 which was incompatible with Spring 2.0 final….

And I could post hundreds of framework-headaches … I am not saying frameworks and libraries are bad (otherwise, I wouldn’t use them), but they’re not paradise either…

However, I think we have a deeper problem than just the lack of maven2 package..  Just take a look at the number of jars in Maven2 repository. And this doesn’t take into consideration the jars offered on Java.net repository, or Apache Incubating one.

So, what is the problem exactly ? Well, it is going to be IMPOSSIBLE for Debian to keep up with the crazy and constantly increasing amount of Java jars available. So, instead of trying to re-package Java libraries once again (mainstream developers already have to package them for maven1 and maven2), why not work on some integration between Debian Packages and Maven2 ones ? And when you think a bit about it, Debian has the same problem with PHP PEAR and and Perl CPAN.

Maven2 is a full-blown java package system that expresses dependencies, etc. So, to my opinion, what Debian should do is the following :

• Create a notion of a “soft-depencency”. Basically, this  would mean that any Package can have hard dependencies (the current deb dependencies), and soft dependencies which could be resolved by a soft-dependency subsystem.
• In order to deal with Java, CPAN and PEAR’s case, a soft-dependency could be expressed as prefix:dependency. A soft-dependency manager could register a prefix, and the dependency string would be subsystem-specific.  So, for instance, if I need freemarker, the dependency could be maven2:freemarker/freemarker/2.3.8

Now, the question is whether Debian is ready to accept such a shift in its concept ?

width="200" height="20"
hasText="true"
progressValue="7"
maxProgressValue="10" id="testBar" showOnlyIntegers="true" />

Now, you want to animate it so that it progressively reaches its progressValue. Something like the following will be needed :

function animateProgressBar(progressBarName, targetValue anim) {
dojo.event.connect(
anim,
“onAnimate”,
function(e) {
var bar = dojo.widget.byId(progressBarName);
bar.setProgressValue(targetValue * e.x / 100);
}
);
}

dojo.addOnLoad(
function() {

var anim = new dojo.animation.Animation(
new dojo.math.curves.Line([0], [100])
, 2000);

animateProgressBar(
“stressLevelProgressBar”,
7 ,
anim
)

anim.play();
}
);

And it should be working…

We can probably draw a parallel with the funny quotes about Java I previously cited.

My opinion is that every Java developer is aware that everything he does is far more complex than it should be. However, abstraction is the key foundation of Computer Science, and most people agree that C is a good abstraction over ASM and Java/.Net are good abstractions over C. So, what is the limit of abstraction ? Are frameworks, Design Patterns, Factories and new abstractions all over the place a good thing ?

What is sure is that people who start using frameworks and libraries use even more of them. So, even if it is not a perfect solution, there is something great in that.

Something important for the web development community in this release is the  support for scripting languages. In fact, the trend, that has been started with Java Hotswap, is to create tools that allow Web Frameworks to be more productive for developers.

Indeed, developers are tired of the develop / build / deploy cycle, and would prefer to work with the PHP-like develop / reload in browser development cycle. Well, scripting is a step in that direction, and would allow to :

• Keep the domain layer as strongly-typed POJOs, that are unit-tested and constantly refactored to keep a good design.
• Write the Web Controllers (the C in MVC) in any scripting language (the view being written in any already-existing templating language, such as JSP or Freemarker). This allows rapid development and since the view is hardly reusable anyways, it’s better to throw it away and recreate something from scratch when needed.

Efforts toward that direction have already been started, as relayed by these few blog entries and articles :

• Christian groovin’ with webwork 2
• Groovin’ with Webwork 2
• Groovin’ with Webwork 2 : using Groovy as actions

So, now the next step is to have universal support for that kind of development, and have better IDE support that will allow refactoring both in the scripted controllers and the view (JSP, Freemarker..). In fact, there is currently no way to make sure nothing is broken besides writing functional (Selenium, Canoo Webtest, HttpUnit…) tests.

In any case, I am curious to see one or two examples of these Meta Patterns to truly understand his thoughts.