Posted on LJ in April 2006, not edited.
Developing software is most like painting a picture.
You start with a vague idea of what you want to accomplish and roughly how to begin. But your mind’s eye can’t really see the whole picture with any accuracy of form, or the details of the picture with any precision. Thus the picture emerges as you paint. At each point during the work, what is already on the canvas lets you anchor your imagination and see a little bit further. As you try to paint this you encounter an imperfection or an aspect of reality, and have to adjust, until you have converged, by this cycle of imagination and recording, onto a picture that is whole enough.
If you look for a more methodical approach, you may find yourself receiving the advice of professional illustrators or graphic designers, because they are the ones with technical advice to give. They will generally suggest you “design” your painting and imply with this a regime where you can define the salient features and hand over, as it were, the completion to unskilled apprentices. It will be surprising if this yields any great work. There is in fact a form of design you can apply, but it’s different. It is to prototype the whole or the details of the painting, possibly in a more casual medium, until you have clarity in your mind about what you should do. Then you execute the real painting over the most basic pencil design. You do, however, put down the forms in a certain temporal order that you have learnt works.
Now, your customer, if you have one, will tend to have an opinion about what you paint. You can reach a clear enough agreement about what sort of thing you should paint, like a nude, or a still life, or a portrait, and a fabulously precise agreement about the base technologies you should use (oils, canvas, dimensions). The customer may even try to specify some gross stylistic aspects of the work, like it should be Impressionist, or avoid large patches of green. The thing is, all this agreement will do little to ensure the customer is pleased. The customer will generally be pleased if they like your work and not otherwise. So, gradually, what you sell becomes your reputation rather than any one painting.
You will paint the same painting several times. Your customers will not see it that way of course (they each get a unique work) and at the same time they will (I want a painting like his, but better). There is nothing wrong or even particularly boring with this. You will want to paint the same kind of thing again but do your mistakes right, or at least differently. Each time the parts you’ve executed before will be trivially easy, and you’ll learn something over the difficult and new parts. When you stop learning, it’s time to end the series and start something fresh.
Your manager, if you have one of these, will expect you to produce the painting by filling the canvas at a regular rate, like left to right and top to bottom, as if you were a mechanical printer. They will find this thought appealing because it lets them measure the filled and unfilled portion of the canvas and calculate whether progress is according to schedule. Your manager will generally express dismay when you report that 98% of the canvas is filled, but none of the main features of the painting are really in place, and anyway you need to redo the background because it just doesn’t hang together. You will argue vainly with your manager that a good painting is released when it’s ready.
At first, nobody will like your paintings. They will indeed be awful, but even after this stage nobody will like your good paintings for a while. Then a few people will believe in your paintings and declare to others that they are good. Suddenly (if you’re lucky) lots of people you didn’t expect will start wanting your paintings, but many of them will make remarks that indicate they like them for misunderstood or superficial reasons, and that will be kind of tiresome. Anyway, your paintings will soon become dated and forgotten, unless you’re one of the lucky few and they become cultural artefacts.