The deeper current of Darwin’s thought

When people think of Charles Darwin, they first think of evolution.  But as Matt Ridley notes in this excellent The Spectator piece (here), there are currents of Darwin’s thought that flow on almost unrecognized:

Charles Darwin, who was born 200 years ago next month, has spent the 150 years since he published The Origin of Species fighting for the idea of common descent…  But in some ways it is less radical and topical than his other, more philosophical legacy: that order can generate itself, that the living world is a ‘bottom-up’ place.

As regular readers know, I’m skeptical of the way many technologists don the cloak of “innovator”, or “revolutionary”, or “pathfinder” (here and here).   The role of innovators is humbler in Darwin’s world:

Every technology has traceable ancestry; ‘to create is to recombine’ said the geneticist François Jacob. The first motor car was once described by the historian L.T.C. Rolt as ‘sired by the bicycle out of the horse carriage’. Just like living systems, technologies experience mutation (such as the invention of the spinning jenny), reproduction (the rapid mechanisation of the cotton industry as manufacturers copied each others’ machines), sex (Samuel Crompton’s combination of water frame and jenny to make a ‘mule’), competition (different designs competing in the early cotton mills), extinction (the spinning jenny was obsolete by 1800), and increasing complexity (modern cotton mills are electrified and computerised).

It’s a nice bit of irony that so many of these self-styled technology revolutionaries — who often believe their innovations had no precedents — are at heart believers in theories like spontaneous generation or intelligent design.  Not the company they would imagine themselves keeping, eh?

How much complexity theory can we apply in IT?

I’m fascinated by complexity theory and attempts to apply its insights to the software business.  If you’re interested in those topics you could do worse that to add two bloggers — Jurgen and Bas — to your newsreader (don’t forget about Crossderry). 

Both touch on complexity regularly (Jurgen’s latest here, Bas’s latest here) and they’re clearly big fans of the theory and its implications.  I agree there’s much that’s applicable, especially the concepts of iteration and feedback, which can even be used in “waterfall” approaches (here and here).  My academic background makes me especially sympathetic to the limits of central planning (start here re: Hayek).

That said, I’m not sure we can rely on self-organization for everything.  The most effective models of complex adaptive systems are derived from simple rules that generate complex phenomena.  This approach is mimicked effectively in agile, iterative, and other rapid development techniques (list of SW methodologies here).  Simple feature lists, regular interactions with stakeholders, short cycles, many versions of usable work product, etc. can generate feature-rich and useful applications.

However, the scalability and stability of these applications is often problematic.  IMO, this result is to be expected given the evolution of complexity among living things.  We like to point to complex creatures and structures — e.g., human brains — to support applications of complexity theory. 

But do we remember that most life is still very simple (about half of the biomass is microscopic)?  Also, aren’t complex creatures the ones that have had the spectacular denouements over the eons?  Betting on self-organization isn’t always a winning bet.  As I said, I instinctively like leveraging complexity concepts, but we must remember that they cut both ways.

%d bloggers like this: