Friday, September 01, 2006

 

The Swarm

When we learn about the giant software and the quick and dirty, we have only a few solutions left: the swarm and the silver bullet. Let me first talk about "The Swarm".

Quantity Then Quality

To avoid the quick hapahazard solutions and the overgrown software systems, we create tiny little pieces and then make them try to work together. In this solution, we use the quantity of the parts involved compared to the quality of the parts involved.

This means writing a lot of small solutions that tend to band together in a loosely coupled manner to tackle one problem. We see this being employed a lot in the scripting solutions in most Unix systems where you have self-contained tools that seem to work together to solve a problem. This seems like the best way of going about things, and is usually sufficient to solve the problem being faced.

The problem then becomes seeking control of the individual parts of the whole solution. In a bee swarm attacking a bear, the bees act as individual agents part of a whole cause -- stinging each at their own pace. In a locust swarm, it's first come first served: each locust takes his pick and sticks with it.

United We Stand

The good thing about this behavior is that you can lose one bee (in the swarm) and it's alright -- you easily have a couple dozen of them attacking anyway. That gives you more solutions to one problem, but not necessarily effective solutions to that single problem.

Another good thing about the solution (the swarm) is that you don't have to worry about making many different parts -- you just create one "bee" pattern and just launch an army of it to solve a particular problem. I'm not talking about a cluster or a replicated solution, I'm talking about one piece of software that you just use to solve *almost* the same problem (looking at tools like sed, awk, grep, etc.).

Divided We Fall

We see that alone, these tools are just that -- tools. Them alone cannot be as useful if you had an army of these tools at your disposal. Of course the scale of the tool says a lot about the scale of the problem that it tries to solve. To be useful (or more useful) you'll be using them along with another tool or set of tools (usually it's called a combo -- the flex-yacc combo, the grep-sed-awk combo, etc.).

You then run into a chaining problem -- which is like a dependency problem, but not really exactly the same as a dependency problem. Whenever you chain solutions, you introduce more and more points of failures. Instead of reducing the moving and breakable parts, you increase them by adding more tools and more chains.

Remember what they say about chains? "It's only as strong as the weakest link."

John, John, and... John?

Sometimes the tools also look so much alike, it's too hard to distinguish really how they differ from each other. Have you ever wondered by toolboxes contain too many tools in too many sizes and that you almost always only needed one or two of them? Sure, they've made screws "idiot proof" but they made a toolbox that's "genius required".

So how do we avoid the swarm problem?

Reuse

We can re-use one solution, and make a variation to it -- but make that variation part of the original solution as an "option". It's so much easier figuring options out than wading through a number of solutions that all almost did the same thing. I've counted at least two instances when I hoped "sed and awk" was just one tool and felt that PERL was becoming more and more attractive as the times went by.

Consolidate

If the solutions looked the same, make them one tool/solution with minor differences. You can make libraries, frameworks, and even whole applications out of the process of consolidation -- and it's not a bad thing to put together related information in one place. Just make sure that the solution you're writing won't require more people to maintain it than it actually took to write it.

Conserve

Whenever you feel like wasting effort, take time to see what's already out there. Before you write another line of code look at what's already available. After all, you don't want the quick and dirty to become a giant anytime soon.

Conclusion

The swarm is one of the best ways of actually solving problems -- break it down into small autonomous pieces, and make them work together to a common goal. Keeping each unit of the solution self-contained is a good thing especially when maintainance time comes, as well as when bug hunting season is in.

Other than avoiding making a swarm of giants or a quick and dirty swarm, there are only a couple of things to remember: keep each unit relevant and each unit should be justified. Units that stand alone are great, and making units work with each other to solve a greater deal is great(er).

Look at Service Oriented Architectures and Distributed Computing to see how a swarm can bring down even the biggest bears and bulls round.

More heads are better than one.

This page is powered by Blogger. Isn't yours?