Friday, September 01, 2006
The Swarm
When we learn about the giant software and the quick and dirty, we have only a few solutions left: the swarm and the silver bullet. Let me first talk about "The Swarm".
Quantity Then Quality
To avoid the quick hapahazard solutions and the overgrown software systems, we create tiny little pieces and then make them try to work together. In this solution, we use the quantity of the parts involved compared to the quality of the parts involved.
This means writing a lot of small solutions that tend to band together in a loosely coupled manner to tackle one problem. We see this being employed a lot in the scripting solutions in most Unix systems where you have self-contained tools that seem to work together to solve a problem. This seems like the best way of going about things, and is usually sufficient to solve the problem being faced.
The problem then becomes seeking control of the individual parts of the whole solution. In a bee swarm attacking a bear, the bees act as individual agents part of a whole cause -- stinging each at their own pace. In a locust swarm, it's first come first served: each locust takes his pick and sticks with it.
United We Stand
The good thing about this behavior is that you can lose one bee (in the swarm) and it's alright -- you easily have a couple dozen of them attacking anyway. That gives you more solutions to one problem, but not necessarily effective solutions to that single problem.
Another good thing about the solution (the swarm) is that you don't have to worry about making many different parts -- you just create one "bee" pattern and just launch an army of it to solve a particular problem. I'm not talking about a cluster or a replicated solution, I'm talking about one piece of software that you just use to solve *almost* the same problem (looking at tools like sed, awk, grep, etc.).
Divided We Fall
We see that alone, these tools are just that -- tools. Them alone cannot be as useful if you had an army of these tools at your disposal. Of course the scale of the tool says a lot about the scale of the problem that it tries to solve. To be useful (or more useful) you'll be using them along with another tool or set of tools (usually it's called a combo -- the flex-yacc combo, the grep-sed-awk combo, etc.).
You then run into a chaining problem -- which is like a dependency problem, but not really exactly the same as a dependency problem. Whenever you chain solutions, you introduce more and more points of failures. Instead of reducing the moving and breakable parts, you increase them by adding more tools and more chains.
Remember what they say about chains? "It's only as strong as the weakest link."
John, John, and... John?
Sometimes the tools also look so much alike, it's too hard to distinguish really how they differ from each other. Have you ever wondered by toolboxes contain too many tools in too many sizes and that you almost always only needed one or two of them? Sure, they've made screws "idiot proof" but they made a toolbox that's "genius required".
So how do we avoid the swarm problem?
Reuse
We can re-use one solution, and make a variation to it -- but make that variation part of the original solution as an "option". It's so much easier figuring options out than wading through a number of solutions that all almost did the same thing. I've counted at least two instances when I hoped "sed and awk" was just one tool and felt that PERL was becoming more and more attractive as the times went by.
Consolidate
If the solutions looked the same, make them one tool/solution with minor differences. You can make libraries, frameworks, and even whole applications out of the process of consolidation -- and it's not a bad thing to put together related information in one place. Just make sure that the solution you're writing won't require more people to maintain it than it actually took to write it.
Conserve
Whenever you feel like wasting effort, take time to see what's already out there. Before you write another line of code look at what's already available. After all, you don't want the quick and dirty to become a giant anytime soon.
Conclusion
The swarm is one of the best ways of actually solving problems -- break it down into small autonomous pieces, and make them work together to a common goal. Keeping each unit of the solution self-contained is a good thing especially when maintainance time comes, as well as when bug hunting season is in.
Other than avoiding making a swarm of giants or a quick and dirty swarm, there are only a couple of things to remember: keep each unit relevant and each unit should be justified. Units that stand alone are great, and making units work with each other to solve a greater deal is great(er).
Look at Service Oriented Architectures and Distributed Computing to see how a swarm can bring down even the biggest bears and bulls round.
More heads are better than one.
Quantity Then Quality
To avoid the quick hapahazard solutions and the overgrown software systems, we create tiny little pieces and then make them try to work together. In this solution, we use the quantity of the parts involved compared to the quality of the parts involved.
This means writing a lot of small solutions that tend to band together in a loosely coupled manner to tackle one problem. We see this being employed a lot in the scripting solutions in most Unix systems where you have self-contained tools that seem to work together to solve a problem. This seems like the best way of going about things, and is usually sufficient to solve the problem being faced.
The problem then becomes seeking control of the individual parts of the whole solution. In a bee swarm attacking a bear, the bees act as individual agents part of a whole cause -- stinging each at their own pace. In a locust swarm, it's first come first served: each locust takes his pick and sticks with it.
United We Stand
The good thing about this behavior is that you can lose one bee (in the swarm) and it's alright -- you easily have a couple dozen of them attacking anyway. That gives you more solutions to one problem, but not necessarily effective solutions to that single problem.
Another good thing about the solution (the swarm) is that you don't have to worry about making many different parts -- you just create one "bee" pattern and just launch an army of it to solve a particular problem. I'm not talking about a cluster or a replicated solution, I'm talking about one piece of software that you just use to solve *almost* the same problem (looking at tools like sed, awk, grep, etc.).
Divided We Fall
We see that alone, these tools are just that -- tools. Them alone cannot be as useful if you had an army of these tools at your disposal. Of course the scale of the tool says a lot about the scale of the problem that it tries to solve. To be useful (or more useful) you'll be using them along with another tool or set of tools (usually it's called a combo -- the flex-yacc combo, the grep-sed-awk combo, etc.).
You then run into a chaining problem -- which is like a dependency problem, but not really exactly the same as a dependency problem. Whenever you chain solutions, you introduce more and more points of failures. Instead of reducing the moving and breakable parts, you increase them by adding more tools and more chains.
Remember what they say about chains? "It's only as strong as the weakest link."
John, John, and... John?
Sometimes the tools also look so much alike, it's too hard to distinguish really how they differ from each other. Have you ever wondered by toolboxes contain too many tools in too many sizes and that you almost always only needed one or two of them? Sure, they've made screws "idiot proof" but they made a toolbox that's "genius required".
So how do we avoid the swarm problem?
Reuse
We can re-use one solution, and make a variation to it -- but make that variation part of the original solution as an "option". It's so much easier figuring options out than wading through a number of solutions that all almost did the same thing. I've counted at least two instances when I hoped "sed and awk" was just one tool and felt that PERL was becoming more and more attractive as the times went by.
Consolidate
If the solutions looked the same, make them one tool/solution with minor differences. You can make libraries, frameworks, and even whole applications out of the process of consolidation -- and it's not a bad thing to put together related information in one place. Just make sure that the solution you're writing won't require more people to maintain it than it actually took to write it.
Conserve
Whenever you feel like wasting effort, take time to see what's already out there. Before you write another line of code look at what's already available. After all, you don't want the quick and dirty to become a giant anytime soon.
Conclusion
The swarm is one of the best ways of actually solving problems -- break it down into small autonomous pieces, and make them work together to a common goal. Keeping each unit of the solution self-contained is a good thing especially when maintainance time comes, as well as when bug hunting season is in.
Other than avoiding making a swarm of giants or a quick and dirty swarm, there are only a couple of things to remember: keep each unit relevant and each unit should be justified. Units that stand alone are great, and making units work with each other to solve a greater deal is great(er).
Look at Service Oriented Architectures and Distributed Computing to see how a swarm can bring down even the biggest bears and bulls round.
More heads are better than one.
Sunday, August 13, 2006
The Quick and The Dirty
Experienced software engineers and designers usually already have a bag of tricks to pick out solutions from -- ranging from the most trivial routines to the most complex designs. After the countless years and projects that they have worked on, the skills they've learned, and the solutions they've deployed, they usually are able to distill the learnings into easy to recall and easy to implement techniques.
Some software projects (see Evolving a Giant) can benefit from these experienced engineers and designers -- but then the same forces encountered in these situations applied to even the most experienced engineers and designers can lead to another type of problem. This problem is something I like to call "The Quick and The Dirty".
The Quick Solution
Experience is the best teacher, and lessons learned through experience are hard to forget. To avoid complex solutions, a lot of engineers have found that if you avoid over-engineering a solution, you avoid a lot of the complexities associated with that solution. If they kept it simple and it works, they get the job done quicker than if they spent a whole lot of time over-engineering it.
The quick solution is usually effective -- it solves the problems, and creates less problems that need to be faced at the moment. The quick solution is usually sufficient especially if the enhancement/addition to the software is trivial and can be addressed with tricks from the engineer's bag. Of course, this assumes that the "solution" works, otherwise it's not a solution.
The Dirty Solution
Another thing experience brings is confidence. Usually, the more experienced engineers are more confident with their solutions compared to the solutions of those with considerably less experience or exposure. That leads to problems especially if you have a team composed of experienced and not so experienced engineers.
The dirty solution comes from both the experienced and the not so experienced engineers -- the solution becomes dirty if it needs to get cleaned up even after the solution has been deployed. Experienced developers have gone through the nightmare of maintaining their own dirty code, and the dirty code of others. This leads to the same problems that are encountered when working with a humongous software project that has already grown out of proportion.
Dirty doesn't necessarily mean not-elegant -- on the contrary, a lot of the "elegant" solutions are so complex they end up being more dirty than the quick solution (for examples, look at the merge sort and quick sort algorithm and compare these to insertion sort and even bubble sort). Dirty solutions have any one or all of the following properties (please feel free to add to them if you feel like I'm missing out on some things):
A solution can be a method, a class, an application, a framework, a design element -- or anything that is part of contributing to the solution of a certain problem. Dirty solutions work, but they are hard to maintain which is a reallistic problem that people in the software development industry face quite often.
Dirty solutions aren't really "bad", they're just dirty. If they're dirty, most probably they need cleaning -- so some people avoid dirty solutions at the cost of being slow.
The Quick and Dirty
Suppose the timeline requires a solution fast -- an experienced engineer under no circumstances will let the project run off track, because he knows how important keeping the project manager off his back is. But the same forces at work in "Evolving a Giant" are the same ones at play in almost all software projects.
These situations usually turn out quick and dirty solutions which are harder to find, and harder to fix.
Avoid the Quick and Dirty
Just like how you might avoid a black fly buzzing around you while eating your sandwich for lunch, you just might want to avoid these quick and ditry solutions. This section gives advice on how your organization can avoid these quick and dirty solutions, by putting in place some measures and techniques for minimizing the forces that bring about these solutions.
Haste makes waste, and experience teaches us this. Therefore, when planning and shceduling deliverables, leave ample room for leeway -- and plenty of time for Refactoring. Make sure that when they're in a productive zone that they are not bothered unless the building is on fire and they're not moving away from the workstation.
If you're a project manager and are working on a timeline, make adjustments to the timeline as much as you can before harassing your developers. If you're a developer or a team lead, allocate enough resources (i.e. time and effort) to making sure that you stay as creative as you can be by taking enough breaks and keeping everyone fresh. Keeping someone fit to be productive is better than pulling a genius out of your hat.
If you find yourself writing a solution that doesn't look clean to you then it's not clean. Unless you yourself see it cleanly, you can't expect someone else to see it cleanly.
Making sure that your solution to the problem is clean will force you to stop worrying about the details and keep your eye on the overall challenge. Ask yourself if the solution you came up with is the cleanest possible solution you can think of given the time constraints -- as much as we want to have all the time in the world to figure out the cleanest solution, we just don't have it.
Conclusion
Making sure that your solution is quick and clean instead of quick and dirty takes a lot of effort. You will see that dirty solutions only postpone the agony of maintaining the system -- and like the wise people say, an ounce of prevention is better than a pound of cure.
Cliche's aside, the software industry relies on quick solutions to common problems -- so if your problem has been solved before, look at the solutions first before coming up with your own. After all, your limitation is a function of your experience and your knowledge: knowing this allows you to work on both and push your limitations further.
In an organization, it's important that you see your limitations and know how to work within them. So if you don't have enough experience, get as much as you can as soon as possible. If you don't have the knowledge, get as much of it as you can as soon as possible. That way, you'll be able to get quick and clean solutions like the experienced ones already do.
What's the use of being quick if you're going to the wrong place? What's the use of heading for the right direction if you don't start moving?
Some software projects (see Evolving a Giant) can benefit from these experienced engineers and designers -- but then the same forces encountered in these situations applied to even the most experienced engineers and designers can lead to another type of problem. This problem is something I like to call "The Quick and The Dirty".
The Quick Solution
Experience is the best teacher, and lessons learned through experience are hard to forget. To avoid complex solutions, a lot of engineers have found that if you avoid over-engineering a solution, you avoid a lot of the complexities associated with that solution. If they kept it simple and it works, they get the job done quicker than if they spent a whole lot of time over-engineering it.
The quick solution is usually effective -- it solves the problems, and creates less problems that need to be faced at the moment. The quick solution is usually sufficient especially if the enhancement/addition to the software is trivial and can be addressed with tricks from the engineer's bag. Of course, this assumes that the "solution" works, otherwise it's not a solution.
The Dirty Solution
Another thing experience brings is confidence. Usually, the more experienced engineers are more confident with their solutions compared to the solutions of those with considerably less experience or exposure. That leads to problems especially if you have a team composed of experienced and not so experienced engineers.
The dirty solution comes from both the experienced and the not so experienced engineers -- the solution becomes dirty if it needs to get cleaned up even after the solution has been deployed. Experienced developers have gone through the nightmare of maintaining their own dirty code, and the dirty code of others. This leads to the same problems that are encountered when working with a humongous software project that has already grown out of proportion.
Dirty doesn't necessarily mean not-elegant -- on the contrary, a lot of the "elegant" solutions are so complex they end up being more dirty than the quick solution (for examples, look at the merge sort and quick sort algorithm and compare these to insertion sort and even bubble sort). Dirty solutions have any one or all of the following properties (please feel free to add to them if you feel like I'm missing out on some things):
- Unnecessary elements
- Repetitive elements
- "Case Specific" elements
- Unjustified existence
A solution can be a method, a class, an application, a framework, a design element -- or anything that is part of contributing to the solution of a certain problem. Dirty solutions work, but they are hard to maintain which is a reallistic problem that people in the software development industry face quite often.
Dirty solutions aren't really "bad", they're just dirty. If they're dirty, most probably they need cleaning -- so some people avoid dirty solutions at the cost of being slow.
The Quick and Dirty
Suppose the timeline requires a solution fast -- an experienced engineer under no circumstances will let the project run off track, because he knows how important keeping the project manager off his back is. But the same forces at work in "Evolving a Giant" are the same ones at play in almost all software projects.
These situations usually turn out quick and dirty solutions which are harder to find, and harder to fix.
Avoid the Quick and Dirty
Just like how you might avoid a black fly buzzing around you while eating your sandwich for lunch, you just might want to avoid these quick and ditry solutions. This section gives advice on how your organization can avoid these quick and dirty solutions, by putting in place some measures and techniques for minimizing the forces that bring about these solutions.
Experienced engineers are like artists --- they take their time when solving a challenging problem.
Haste makes waste, and experience teaches us this. Therefore, when planning and shceduling deliverables, leave ample room for leeway -- and plenty of time for Refactoring. Make sure that when they're in a productive zone that they are not bothered unless the building is on fire and they're not moving away from the workstation.
Being creative is easier than being harassed. You can't squeeze creativity out of a person, so don't even try.
If you're a project manager and are working on a timeline, make adjustments to the timeline as much as you can before harassing your developers. If you're a developer or a team lead, allocate enough resources (i.e. time and effort) to making sure that you stay as creative as you can be by taking enough breaks and keeping everyone fresh. Keeping someone fit to be productive is better than pulling a genius out of your hat.
If it doesn't look clean, then it's not.
If you find yourself writing a solution that doesn't look clean to you then it's not clean. Unless you yourself see it cleanly, you can't expect someone else to see it cleanly.
Making sure that your solution to the problem is clean will force you to stop worrying about the details and keep your eye on the overall challenge. Ask yourself if the solution you came up with is the cleanest possible solution you can think of given the time constraints -- as much as we want to have all the time in the world to figure out the cleanest solution, we just don't have it.
Conclusion
Making sure that your solution is quick and clean instead of quick and dirty takes a lot of effort. You will see that dirty solutions only postpone the agony of maintaining the system -- and like the wise people say, an ounce of prevention is better than a pound of cure.
Cliche's aside, the software industry relies on quick solutions to common problems -- so if your problem has been solved before, look at the solutions first before coming up with your own. After all, your limitation is a function of your experience and your knowledge: knowing this allows you to work on both and push your limitations further.
In an organization, it's important that you see your limitations and know how to work within them. So if you don't have enough experience, get as much as you can as soon as possible. If you don't have the knowledge, get as much of it as you can as soon as possible. That way, you'll be able to get quick and clean solutions like the experienced ones already do.
What's the use of being quick if you're going to the wrong place? What's the use of heading for the right direction if you don't start moving?
Wednesday, July 26, 2006
Evolving a Giant
I had the chance to work with a couple of different firms who have created software on which they run their businesses on. The software they created are not really for public consumption, but the service they offer using the software is for public consumption. They don't distribute their application to clients, but they provide services through the software created. Even the services they offer are written as software, but not in the traditional software service sense -- but rather as end-user experiences.
Now these companies are _not_ software development companies, although they employ people that know how to more or less write software. After a few years of existence, they decide to expand their service offerings: they do aggressive marketing campaigns, thinking that the software they already have will be enough for what needs to get done. Now here's the catch: they have a handful of developers and tons of different sometimes totally unrelated products/services to roll out. The deadlines are insane, and the competition in the industry is cut throat. What are the problems and what are the possible solutions?
The Giant Software
To set the table up, the companies have this piece of giant software -- or should I call it the giant slayer, because it taughts itself as the one that will bring the company to the top and in effect slay the giant already atop the mountain. But the software itself is huge -- too many components written by too many people who have gone and are being read/modified/maintained by new people every so often. So basically, you get a mish-mash of solutions in one giant piece of software.
The software is not trivial -- it is complex enough that only people with appropriate software experience with the programming language used will be able to change the software significantly, and effectively maintain it. It's complex and requires high performance, so you can imagine the programming language candidates for these applications.
The fun part is, the software is currently working. It delivers the required functionality within schedule and according to specifications.
Here's the Rub...
One day, one of the major features of the software will need to be changed -- meaning: code revision, testing, and deployment to production. Deadline: one week. So the developers go ahead and hack on the solution, to beat the deadline and deliver the service -- but that problem is one week is not enough for the giant software to change. Even if there are a couple of people working on the solution, the reality is that there is not enough time given the complexity of the system to make the necessary changes.
So what are the developers left with? Shorten the testing time, lengthen the code revision time, and deploy (and optionally cross their fingers).
The service is delivered on time, and the marketing machinery starts rolling -- more services come, and the deadlines get more and more insane. Until the inevitable happens.
Downtime
A change one developer made to accomodate one feature now breaks at least one other service. As though it doesn't get any worse, the change was supposedly trivial -- just a couple of lines of code changed here and there, and suddenly everything starts breaking. The developers start bug hunting, and in the meantime some of the services go down.
Developers start losing morale, there's no solution in sight, and the bug hunting and debugging goes on longer than necessary. Eventually the problem gets fixed, and the team is exhausted, but the marketing is continuosly rolling -- there's no time to recoup the losses, and no time to celebrate the solution (not even enough time to fix the already gigantic mess the software is).
Lessons Learned
The first lesson learned goes something like this:
This talks about maintaining the software's size so that the company shouldn't require more than the same number of people that originally created the software to maintain it. That means keeping lines of code low, and keeping components isolated and self-maintaining.
Another problem encountered when the software written is already so big, is the fact that interdependency (or close coupling) is such a temptation just too hard to avoid. It's increasingly harder and harder to isolate the common parts of a large software project that it is most prudent to isolate the common parts early on before letting it grow out of proportion.
This is usually a management issue -- we all would love to get the best developers for the price we're willing to pay. Sometimes though, they're just not available, already somewhere else, and what's left are the people submitting their resume's to your HR department. In cases like this, we either invest in them and make them the geniuses that we need for the business or deal with the fact that they can only do so much.
The lesson relates to managing both the resources and the software. It talks about fixing the software so that it becomes easier to extend/fix in the future. That's a whole different discussion altogether though.
At any rate, in order for the software to not turn into a giant piece of hell to touch, you make sure the people you have will be able to maintain it in its current form. Otherwise, any additions will make it less maintainable than it already is, without it getting fixed in the first place.
Although it is a tough pill to swallow, continous maintenance of the software for bug fixes, performance enhancements, and issue resolutions need to come first before feature additions. If the software is too easy to break, then that's one bug that needs fixing. If the software is too slow for the requirement, then that's one bug that needs fixing. If the issues are recurring, then that's another bug that needs fixing.
Unless you can extend your software painlessly in its current form, then you should work harder to make your software easier to extend -- because extending it will be inevitable anyway, invest early and reap rewards late.
Conclusions
So if you face yourself with a giant, it's a good idea to turn that giant into a midget by making sure it's as nible as possible before adding the functionality you think you need. It won't only result to savings in the long run, but it will save you from the pain of a falling giant.
Standing on the shoulders of giants doesn't necessarily require that there be a giant to stand on the shoulders of. There are no nimble giants, so unless you don't plan on moving, you can grow yourself one.
Now these companies are _not_ software development companies, although they employ people that know how to more or less write software. After a few years of existence, they decide to expand their service offerings: they do aggressive marketing campaigns, thinking that the software they already have will be enough for what needs to get done. Now here's the catch: they have a handful of developers and tons of different sometimes totally unrelated products/services to roll out. The deadlines are insane, and the competition in the industry is cut throat. What are the problems and what are the possible solutions?
The Giant Software
To set the table up, the companies have this piece of giant software -- or should I call it the giant slayer, because it taughts itself as the one that will bring the company to the top and in effect slay the giant already atop the mountain. But the software itself is huge -- too many components written by too many people who have gone and are being read/modified/maintained by new people every so often. So basically, you get a mish-mash of solutions in one giant piece of software.
The software is not trivial -- it is complex enough that only people with appropriate software experience with the programming language used will be able to change the software significantly, and effectively maintain it. It's complex and requires high performance, so you can imagine the programming language candidates for these applications.
The fun part is, the software is currently working. It delivers the required functionality within schedule and according to specifications.
Here's the Rub...
One day, one of the major features of the software will need to be changed -- meaning: code revision, testing, and deployment to production. Deadline: one week. So the developers go ahead and hack on the solution, to beat the deadline and deliver the service -- but that problem is one week is not enough for the giant software to change. Even if there are a couple of people working on the solution, the reality is that there is not enough time given the complexity of the system to make the necessary changes.
So what are the developers left with? Shorten the testing time, lengthen the code revision time, and deploy (and optionally cross their fingers).
The service is delivered on time, and the marketing machinery starts rolling -- more services come, and the deadlines get more and more insane. Until the inevitable happens.
Downtime
A change one developer made to accomodate one feature now breaks at least one other service. As though it doesn't get any worse, the change was supposedly trivial -- just a couple of lines of code changed here and there, and suddenly everything starts breaking. The developers start bug hunting, and in the meantime some of the services go down.
Developers start losing morale, there's no solution in sight, and the bug hunting and debugging goes on longer than necessary. Eventually the problem gets fixed, and the team is exhausted, but the marketing is continuosly rolling -- there's no time to recoup the losses, and no time to celebrate the solution (not even enough time to fix the already gigantic mess the software is).
Lessons Learned
The first lesson learned goes something like this:
Any one piece of software should not reach gigantic proportions such that the resources required to maintain it is greater than what is currently available.
This talks about maintaining the software's size so that the company shouldn't require more than the same number of people that originally created the software to maintain it. That means keeping lines of code low, and keeping components isolated and self-maintaining.
Another problem encountered when the software written is already so big, is the fact that interdependency (or close coupling) is such a temptation just too hard to avoid. It's increasingly harder and harder to isolate the common parts of a large software project that it is most prudent to isolate the common parts early on before letting it grow out of proportion.
Listen to Developers
This means asking them what they think and how they feel about the problem and the solution. Ask them how long they will take to implement it, and how easy or hard it is compared to the actually pausing and fixing the software first before implementing something new.
This is usually a management issue -- we all would love to get the best developers for the price we're willing to pay. Sometimes though, they're just not available, already somewhere else, and what's left are the people submitting their resume's to your HR department. In cases like this, we either invest in them and make them the geniuses that we need for the business or deal with the fact that they can only do so much.
The lesson relates to managing both the resources and the software. It talks about fixing the software so that it becomes easier to extend/fix in the future. That's a whole different discussion altogether though.
At any rate, in order for the software to not turn into a giant piece of hell to touch, you make sure the people you have will be able to maintain it in its current form. Otherwise, any additions will make it less maintainable than it already is, without it getting fixed in the first place.
Don't fix what's not broken -- but if it's easy to break, it's considered broken.
Although it is a tough pill to swallow, continous maintenance of the software for bug fixes, performance enhancements, and issue resolutions need to come first before feature additions. If the software is too easy to break, then that's one bug that needs fixing. If the software is too slow for the requirement, then that's one bug that needs fixing. If the issues are recurring, then that's another bug that needs fixing.
Unless you can extend your software painlessly in its current form, then you should work harder to make your software easier to extend -- because extending it will be inevitable anyway, invest early and reap rewards late.
Conclusions
So if you face yourself with a giant, it's a good idea to turn that giant into a midget by making sure it's as nible as possible before adding the functionality you think you need. It won't only result to savings in the long run, but it will save you from the pain of a falling giant.
Standing on the shoulders of giants doesn't necessarily require that there be a giant to stand on the shoulders of. There are no nimble giants, so unless you don't plan on moving, you can grow yourself one.
Friday, June 23, 2006
Evolution
The usual first problem with change is the question of "what happens to the old one?" Whether it's a process, a technology, a solution, a religion, a name, a pet, or anything. Comfort and familiarity with an existing entitiy or concept is something that prevents a lot of people and companies from change. Another thing that really prohibits change or jumping into change is cost and risk.
Cost is the amount of resources and effort required to perform or obtain a task or resource. It is usually monetary, but most usually it is more time than anything else.
Risk the likelyhood of loss or failure in acquiring a resource, investing time/money, and even emotional capital.
How do you introduce change so that it's 1) not so costly 2) not too risky or 3) more appropriate for what you need?
You evolve. For the next few posts, I will propose some ways of evolving your current processes, practices, and approaches so that becoming Agile is not as costly nor Risky as it sounds.
The only constant in the world is change.
Cost is the amount of resources and effort required to perform or obtain a task or resource. It is usually monetary, but most usually it is more time than anything else.
Risk the likelyhood of loss or failure in acquiring a resource, investing time/money, and even emotional capital.
How do you introduce change so that it's 1) not so costly 2) not too risky or 3) more appropriate for what you need?
You evolve. For the next few posts, I will propose some ways of evolving your current processes, practices, and approaches so that becoming Agile is not as costly nor Risky as it sounds.
The only constant in the world is change.
Monday, June 05, 2006
Management
Do you notice that management gets in the way of actually being able to effectively deliver a product? In traditional business models, managers tends to "over-manage" the resources: Gantt Charts, Planning Sessions, Status Meetings, Team Building, Status Reports, and all the management initiated activities tend to get in the way of delivering. Of course, there has to be a point to management, otherwise what is the need for a manager?
Here's an idea: let management do the work that needs to be done. What do I mean by "work that needs to be done"? It's really simple: make sure that there is progress, and then get back to doing something actually contributory to the project.
When I write "something actually contributory to the project", then that means something *tangible* or something (an action, or set of actions) directly contributory to a project's value creation model. And in software development, that's only one of two things: code or documentation.
When managers need to write code, then that means there are not enough developers writing the code that needs to be written. When managers need to write documentation, then that means that the developers aren't writing the documentation that's necessary in the project. These are cues to the manager, that there has to be something done about it. And if a manager has to write code to deliver a product on time, that means the developers aren't capable, aren't working, or aren't worthy of the pay they're getting; or of course the manager himself over-estimated the capabilities of the team, or undercalculated the risks of a tight timeline.
If there are other things that need to be done by a manager, that's budgeting the time spent on a project, and the available resources at his disposal.
Did you ever notice that the successful software development firms have competent managers who can do what the subordinates do, as well as do very little management as much as possible? Here's a hint:
The role of a manager is to make sure that there's little work for him to do and enough time for him to spend on more imporatant things by doing the things that need to be done so that things get done on their own.
This means getting the right people, getting the right tools, setting the right timeline, and keeping the resources utilized effectively to provide a high level of productivity. Here's a simple goal for a manager: do everything that needs to be done so that you don't need to do it yourself. Seems like a paradox? The point is getting others to do the work that you don't have to do yourself, but need to do otherwise.
Sounds shrewd? Not really... It's really more an advantage to the project and the firm if the manager continues to do only the necessary actions that a manager needs to do: like tracking the progress of a project and keeping the resources busy. That way, a developer can concentrate on developing, a manager concentrates on keeping the developers developing, and the firm can concentrate on delivering its products.
The more you manage, the less things get done.
Here's an idea: let management do the work that needs to be done. What do I mean by "work that needs to be done"? It's really simple: make sure that there is progress, and then get back to doing something actually contributory to the project.
When I write "something actually contributory to the project", then that means something *tangible* or something (an action, or set of actions) directly contributory to a project's value creation model. And in software development, that's only one of two things: code or documentation.
When managers need to write code, then that means there are not enough developers writing the code that needs to be written. When managers need to write documentation, then that means that the developers aren't writing the documentation that's necessary in the project. These are cues to the manager, that there has to be something done about it. And if a manager has to write code to deliver a product on time, that means the developers aren't capable, aren't working, or aren't worthy of the pay they're getting; or of course the manager himself over-estimated the capabilities of the team, or undercalculated the risks of a tight timeline.
If there are other things that need to be done by a manager, that's budgeting the time spent on a project, and the available resources at his disposal.
Did you ever notice that the successful software development firms have competent managers who can do what the subordinates do, as well as do very little management as much as possible? Here's a hint:
The role of a manager is to make sure that there's little work for him to do and enough time for him to spend on more imporatant things by doing the things that need to be done so that things get done on their own.
This means getting the right people, getting the right tools, setting the right timeline, and keeping the resources utilized effectively to provide a high level of productivity. Here's a simple goal for a manager: do everything that needs to be done so that you don't need to do it yourself. Seems like a paradox? The point is getting others to do the work that you don't have to do yourself, but need to do otherwise.
Sounds shrewd? Not really... It's really more an advantage to the project and the firm if the manager continues to do only the necessary actions that a manager needs to do: like tracking the progress of a project and keeping the resources busy. That way, a developer can concentrate on developing, a manager concentrates on keeping the developers developing, and the firm can concentrate on delivering its products.
The more you manage, the less things get done.
Monday, May 22, 2006
Tomato
Tomaytoe, Tomahtoe -- the same thing, referred to differently. Usually this is one of the major problems a non-tech person will encounter when talking with a tech person. Although techinical people are very precise (or very particular) with the words used to refer to things (server application as opposed to server hardware), there's a certain degree of assumed context whenever two technical people talk to each other:
Developer 1: "I built the version, and it ran without bugs. However, when you did it, the bugs came out!"
Developer 2: "Yeah, but I seem to have used a different version of the library from what you used, so I'll just get 2.0 then build-run-test it."
That's usually how developer talk happens within a team. The problem is, there's something missing: context. But they seemed to understand each other! Non-technical people hearing this will most probably say "WTF?!"
Now let's see another conversation between a business person and a developer (which happens all too often):
Business Person: "I just committed a delivery date on the 10th, we need it out by then..."
Developer: "It's not going to happen."
Business Person: "What do you mean, it's not going to happen?!"
Developer: "It's not going to happen."
Business Person: "What do you mean it's not going to happen?!"
Developer: "It's not going to happen."
Business Person: "It's not going to happen on the 10th?"
Developer: "It's not going to happen."
Business Person: "When will it happen?"
Developer: "It's not going to happen."
You can just imagine what will happen to this conversation... Absolutely nothing. The business person is looking for an explanation, and the developer isn't interested in giving one. Who's at fault here? Look at the first statement: I just committed a delivery date on the 10th, we need it out by then...
Does it look the least bit familiar? It's the PHB syndrome... If you don't know what PHB means, maybe you should start reading this too.
At any rate, the business people (especially management people) tend to keep their eye on the sale, and not on the product. While the developers on the other hand, tend to look at the product more than the sale. Here's a way of rephrasing the first statement:
This way of approaching developers will convey 2 important messages: 1) I'm asking for your opinion 2) I think I might be willing to listen. The last thing you want is a developer that doesn't listen, and will sabotage your project. We all don't want that.
Rephrasing your statements can go a long way especially if you want to make sure that some things happen as you expect them, while keeping everyone happy. Try it, then make sure you say it with sincerity. You might get a different answer from the developer.
Do you have communication stories? Let them be known by emailing me your comments and stories through dean [at] orangeandbronze [dot] com -- they might just well be listed here.
Communication is a process of conveying information while dodging the challenges brought about by media, experience, environment, and preference. Effective Communication happens only if the process is effective and the message clear.
Developer 1: "I built the version, and it ran without bugs. However, when you did it, the bugs came out!"
Developer 2: "Yeah, but I seem to have used a different version of the library from what you used, so I'll just get 2.0 then build-run-test it."
That's usually how developer talk happens within a team. The problem is, there's something missing: context. But they seemed to understand each other! Non-technical people hearing this will most probably say "WTF?!"
Now let's see another conversation between a business person and a developer (which happens all too often):
Business Person: "I just committed a delivery date on the 10th, we need it out by then..."
Developer: "It's not going to happen."
Business Person: "What do you mean, it's not going to happen?!"
Developer: "It's not going to happen."
Business Person: "What do you mean it's not going to happen?!"
Developer: "It's not going to happen."
Business Person: "It's not going to happen on the 10th?"
Developer: "It's not going to happen."
Business Person: "When will it happen?"
Developer: "It's not going to happen."
You can just imagine what will happen to this conversation... Absolutely nothing. The business person is looking for an explanation, and the developer isn't interested in giving one. Who's at fault here? Look at the first statement: I just committed a delivery date on the 10th, we need it out by then...
Does it look the least bit familiar? It's the PHB syndrome... If you don't know what PHB means, maybe you should start reading this too.
At any rate, the business people (especially management people) tend to keep their eye on the sale, and not on the product. While the developers on the other hand, tend to look at the product more than the sale. Here's a way of rephrasing the first statement:
Do you think we can get this out by the 10th?
This way of approaching developers will convey 2 important messages: 1) I'm asking for your opinion 2) I think I might be willing to listen. The last thing you want is a developer that doesn't listen, and will sabotage your project. We all don't want that.
Rephrasing your statements can go a long way especially if you want to make sure that some things happen as you expect them, while keeping everyone happy. Try it, then make sure you say it with sincerity. You might get a different answer from the developer.
Do you have communication stories? Let them be known by emailing me your comments and stories through dean [at] orangeandbronze [dot] com -- they might just well be listed here.
Communication is a process of conveying information while dodging the challenges brought about by media, experience, environment, and preference. Effective Communication happens only if the process is effective and the message clear.
Friday, May 12, 2006
Conversation
How do you usually hold conversations with a technical person? Do you follow a plan? Or do you hold it like a normal one? If you're in the technology industry, communication is an important aspect of the operations. If you can communicate better with your peers/colleagues, you basically have 50% of your work done. However, not everyone has the best communication skills in the work place.
The perennial question in IT companies is how do you bridge the divide between the sales group and the tech group. What type of communication breakdowns usually occur? Are there any verbal or written cues that you should look out for? Is there more to it than meets the eye (or ears)?
In the next few posts, I shall discuss the many different issues interpersonal communication issues and problems in the technology industry especially in the small group sense. In the later parts, I shall outline the many pitfalls that non-technical people encounter when talking to technical people.
What you say and mean is not what I necessarily hear and understand.
The perennial question in IT companies is how do you bridge the divide between the sales group and the tech group. What type of communication breakdowns usually occur? Are there any verbal or written cues that you should look out for? Is there more to it than meets the eye (or ears)?
In the next few posts, I shall discuss the many different issues interpersonal communication issues and problems in the technology industry especially in the small group sense. In the later parts, I shall outline the many pitfalls that non-technical people encounter when talking to technical people.
What you say and mean is not what I necessarily hear and understand.
