Survey Question: Software Development Bottlenecks

[Joe McIntyre]

Hi all.

I've always had a love/hate relationship with software development. Today I think I realized where the hate part comes from, and I'm posting to ask if others agree with my perception of the overall development cycle. Basically, it boils down to this. After completing a development cycle, I look back and here is what I think I see:

Time to build space shuttle, go to mars, and collect soil samples:
4,000 man hours
Time to build the little red button that initiates lift-off:
4,000,000,000 man hours

I think my question involves software development in general, but I'm posting to javaranch because java is the only OO language I know. I'm interested to know if other developers would agree with my impressions? I am being facetious, of course, in the example above, but I guess I usually end up feeling that on every project there are always a few tiny bottlenecks that eat up an incredible amount of time. Usually these bottlenecks involve features that cannot be left out, and I am left with no choice but to spend the hours required to implement them.

If it is true that a few small bottlenecks eat up a disproportionate amount of resources, what can be done to change this?

[Bill Cruise]

There are literally entire shelves full of books written on this topic every year. I recommend The Pragmatic Programmer (that's the author's website, not mine) as a good starting point. It has a lot of good practical advice for writing great OOP software. If you check out the books the authors refer to, you'll soon fill a shelf with other great software engineering titles.

[Jimmy Clark]

...but I guess I usually end up feeling that on every project there are always a few tiny bottlenecks that eat up an incredible amount of time.

Just to be clear, you usually end up feeling this way on projects that you work on. And your interpretations are coming from your experiences with them.

Not too much can be said about these projects, since you have provided no details. However, projects that involve creating software require a lot of things other than compilers, IDE and knowledge of how to sort array contents. Planning, skill set analysis, architecture design, requirements analysis, documented technical design, scheduling, are just a few areas that need attention. Projects that are successful have all of these things taken care of to the right degree appropriate for the project and the organization.

Good software architecture design typically handles/prevents bottlenecks from coming up. Without this critical design step, there is nothing to prevent bottlenecks from existing.

If it is true that a few small bottlenecks eat up a disproportionate amount of resources, what can be done to change this?

You can improve your knowledge about software engineering and when given the opportunity to design an architecture, you can make sure that you prevent bottlenecks.

Read the following to learn more:

Death March by Edward Yourdon
Paperback: 256 pages
Publisher: Prentice Hall
Language: English
ISBN-10: 013143635X
ISBN-13: 978-0131436350
[ June 14, 2008: Message edited by: James Clark ]

[Joe McIntyre]

[Edit: not-so-nice response removed. If you don't want to hear other peoples' opinions, it'd be best not ask for them.]
[ June 15, 2008: Message edited by: Bear Bibeault ]

[Joe McIntyre]

For any future posters, I'll clarify my question with details as to the problem that brought about my original post.

Downloaded software for my MacBook Pro to alert me throughout the day with tasks to do. Wasn't satisfied, seemed like an easy programming task, so I fired off my own version in java. Took no time at all to write an app that did what I wanted - to provide audio and visual messages alerting me to begin a new todo task at various times throughout the day. Spent about maybe 45 minutes and I was done... except for two problems.

1) control the volume of the beep tone (in case I was working on audio recording, so it wouldn't blow my speakers out).

2) bring alert dialog box to the top (I'd no doubt be using another application)

Both of these problems were, as far as I can see, insurmountable. Or if they could be solved, they would require days of research and preparation.

So no... this isn't a software architecture problem... not even close. I tried a few tricks, but all of them had flaws (due to bugs).

The reason I posted the comment above is that for the first time I realized that this happens to me on almost every project. I build a space ship that can go to mars and collect soil samples, then when I go to build the little red button that launches the space ship, I run into problems. These tiny little problems either kill the project altogether, or else they burn an inordinate amount of resources to solve.

When you read about the philosophy that spawned OO programming to begin with, the list a number of common problems that the paradigm was designed to solve. For example, one common problem was rewriting the same software modules over and over. So I'm just wondering, what kind of techniques could be employed to get around this bottleneck problem? Of course, I realize that every engineering project (software or otherwise) has its bottlenecks. But what I don't understand is why, in software, most of the links of the chain can hold a force of 1,000 pounds, while the weak link can only seem to hold 3 ounces.

[Ilja Preuss]

All, please "be nice" to each other. Thanks!

[Ilja Preuss]

First, I don't think that this experience is specific to software development. The proverb "the devil is in the details" is much older than computers.

In my opinion, there are two major things to learn to deal with this:

- Expect the unexpected. Break the requirements down. If for something important you don't know how to do it, prototype it early on. Don't expect that something will be easy just because it "should" be. And simply gather experience on where the devil is waiting for you.

- Assume that "this is hard" most often in fact means "I don't know enough to make this easy". Most often, the bottleneck actually is not in the doing, but in the learning how to do it. Get good at getting help for that. Working in a team helps, too.

For 2) take a look at JDesktop. It allows to add tray icons to the system tray, which then can show popup messages just like Thunderbird and a couple of other programs do when an event occurs.

[Joe McIntyre]

It is pointless to ask questions like this. We have profilers for application code, but unfortunately there are no profilers for the developers, or for the development process itself. If there were, we'd all be shocked by the feedback.

Another possibility is that in the open source community, there's only one job title, and that's "developer". In the business world people with different job titles compete with each other. This creates a considerable amount of bad feelings and hostility, however, it does create a better product. To illustrate, what if you had two teams, one gets points for writing good code, another for streamlining the coding process. Whenever code is written, the streamliners try to come up with a simpler way to do the same thing, and if they do come up with the simpler way, then the coding team has to rewrite it the simpler way. (So the coding team hates the streamliners, because every time they streamline something, the coding team has to redo their work.) But rather than typing the following code 50 million times:

    
for(int i = 0; i < 100; i++) {
}

we recognize a bottleneck and convert it to:

i to 100 {
}

Of course, this low level detailed example of actual code does not do the concept justice. Most of our bottlenecks exist at a much higher level... trying to make java interact with the host platform or something like that.

[Ilja Preuss]

Originally posted by Wally Flint:
We have profilers for application code, but unfortunately there are no profilers for the developers, or for the development process itself. If there were, we'd all be shocked by the feedback.

Not sure why you think I would be shocked.

Anyway, profilers for developers would be quite pointless - like profilers for single classes. Profilers for *teams* - that's where the magic happens.

Fortunately, there *are* profilers for teams - they are called Iteration Retrospectives. (If you are a team of one person, I pity you. You might still want to give it a try.)

In the business world people with different job titles compete with each other. This creates a considerable amount of bad feelings and hostility, however, it does create a better product.

Not every where in the business world people with different job titles compete with each other. In fact I would argue that in the best business, they collaborate, and in the very best, they don't take attention to job titles at all. I've heard of several companies where everyone is called "member of technical staff", and of another where everyone is totally free to choose his job title as he wishes.

To illustrate, what if you had two teams, one gets points for writing good code, another for streamlining the coding process. Whenever code is written, the streamliners try to come up with a simpler way to do the same thing, and if they do come up with the simpler way, then the coding team has to rewrite it the simpler way. (So the coding team hates the streamliners, because every time they streamline something, the coding team has to redo their work.)

Why wouldn't they work better as one team, everyone trying to streamline every code, without hating each other? I very much like Pair Programming for this...

But rather than typing the following code 50 million times:

    
for(int i = 0; i < 100; i++) {
}

we recognize a bottleneck and convert it to:

i to 100 {
}

Well, all that needs is someone who is aware of the duplication, and a language which allows to remove it with a reasonable amount of effort.