How to design a Domain or API without UML? I'm NOT trying to be funny!

[Maurice brown]

Hello, I'm wondering how do senior level developers design domains, APIs, BLL, etc without UML? If they are not using UML, I'm also assuming they are NOT doing OOAD.

I ask because no matter what company i work for, interview for, or architect i talk to, most are not using UML. If they do, it a random occurrence. Is this really the practice? Does this really work?

I would not design a database without planning it out. why would anyone develop a long lasting application without using OOAD?

I"m really trying to understand how most companies are doing this. Is it all cowboy programming and they just hope for the best as they develop the various classes or is it more structured?

Is there some secret that I don't know about?

Thanks!

[A.J. Côté]

I typically design in my head, sometimes using a pen and paper for more complex cases. Then I write the classes. Then I generate the database diagrams and UML diagrams from the classes for documentation and to make sure my design make sense, bypassing this steps for trivial cases. I assume that others might proceed this way too.

There is nothing wrong with starting with UML if you feel more comfortable this way. That's the 2 typical ways to to design that I know of and that's why most tools support bi-directional engineering.

[Jeanne Boyarsky]

Maurice,
UML is just a notation. Not using UML doesn't mean people aren't doing design. They could be using PowerPoint, Visio, a whiteboard, etc.

I don't use UML anymore either. I do design. Not each and every attribute, but I do design.

[Junilu Lacar]

I hardly ever (maybe a handful of times in the last ten years) use UML when I design and when I do use diagrams, it's usually a simplified notation on a whiteboard that I use as a starting point, at a very high level of abstraction at the component or module level. I use testing and refactoring while doing Test-Driven Development to do detailed design. I find that a few high-level diagrams are good for setting the context of a conversation with other developers and then focused, expressive tests to communicate the detailed design works best for me and my teams.

[Tim Holloway]

For some of us, UML was invented only yesterday.

I started with flowcharts. Later I augmented them with HIPO diagrams. Originally, I collected a set of plastic drawing stencils, then as computer graphics became standard used GUI tools (including brute-force painting programs). Had a flowcharting program that drew using ASCII Art on a minicomputer. I put together a program of my own that optimized layout of hierarchy diagrams for HIPO. Along came Visio, and it, too was grist for the mill.

Not everything was drawing, though. I got into using outliners for design in the late 1980s, when such tools were really popular. Later, I graduated to mind-map tools, which are still a great favorite of mine - Freemind is cool, because while it's a GUI app, you can type outlines and stuff in and hardly ever actually need to touch a mouse (and, incidentally, it's a Java app!). I also outline in OpenOffice Writer, which has the advantage that I can easily migrate documents between it and FreeMind (subject to content restrictions) and output PDFs and ePub books for distribution as needed.

Along the way, pseudo-code has been a mainstay, although as OOP became my basis for design, flowcharting pretty much became unnecessary. I did "literate programming" even before Knuth coined the term, since I often write comments before code. As time went on, I got tools for that, including Commodore's AutoDocs, which are a lot like Oxygen and JavaDocs. And I wrote an AWK script that would parse tagged comments and build Windows Help files from them (back when I was doing device driver design for Windows).

Most recently, I've taken to using Wikis and Markdown documents.

UML became very popular at the local "Dilbert" corporations back around Y2K. Meaning the places where IT policy is set by Dogbert (external consultants) and Management pretty much exists to hand them large quantities of cash. Rational Rose was entirely too expensive for me to justify privately buying back then, so I didn't get into it. Next large company I ended up at was an IBM-free shop, but I got infected by Oracle-style architectural design, which was based on a core set of documents: Executive Summary, Functional Specification, Implementation Specification, and User's Guide (varying slightly, depending on requirements).

These days, you can do UML for free thanks to ArgoUML (which is a Java app), and I keep that in my stable along with jGantt (I was commissioned to create an EBCDIC Art Gantt program as one of my first mainframe jobs!) I think UML lost a lot of its luster locally. Indications are that a major (2-year) program might have been based on Rational and they spent 18 months drawing stick-figure (actor) diagrams, then panicked when they realized only 6 months remained and just started coding randomly. So most of the UML I do is stuff like swim-lane diagrams, since they're good for designing protocols for workflows, such as web services.

Most often, though I end up doing something like this:

Stage 1: the proverbial back-of-the-envelope sketch, drawn by hand on paper. When doing this on a corporate project, this stage often ends up with an Executive Summary doc, since Management has to approve the project.

Stage 2: brainstorming via Freemind. This is where I capture features as fast they get suggested, as well as build inventory of components, design menu and dialog hierarchies, and so forth.

Stage 3: detailed docs. Either via MS-Word (OpenOffice) style docs or my Trac wiki or both. At this point I may also layout a timeline (Trac/jGantt) and begin the prototype spec manuals.

Stage 4: implemention. Code projects are set up (or stolen), details are thrashed out, Wiki and/or "Word" documents get fleshed out and updated. And we're off to the races.

Incidentally, I've recently added a Kanban-style tool to my stable, but I haven't worked on a collaborative project since I did, so I'm not sure how well it will serve. Prior to this, I've used private message boards. My biggest project has several forums, since the offshore developers don't need to be seeing internal discussions and another forum is for ongoing technical support issues. I tried installing a forum tool into Trac as a plugin, but the results on that were disappointing.

[A.J. Côté]

Hello Tim,
I think you forgot high level diagrams of systems drawn on restaurant napkins at the end of a lunch with the waiter pen although the back of the envelope sketch is close to it ;-).

The system was born. We just kept the napkins and refined the design.

For private forums, innd (InterNetNews daemon) isn't that bad and there is many news clients to pick from to access the data.

Bugzilla comes to mind too although we have recently used RT (request tracker) for that purpose, not only for bugfixes but also to follow the progress and dependencies of any task related to the project.

[Maurice brown]

Thanks guys for taking the time to answer my questions.

So it seems that little high level OOAD is taking place. It me, it would seem that while you can design and application any kind of way that kinda makes sense, it would be better to at least be able to represent this code at a bird's eye view just to see if it DOES MAKE SENSE OR IS A GOOD DESIGN. (I can see at least how converting code into a high level design could be beneficial. At least you can critique what was written even if you don't have time to fix anything.) Of course, No design is going to be completely perfect unless it is really small and has a very limited scope, but I can imagine that If i code without a plan, i could end up with very counter-intuitive code that makes little sense. Then I can imagine that I'd introduce more complications as I try to overcome or shortcut the bad design.

I know that I've looked at some code and architecture and wondered why it was so hard to read and extremely counter-intuitive. Now I'm a novice. But i can still read.

I understand that UML and even OOAD does not guarantee a good design or that the good design will be properly maintained. Nevertheless, I can only imagine that the less structure there is to the design, the fewer eyes on the original design, will result in difficult to read code IF NOT simply poorly designed--more than likely both.

If you guys disagree, please explain. If you agree, please explain. I'm trying to understand the nature of this. I've been mostly a database person. There is bad design there too but much more care is actually taken in DB design and it has less code to worry about.

[Junilu Lacar]

Maurice brown wrote:it would be better to at least be able to represent this code at a bird's eye view just to see if it DOES MAKE SENSE OR IS A GOOD DESIGN.

IMO, it's not the code you need to represent but rather, the ideas that are captured in the software. The code itself is just another way of representing those ideas (I often call them "theories"). If you want more details about this, read Jack Reeves' essays on code as design.

I think what you wrote is indicative of how comparing ourselves to and regarding ourselves as "engineers" often steers us in the wrong direction. Sure, there are many parallels to what we do and what traditional engineers do. But therein also lies the rub. Traditional engineering has very well-established technical foundations. The field of software development is, relatively speaking, very young and immature and it's as much an art as it is a science. Engineers often benefit greatly from plans that are created up front that go into the most minute details. We in software, however, have been far less successful with that kind of approach.

We in software development don't have the kind of well-established technical standards that traditional engineering disciplines have. UML is a perfect example of this shortcoming. It is our industry's attempt at some kind of standardized notation. But UML will never be adopted as a standard to the same extent as the notations used in building blueprints or electrical diagrams, for example. There's just not enough incentive or pressing need to do so, IMO. And I think the difference boils down to this: In traditional engineering, the things they produce are tangible and concrete and for the most part, the notations they use in their diagrams allow them to create high-fidelity representations of the real things.

For example, an engineer can look at a blueprint and see a box with lines ending in arrows along the sides and numbers on them and interpret it as a specification for a room with certain physical dimensions. He can picture the size of the room and even measure it out and see it in real life or at scale. There's very little left to the imagination and very little room for errors in interpretation. Other engineers can look at the diagram and validate it against well-established standards, applying well-known calculations and factors to figure out things like maximum load capacity and thermal efficiency before the structure is ever built.

We often don't have the same level of detail and fidelity in our software diagrams nor do I think it's even necessary or beneficial. If a software developer draws two boxes and a line between those two boxes, there are many things left to the imagination and lots of room for different interpretations as to what those boxes can represent in the software. The elements in the diagram are really just abstract representations of concepts in the developer's mind, concepts that need to be defined, organized, and written out as code that will eventually be used to produce the software that a computer can execute. How do you validate a diagram like that? What well-known calculations and factors can you use to figure out the software's quality attributes based on the information in the diagram? To my knowledge, there are none.

The best way that I know to validate software is to test it. The best way that I know to test software is to test it early and test it often. And the best way that I know to design software is to use tests to drive and inform your decision-making process along the way.

IMO, there are three main things that contribute to good design:
1. Designers and developers with good experience and technical knowledge
2. Constant collaboration between the designers, developers, and those who will be testing and using the software
3. Automated testing and continuous integration

#1 is where things like technical standards, design patterns, and refactoring will come into play. This is also where art and craftsmanship is brought into the mix.
#2 is where things like high-level diagrams and tests are used. Art and craftsmanship also comes into play here but it's mostly about communicating and establishing a shared understanding of the software and what it's supposed to do.
#3 is just plain old good engineering

The order in which I list those things is not arbitrary. I think that the experience and knowledge of the designers and developers contributes the most to the quality of the software and its design. Collaboration and communication is a close second. Without these, the chances of producing good quality software are slim to none. And if you have good developers and designers, chances are they also practice good engineering so if you have #1 then #3 will most likely follow.

What's a novice like you to do? IMO, good design is a skill and as with any other skill, you need a lot of Practice, Practice, Practice! And Read, Read, Read!

I'm not saying that UML is useless, mind you. By all means, familiarize yourself with the notations. But realize also that there's so much more to good design than just creating diagrams. A diagram is just one tool of many at your disposal for communicating ideas and establishing a shared understanding with others. And there's a limit to the effectiveness of a UML diagram as a tool for communicating ideas. Understand what those limits are so you don't end up wasting your time and effort.

Frankly, I think you would benefit more from reading and learning about the concepts and practices that can help you produce good designs. Learn about Software Design Principles, Software Design Patterns, and Test-Driven Development. To me, these are more important than knowing how to draw UML diagrams.

[Tim Holloway]

Just some random thoughts.

Whenever I walk into a cubicle farm and someone's wall is pasted with a large map of UML diagrams, I tend to snicker. One of the problems with the automated UML tools is that they have no sense of proportion. You end up with acres of diagrams of things not even worth looking at with the important components more or less anonymously embedded in them.

It's pretty well been demonstrated that the more people involved with a project, the more likely it is to fail. Most of the really successful systems I've encountered were the result of a very small number of people. They were often architecturally horrible, but just as often were critical business assets. Code quality may or may not have been good, but the stuff ended up running for decades, so if it wasn't all the elegant stuff that yours truly takes pride in, at least it was functional. Eventually, though, systems like those are the ones that eventually breed disasters (see Fred Brook's "Second System Effect"). The disaster being when the original system has outgrown itself and a whole lot of people get involved in trying to implement a successor. As the saying goes: "An elephant is a mouse designed by a committe".

I do have a lot of tools in my toolchest, as you can see. But I don't necessarily take them all out and use them on every assignment. They're a means to an end, not an end themselves.

[Junilu Lacar]

Tim Holloway wrote:Whenever I walk into a cubicle farm and someone's wall is pasted with a large map of UML diagrams, I tend to snicker.

I tend to be like then and thinking "Somebody has way too much time on their hands..."

[K. Tsang]

For me personally, design is done in my head and through diagram sketches to indicate the flow (see if it's feasible etc).

UML IMHO is more like formalized diagrams that can go with the design spec or documentation.

[Junilu Lacar]

K. Tsang wrote:UML IMHO is more like formalized diagrams that can go with the design spec or documentation.

As with any tool, its effectiveness largely depends on the context in which it is used. I have nothing against UML diagrams per se but I have seen them misused and abused too many times. Back in the early 2000s when tools like Rational Rose were at the peak of their popularity, a lot of people spent an inordinate amount of time creating and editing pretty pictures. A lot of trees were sacrificed at the altar of UML. I even had a go at it at some point but soon abandoned the effort because it was way too much work for my brand of laziness (work smart, not hard). A picture may well be worth a thousand words but there's no way, IMO, that a UML diagram by itself can adequately stand in for the richness of the ideas and information that gets exchanged when people actually spend some time to collaborate and discuss in front of a whiteboard.

[Junilu Lacar]

K. Tsang wrote:design is done in my head and through diagram sketches to indicate the flow (see if it's feasible etc).

Diagrams might be good enough to give you a sense of whether a particular design might be feasible but that's about as far as I would trust them. Unlike designing something physical in nature, you can't just calculate whether or not your still unimplemented software design will hold up in reality. You have to actually write the code and produce working software that you can test and run through its paces. I think that Jack Reeves makes a point along the same lines in his essays that I cited.

This goes back to what I wrote earlier about software design dealing largely with ideas that exist in people's heads. When more than one person is involved in the development of software, the ideas they each have in their heads need to be aligned. This is why collaboration and discussion is important. Diagrams can help in establishing a common understanding between people. See the diagram on this page. Even when you are designing software by yourself, there is still another entity involved: the computer.

This is why I like to refer to the ideas you have about software as "theories." You draw a very rough representation of an idea on the board. It's a picture of your theory of how the software should work. You write code that reflects the idea and fills in the many details that are needed to make the idea a whole piece of working software. The code is the embodiment of your ideas, organized in a way and in a form that can be translated into instructions that a computer can execute. Writing the code and organizing it in a way that makes sense when you read it, as Jack Reeves asserts, is design. Actually creating the software that the computer runs is the "implementation" or "build" part of the process.

You then run the software to see if the theory that you described in code matches how the computer actually interprets what you wrote. This is testing. If the results don't match your expectations, then there's a flaw in your theory, or in the way you described it in the code, or both, so you go back to the drawing and/or the code and tweak things. Then you build and test the software again. Rinse and repeat until you get the results that show that you and the computer are on the same page about what should be done. The computer now "gets" your idea, your theory.

This is the difference between traditional engineering work and software development work. Engineers can plan out their projects to minute detail because it helps them avoid mistakes and rework. Their work (edit:is very imperative -- wrong term) can be done in a prescriptive manner, where tasks are completed more or less sequentially and work follows a natural and logical progression. They work with constraints which are known or can be determined beforehand using well-established formula and parameters. On the other hand, software development is empirical and while we can still define constraints and do some planning up front, and we can follow general principles and patterns, the work we do is much more fluid and involves a greater degree of variability. It's usually not practical or even possible to account for all the different variables up front. We have to work incrementally and iteratively instead, discovering and accounting for the different things that can affect our system as we go.

[Maurice brown]

Thanks guys for the feedback.

Junilu Lacar, i'm going to try to read your articles.