Let me caveat this by saying that I'm a fan of some of the more recent development trends like TDD, Agile, and what these changes have done for software engineering generally. As a software engineer, I know first-hand the temptation to code for every eventuality, especially when they seem so obvious sometimes. These types of approaches help make sure that the developer stops developing at some point by providing a terminating point. In TDD, you start with a degenerate unit test, code until the test passes. Then you incrementally add to the test until you match what's in the requirement (and no more). Once that final test passes, you stop writing code. In Agile, the team agrees on what something called the MVP (minimum viable product) is, and commits to delivery in a sprint or two. That time-boxes everything so that the there literally is not time to do much in the way of over-engineering.
In spite of the massive cost-saving benefit of making sure that we software engineers don't stray off of the beaten path, there's a down-side. That down side is this: when are you supposed to do system design? I would suggest that with competing priorities, evolutionary designs simply don't work in an organization. This shouldn't be a surprise, as Martin Fowler suggests that evolutionary design, as done in practice, is actually done in a way that leads to 'chaos' [Fowler, Is Design Dead]. I've seen this play out in such a way that any attempts at refactoring were actively discouraged by the senior engineers and developers in an organization! When that happens, any chance of a reasonable design evolving is definitely dead.
Assuming that's a worst-case situation, what actually happens in real-world software engineering? A product owner or some customer advocate presents a set of requirements, which then the software engineering team is responsible for delivering.
In the best case, the team can do an estimate, and then propose the estimate to the product owner. Let's be realistic and assume that the estimate is actually an under-estimate by, say, 50% (that is to say, the actual work will take 150% of the LOE suggested by the engineering team). Simply put, the engineering team says 2 sprints, the actual LOE is 3 sprints. At the end of those 2 sprints, there's another feature that the team is expected to start work on. Then, when does this alleged design work happen? At the end of 2 sprints, the team is realistically still finishing up with the previous work, and so has no actual time to do design work.
In the worst case, the product owner has already committed a delivery timeline to the customer. This example doesn't require a lot to show that design won't actually happen. The engineering team is not in a position to do any real design, and has likely started out behind schedule already.
The lesson here is straightforward: the only time we engineers have to do real design is time that we carve out for ourselves.
To conclude, there are many forces acting against thoughtful design in the newer development methodologies. I've specifically examined Agile as it is often implemented, but you can read the same thing in Martin Fowler's discussion on XP. What he closes with, and I wish to re-iterate, is that without the will, design will simply not happen - there are too many forces pushing in the opposite direction (probably as a reaction to the design-heavy requirements analysis - not going to bother to complete this argument though). So, as engineers, we can't assume that good design will arise by KISS, YAGNI, DRY, or any other cute phrases to help us simplify. We must intentionally bake design into our DNA.
