The Fog of Software Development

Reflecting on my childhood, I recall playing The Legend of Zelda: A Link to the Past on my Nintendo. In the game's early stages, the map is shrouded in fog, leaving players uncertain of their objectives. Initially, the map appears as a vast, unclear expanse, except for the area where the player is positioned. As you explore, the map gradually becomes clearer, and your understanding of the quest deepens. You undertake smaller missions, level up your skills, and enhance your weapons, enabling you to progress in the game.

At first, navigating the game proved challenging due to its ambiguity and the confusion regarding required actions. Completing it took me a month on my first attempt, but on my second try, I finished in under a week. My comprehension of the game evolved significantly from the beginning to the end.

Developing software resembles poker more than chess; it is a game played with incomplete information, requiring both skill and a bit of luck. Much like examining history, you only gain clarity about outcomes after they unfold. In hindsight, events seem inevitable, yet at the moment, countless possibilities exist.

Creating Software: A Challenging Task

Hofstadter's Law states: "It always takes longer than you expect, even when you take into account Hofstadter's Law." In software development, the process often takes longer and proves more complex than anticipated. This truth is amplified in the context of software projects.

Why is it so challenging?

Sales teams and customers frequently present high-level requirements that sound straightforward, but they often only cover about 50% of the necessary work. Both parties desire quick turnaround times and appealing cost estimates—figures that, while reassuring, do not accurately reflect the time or resources needed. This is the estimate that everyone wants to hear, except the developers.

Typically, no one factors in potential issues such as bugs, misguided software designs, or personnel changes—common hurdles in software development. Even optimistic estimates can still be overly ambitious, prompting plans for multiple development teams to work simultaneously. While this approach may appear to hasten the project on paper, it often collides with Brooks's Law: "Adding manpower to a late software project makes it later."

Adding more personnel is a common solution for a lagging project, but it can also be implemented from the outset to expedite delivery. However, in practice, increasing personnel, time, and requirements raises the likelihood of complications.

Enterprise projects are not merely twice as complex as smaller ones; they can be ten times more challenging. The necessity for better development processes—such as standards, code reviews, documentation, source control, and DevOps—grows with the complexity of the code and customizations. Effective communication and coordination become paramount.

Developers engaged in enterprise projects must possess superior skills, as the risk of inadvertently obstructing one another increases with larger teams. The difficulty in creating software is precisely why developers command high salaries. If software creation were simple, everyone would be able to do it, and projects would not experience delays.

Code is interconnected and dependent. With numerous developers altering, adding, and deleting code simultaneously, conflicts are inevitable. Software projects are not late; they are simply underestimated. In software development, obstacles are part of the journey. Uncovering requirements, adapting designs, and modifying software are essential steps in crafting the right product.

Nothing in Software Development Is as Straightforward as It Appears

Users often lack a clear understanding of how software should operate and tend to express how it should not work after reviewing their requests. The obstacles encountered are crucial to the path forward.

"Software and cathedrals are much the same; first we build them, then we pray." This anonymous quote encapsulates the essence of software development—the challenges we face become part of the creative process. Errors, missteps, and the creation of flawed software are inherent to building something worthwhile.

No one fully comprehends how the software should function at the outset, which accounts for the frequent inaccuracies and omissions in requirements. Discovering a bug serves as valuable feedback, indicating that changes are necessary. The more issues you identify before reaching production, the better the outcome.

Next reads:

Developers Should Celebrate the Challenges of Software Development

The 10 Commandments for Thriving in Software Development

The Lessons Developers Learn Through Experience

Related articles:

Richard Feynman on the Importance of Inquiry

Egoless Programming: Distinguishing Identity from Profession

A personal account of the speaker's journey from novice to proficient software engineer, highlighting key lessons learned along the way.

A condensed history of four decades in software engineering, summarizing invaluable insights and experiences in just 19 minutes.