So, what is it about large projects that is so difficult? Frederick Brooks pointed out in his seminal work the Mythical Man-Month that small teams are much more effective than large teams. But why? What is it about large teams, and large efforts, that is so intractable?
The agile software development movement is a reaction to much of what is perceived as dysfunctional about traditional methods of software development, and the use of large, process-intensive teams creating documents is one of those practices that is rejected by the agile movement. The feeling is that projects are better off if one simplifies how people coordinate their work. The best way to do this is to put the entire team in the same room and make sure that they talk. Documentation is largely dispensed with. The results are impressive. My former company Digital Focus (now Command Information) built its business around this approach.
Documenting is a great aid for the documenter. As the author of four lengthy books, I can tell you the process of writing compels the author to organize one's thoughts. In a software project, the creation of documentation should never be seen as a milestone of progress. This is because documents are not actionable. Documentation of requirements or business processes does not convey knowledge. At best, it is only a starting point. Disbanding a team that assembles requirements as documents squanders the knowledge that team accumulated.
Only knowledge is actionable. Knowledge is the result of humans interpreting information from documents, from the explanations of others, and from their experiences and building understanding in their minds over time. The phrase people, process, and technology that one often hears is therefore misleading and dangerous. The flaw is in the concept of process, which is often taken to mean process documents.
I have seen too many projects planned around the concept that you just have to have someone create documents, and then bring in people to execute based on those documents, using some provided technology. It does not work that way, and that type of thinking leads to failure. You need to focus on the knowledge that is created as a result of creating documents, systems, and the other durable elements of your organization. The act of creating information creates knowledge, and that knowledge is the core of your business process; and that knowledge resides inextricably in those individuals.