Document Reviews


General notes
The following description is based on the comprehensive SEM Guidelines "Reviews" for planning, executing and documenting reviews, which is published by PSE QM. The reviews are supported by separate review forms for comment technique and session technique (Word for Windows files).

What is a review?
This is the most common definition:
A review is a systematic, critical and documented check of development results at the end of defined work stages. It has the goal of finding errors.

What is a document review?
In this case the reviewed development result is a document such as:

  • Tender
  • Specifications of Definition and Design phases
  • Project plan
  • QA plan
  • And other development results which must be reviewed or are recommended to be reviewed.

What does end of a defined work stage mean?
A review is only expedient if the document is actually ready, i.e. has the degree of maturity needed for checking. This is generally the case if a work stage has been concluded (e.g. Producing the QA plan) and the author himself considers the document ready and correct, i.e. cannot identify any remaining errors. The next defined stages can then be: Review of the QA plan, Revision of the QA plan and Release of the QA plan.

Discussing half-finished documents and development meetings cannot be considered reviews!

How do I conduct the check?
It is advisable to adopt a systematic approach, i.e. to choose a tried-and-proven review method. Popular and proven review methods are (including a combination of these)

  • comment technique and
  • session technique.

What form does a comment technique review take?
The documents are first distributed to the reviewers:

  • Document (preferably with line numbers on the paper)
  • Reference documents (what is it being reviewed against?)
  • Project standards and specifications (if available)
  • Record templates
  • Other useful documents

The reviewers work through the document on their own and pass their findings to the sender (generally the author, initiator, facilitator, inviter, etc.). The latter evaluates (possibly in consultation with others) the comments and the author incorporates the changes (if the necessary corrections are very extensive, it is advisable to conduct a follow-up review on a smaller scale).

What form does a session technique review take?
The documents are first distributed to the reviewers. At a fixed time for the session, the document is worked through jointly using the records of the review participants. The facilitator / keeper of the minutes then draws up the minutes for the session and evaluates the errors. The author now knows the extent of the improvements which are necessary.

A highly formalized form of the session technique is the intensive inspection based on M. Fagan (this is far more effective and efficient than the traditional session technique but requires a relatively long period of time). The number of participants is restricted to 3 - 6 who are assigned precisely defined roles. These are usually:

  • Author
  • Reader
  • Tester
  • Facilitator

In addition, up to 3 further inspectors can be employed who perform specific roles such as interface tester, user, standards tester, etc. and who check the document from the perspective of their roles. Such a session should not exceed 2 hours. This inevitably restricts the extent of the documents that can be worked through during a session and may require the document to be checked in several sessions.

Which method should I choose for my document review?
For particularly "tricky" documents or parts of documents, the intensive inspection is highly recommendable. The following decision table should be used for other documents:

For comment technique:
  • Many participants are possible
  • No dates for a meeting need be agreed
  • No "location" problems
  • No time taken up in meetings
  • No time taken up in traveling
Against comment technique:
  • No dialog
  • No synergies through meetings
  • Low error identification rate
For session technique:
  • High error identification rate
  • Misunderstandings can be resolved quickly (documents)
Against session technique:
  • Sessions should not be too large
  • The "location" problem arises (D, USA, A, etc.)
  • Dates need to be agreed
  • Negative psychological components can arise (pressure from superiors, antipathy, etc.)
For intensive inspection:
  • Role perspective promotes error finding
  • Thorough check of the document against specifications
  • Effort per error identified is the same as for a session review, but more errors are found
Against intensive inspection:
  • Low review speed
  • Training available?

Why documented check?
Irrespective of the method selected, each review must be documented by a review report:

  • To ensure that the author has an error report for eliminating errors which has been accepted by everyone involved.
  • To ensure that all participants and persons informed have a document setting out experiences which can then be used for planning and executing further reviews.
  • To ensure that errors and the causes of errors can be analyzed.
  • In order to be able to present a development result which constitutes a record of quality to ISO 9001 in order to be able to substantiate a check if required.

A review report should consist of the cover sheet complete with error statistics, error list and, if appropriate, an analysis report.

Who does the document review affect?

  • The author: I want to know that my result has been approved by other participants and that as few errors as possible will be passed on to the next development phase.
  • The team: We want to ensure that we can build on something secure in the next phase, we have also a basis for further work in the form of a result which has been accepted by the customer, we know the entire subsystem as a result of the reviews and can work more successfully in future, we have learned to discuss things with each other on a professional level.
  • The client: The document has been checked and accepted by me. I know what the situation is. I know what I will have to pay for.
  • The contractor: I have had the document examined by experts. I expect no more pitfalls.

What needs to be done if too little time is available for reviews?
Set priorities: Where is it particularly important to invest time and money in document reviews?

  • In early documents (the effort for eliminating errors is still relatively low)
  • If the consequences of errors are serious (e.g. security-related software, client's image, key document)
  • If there is a high probability of error (e.g. authors new to this field (subject area and tools), communication problems (location and language), complex subject "that has always been the source of many errors", etc.).

You should consider, however: It takes an average of one hour to identify an error (calculated on the basis of 1100 reviews in the PSE); an hour that you can invest in reviews will save you approx. 5 hours up to the point of acceptance.

Where can I expect the most errors?
Experience shows that errors in the development documents are concentrated on a few areas:

  • Forgetting individual cases in a CASE-statement (20%)
  • Ignoring extreme conditions (17%)
  • Misunderstanding earlier documents (12%)
  • Wrong queries (12%)

In addition to the actual review, it is also advisable to analyze the review
Following a review, a brief analysis of the review should be conducted in order to draw conclusions about one's own behavior and the actual errors (you can find further information about this on the review form). In terms of the review process, this means:

  • Were all aspects of the planning correct (participants, deadline, material, etc.)?
  • How did the review itself proceed (concentration, error finding, rooms, etc.)?
  • How was the error evaluation arrived at (consensus, laid down by the manager, etc.)
  • It is also worthwhile noting down positive aspects in the analysis report!

With regard to the development process, you should ask the following questions:

  • What errors keep on recurring?
  • Why?
  • What could be done to counteract these (measures and persons responsible)?

If the sources of error are eradicated, there should be nothing to stop the development process from being improved.


Siemens AG Österreich, Program and Systems Engineering PSE
Contact: stdSEM Webmaster
Last modified: 08/27/98 15:42
Copyright © Siemens AG Österreich 1997. All rights reserved.