Collect Measurement Data

The data necessary for analysis are obtained and checked for completeness and integrity.

• Obtain the data for base measures. Data are collected as necessary for previously used as well as for newly specified base measures. Existing data are gathered from project records or from elsewhere in the organization. Note that data that were collected earlier may no longer be available for reuse in existing databases, paper records, or formal repositories.
• Generate the data for derived measures. Values are newly calculated for all derived measures.
• Perform data integrity checks as close to the source of the data as possible. All measurements are subject to error in specifying or recording data. It is always better to identify such errors and to identify sources of missing data early in the measurement and analysis cycle. Checks can include scans for missing data, out-of-bounds data values, and unusual patterns and correlation across measures. It is particularly important to do:
  • Test and correct for inconsistency of classifications made by human judgment (i.e., to determine how frequently people make differing classification decisions based on the same information, otherwise known as “inter-coder reliability”).
  • Empirically examine the relationships among the measures that are used to calculate additional derived measures. Doing so can ensure that important distinctions are not overlooked and that the derived measures convey their
  • intended meanings (otherwise known as “criterion validity”).