Weighting summary report
This section describes how to check reports for respondent weighting during & after weights have been calculated in QPSMR Companion. For further information, see Weighting respondents for an overview of rim/target weighting.
Assuming that the weights have been calculated to the required precision, you should check the weighting report. For each target it shows:
- Entry name used for the target
- The response number for this target
- The type of target set for this response
- The actual un-weighted number of records with this response
- The usable number of records. This may differ from the actual if fixed or zero weights have been used
- The target set for this response
- The progress which approaches 100% as the calculations are in progress
- The weighted total achieved which should be the target or very close to it
- The ESS (effective sample size) for this response, see below
- The efficiency for this response, see below
- The lowest weight allocated to a record with this response
- The highest weight allocated to a record with this response
- The response text
Most of this information is needed when the weighting has failed, to give clues as to where the problem lies.
Important: The ESS/efficiency must be checked, especially that in the first row (All targets). It is also worth checking the minimum and maximum weights to see if any records are being ignored or given too much influence over the results.
ESS and efficiency
The ESS is a very important figure and shows the effective number of records in the analysis once the weighting has been applied. In addition, it takes into account the “weighting effects”.
If the survey contains 1000 records and the ESS is 750 then we have effectively “lost” one quarter of the data. In addition, if too many rims are used it can reduce the effective sample size to a small proportion of the number actually interviewed. If the ESS is too low it is worth reconsidering the targets applied.
Efficiency is just another way to express the “weighting effect” and shows the ESS as a percentage of the un-weighted total, for example 50% efficiency means that you have effectively only got half the records when weighted than you had before the weights were applied.
The summary shows the ESS and efficiency for all the targets and you should check that none of them are too low.
Saving
You should save the summary report using [Reports] [Summary] and copy paste into a spreadsheet program, or view and save to a CSV file. If you have not saved the summary report when you close the Weight data window you will be prompted to do so.
Failed to reach targets
There are many reasons why the program may fail to reach the targets set. It could be a problem with the targets or the data.