Weighting problems

This section describes some of the common reasons why QPSMR Companion is unable to set weights to achieve the targets set. This weighting problems section also covers why you may get zero weights for some records and other odd results. See Weighting respondents for an overview of rim/target weighting.

Weighting calculations failed

If the weight calculations appear to be taking too long, you can stop them at any time and check for unusual weights and if all looks OK you do not need start again from the beginning.  In addition, you can continue to calculate from where you left off with [Calculate] [Calculate weights].

Furthermore, if you are having problems you should start with less “Significant digits required”; maybe set this to 5 or 6.  If all looks OK you can increase the significant digits and continue to calculate from where you left off with [Calculate] [Calculate weights].

Targets do not add up

If setting targets for single-coded rims, which is the most common type of respondent weighting, it is very important to check that the targets add up exactly.

Rounded percentage targets will not always add up to 100%.  If the targets add to 99.99% or 100.01%, then the program can never achieve these targets. If you require weighted values targets, then you must ensure they add up to the weighted total set for all records.

Missing data

If a rim’s records are not in any response (left blank) and the targets add to 100%, then give these records a weight of zero (0.0) to achieve the targets. This excludes these records from any analysis.

If there is missing data and you want to include them then you need to allocate a target to them.  You can do this by having the targets add to less than 100% and the missing data records will make up the rest.

Conflicting targets

If you have more than one set of targets referring to the same respondents, it is important that the sum of the targets is exactly the same for both sets.  For example if we have two rims:

  1. North small
  2. North large
  3. South

and

  1. North established
  2. North new
  3. South

If the same records are in both sets of “North” then the sum of the first two targets in each rim must be exactly the same.  Obviously, if the South in both is the same records then you would have to make these targets exactly the same, or the Companion program might ignore one of them.

Similar targets

If more than one target refers to almost the same list respondents this can cause zero weights, here is a trivial example:

  • You set target D to 20 and there are 8 records in the data
  • You set target K to 20 and there are 11 records in the data

If the 8 records in target D are all also in target K, then you must fill target K with those 8 records and the other 3 records will get a zero weight.  This will take a lot of iterations as these weights get smaller and smaller.

It can also cause failure, here is a very similar example:

  • Target D is set to 20 and there are 8 records in the data
  • Target K is set to 17 and there are 11 records in the data

If the 8 records in target D are all also in target K, then the program cannot reach the K target without allocating negative weights to the other three.

Companion will never allocate negative weights.

Here is another trivial example that shows how a similar problem can give rise to very large weights:

  • Target D is set to 20 and there are 8 records in the data
  • Target M is set to 30 and there are 9 records in the data

If the 8 records in target D are all also in target M, then target M must include those 8 records, and the other record will get a weight of 10.0 to make the total up to 30.

Runaway weights

If you are using percentage targets, then it is important to “tie down” the weights in some way.

By default Companion includes a target for all the records in the data file so that the weighted total will be the same as the un-weighted total.  Without this, the weights could get larger and larger, or smaller and smaller, whilst the weights are adjusted to approach the percentage targets set. You can change this first target to give a value that you want the weighted total to be.

You could set this first row to ignore if you have tied down the overall weights using some other value targets.

Finding weighting problems

It is not always obvious why a weighting scheme cannot be achieved.  If the calculations are taking a lot of iterations, then stop the run and look at the summary and the data. If you have weighting problems, you first look at the summary at the top of the screen.

  • The lowest Progress percentages will show the targets that the calculations are struggling to meet.
  • Look also at the Min and Max figures for extreme weights.
  • Check the Actual figures to make sure that the rims add up to the total.

You may find it easier to copy and paste the report summary into a spreadsheet program to check it. If you have very small or very large weights you can look at the data at the bottom of the screen.  It may be helpful to click on the weight column to sort it into ascending order. For the extreme weights you can look for patterns in the rim values alongside to get clues as to where the problem or conflict lies.