How likely is it that simple randomization resulted in particular group sizes in a RCT?

I was the editor for a randomized trial submission to our journal which had a total sample size of 132. The authors used simple randomization (i.e. no restricted randomization, such as blocking or minimization), and they ended up with 58 in one group and 74 in the other. This is pretty clearly a moderate imbalance, and not really desirable in a clinical trial.

Question

The question I had was: how probable was it that this result was due to chance alone? Could there have been another explanation for this disparity, such as a faulty randomization generation algorithm? I satisfied myself that the group size disparity would be acceptable if the probability that it could have occurred by chance alone was > 5% (the typical value used for statistical significance).

Solving the problem

We can solve this problem pretty trivially using Stata, using the -bitesti- command. This command uses the binomial distribution to calculate exact P values.
-bitesti- expects three numbers:
  1. the total sample size (132 in this example)
  2. the size of one of the groups (it doesn’t matter which — we’ll use the 58/132 patient group)
  3. the expected proportion in each group (in this case, we expect half of the patients in one group and the other half in the other group, so this proportion is 0.5, at least asymptotically)
Voila_Capture 2015-05-26_11-12-54_AM
The P value I’m interested in reflects the probability that group sizes would have been smaller than 58 (or, equivalently, larger than 74 in the other group) simply due to random chance. This probability is 0.19. Therefore, it is most certainly possible that these groups sizes could have arisen by chance alone using simple randomization.

 A Related Question

How different would the group disparity have to be in order to be statistically improbable (i.e. unlikely just due to chance alone)?
b2

The answer is 54 in one group and 78 in the other. This would have a probability of occurring, by chance alone, less than 5% of the time (p = 0.045).

How about a confidence interval-based approach?

Although the -bitesti- command does not give us a confidence interval, we can obtain one easily using the -cii- command. The -cii- command can also be used to assess the probability of a group imbalance occurring by chance alone when using simple randomization:
b3
You see that the proportion we expect (0.5) is included in the CI, so the groups sizes of 58/132 and 74/132 could have arisen by chance.
However, using the same command with groups sizes of 54/132 and 78/132 excludes 0.5 in the CI, thereby ruling out chance alone as an explanation of the disparity in group sizes:
b4

Further Reading

Advertisements

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s