How do I determine difference in medians and confidence interval for this difference

[#1]

I have been asked to report the difference and 95% confidence intervals around the difference for means and medians in a study. It is easy to get these values for means, but I’ve found it difficult to figure out how to do this for medians and non-parametric data.

I am working with two groups of subjects, non-paired integer data.

Any help is appreciated.  I am running JMP 12.

[#2 · Accepted solution]

Hi @strickerp,

Determining the standard error of the median as an estimator has complications, so exact confidence intervals aren't typically defined (though there are approximate formulae around that work well for large samples). The best way to form a confidence interval for such a statistic is by bootstrapping it. Are you using JMP Pro 12 by chance? If so, you can generate bootstrapped confidence intervals about the median difference by using JMP Pro's Right-click bootstrapping:

Using JMP Pro: 

1. Launch Fit Y by X with your variable
2. Go to the Red Triangle > Quantiles.
3. Right-click on top of the reported medians and select "Bootstrap." Enter the number of bootstrap samples you want (I would suggest 5000, which will take some time). You will get a table with one row per bootstrap sample, and columns for the median that was found on each iteration for each level of your X variable.
4. Since you're interested in forming a confidence interval about the difference of medians, select both columns in the table, right click on the row header > New Formula Column > Combine > Difference. 
5. To get your bootstrapped confidence interval, go to Analyze > Distribution, place your new difference score column in for Y and click okay.
6. JMP knows this is a bootstrapped table, so it will return a section titled "Bootstrap Confidence Limits." For 95% intervals, look at the .95 rows of coverage.  

Here's a quick video showing those steps:

 I hope this helps!

@julian

[#3]

If you don't have JMP Pro, you can produce the bootstrap yourself, but it requires the extra steps of sampling your table with replacement so you can compute the median for each group at each iteration. You could this via scripting, but here's how you could go about this interactively by using my Sample With Replacement Add-In (download this first).

1. Launch the Sample with Replacement Add In (Add-In Menu > Sample With Replacement > Sample Now)
2. Select the columns you need for the particular comparison of medians (your Y, and your X factor)
3. Specify at least 5000 for the number of samples, and click OK (leave sample size alone, this is only for over- or -under sampling). You will have a new table that is as many rows as your original table times the number of samples you drew. For a large initial table this could be huge.
4. Launch Tables > Summary
5. Group by Sample Number
6. Subgroup by your Factor
7. Select your Y, then click "Statistics" and select "Median." Click OK. You will now have the same summary table we got from the Right-Click bootstrap, so the steps now are the same
8. Select the two median columns, right-click in the header > New Formula Column > Combine > Difference
9. Launch Analyze > Distribution, place the difference column in for Y, and click OK
10. There will not be a Bootstrap limits section, but you can use the Quantiles to find your coverage limits. For 95% confidence, read off the values for the 2.5% and 97.5% quantiles.

Here's another video showing this quickly: 

Hope this helps!

@julian

[#4]

This is wonderful- thank you so much!