Subscribe Bookmark RSS Feed

Significance of p-values for Sign test for paired results


Community Trekker


Oct 10, 2012

I'm comparing detection frequency of analytes in 150 paired stream samples using two different analytical methods. The results are provided in two columns with concentration reported for each method in ng/L. Nondetections are reported as <RL where RL is the reporting level. I've recoded the results with a 0 for a nondetection and a detection as a 1. I used the matched pair platform (jmp 9.0.3) to compare differences in detections.

We want to know whether one method has significantly more detections than the other method by analyte. I planned to use the non-parametric sign test because we only want to compare detections. The results for one analyte are shown below. The mean difference shows there are about 10 more detections (0.0667 x 150) in 2437 than in 2033 method.

1)  The sign test p-value for 2-sided test is not significant (Prob ≥ |M| 0.0755), while the one-sided test is (Prob ≤ M 0.0378). Also the sum of the one-sided p-values does not equal 1, as in the case of the t-test. Can this be correct?

2) Is there a reference or any information on the Test Statistic M?

3) Is the sign test rather than the t-test or Wilcoxon signed rank test the best approach with all data either 0 or 1? Any suggestions on a better way to do this analysis?

Matched pairs:

Difference: detection2033-detection2437


detection2033 0.18667 t-Ratio -1.98016

detection2437 0.25333 DF 149

Mean Difference -0.0667 Prob > |t| 0.0495

Std Error 0.03367 Prob > t 0.9752

Upper 95% -0.0001 Prob < t 0.0248

Lower 95% -0.1332

N 150

Correlation 0.50774

Sign Test


Test Statistic M -5.000

Prob ≥ |M| 0.0755

Prob ≥ M 0.9855

Prob ≤ M 0.0378

Wilcoxon Signed Rank


Test Statistic S -67.500

Prob>|S| 0.0476

Prob>S 0.9762

Prob<S 0.0238