Randomization Test Add-In

2 Kudos

Thanks to Don MacCormack donMack@jmp for his long hours, patience and persistence in making this add-in come to life!

_______________________________________________________________________________

Basic Use Instructions

(Use the View as PDF option on the side panel to save these instructions as PDF.)

Get Data

1. Select the type of test. Note that additional details on the particular tests are provided at the end of this document.

2. Select the JMP data set to use. The data table opens in the background (behind the add-in), and available variables are displayed below.

Notes: The add-in links to a number of data sets in the JMP Sample Data Directory. Look at the data table opened in the background to better understand the data - all native JMP features are available. The Use Other Data button allows you to import your own data set (however, the data set is not permanently added to the list).

3. Select the variable(s) of interest. A block plot (similar to a dot plot) and summary information for the variable(s) display under Sample Information (on the far right).

4. Click on the gray disclosure icon to hide the Get Data panel until you need it again.

Simulation Controls

Specify:

1. the number of resamples to generate (the default is 1000),

2. the resampling method to use (the default is resampling with replacement),

3. what to calculate for each resample, and

4. the hypothesized value (click Tab or Enter to accept this entry).

Click go.

Note that options update based on the test selected. To see descriptions of available resampling methods, click on the question mark.

Results

1. Summary statistics for the resamples are provided next to Simulation Controls.

2. A block plot of the selected statistic for the resamples is produced, with a vertical line drawn at the observed value of the original sample statistic.

3. To display the area in the tail or tails (the empirical p-value) for the observed sample statistic, use the options under Tail Selection.

4. To change the value(s) or the proportion(s) in the tail(s), use the options under Tail Info or click and drag the grabber boxes (on the solid lines). Note that a warning appears if the actual proportion out is different from the specified proportion out (this can happen if there are multiple values at the specified proportion - click on Why is this different from Proportion Out? for details).

5. Click on a point in the block plot to see the corresponding resampled data under Selected Sample.

6. Click on a point any graph to display its value.

Other notes:

Use Reset under Simulation Controls to start a new simulation. If you change the resample method, statistic, or the hypothesized value the simulation will also reset.
Use Reset Tails under Tail Selection to reset tail selections and return to the original simulation graph display.
Use Make Table under Tail Selection to make a new data table with the resampled statistics.
Use Make Table under Selected Sample to make a new data table with the resampled values for a selected resample.

_______________________________________________________________________________________

Use Notes for Particular Tests

One Quantitative Response:

Hypotheses can be tested for means or medians.
The only resample method available is resampling. How is this done? Original sample values are adjusted so that the distribution is centered under the hypothesized value (by adding or subtracting the difference between the observed and hypothesized value). Then, samples are drawn, with replacement. Each resample is the same size as the original sample.

Two Unpaired Groups, Quantitative Response:

Select one continuous Y and one categorical X.
The number of levels and labels for the categorical variables are displayed at the bottom of the Get Data panel. If a variable has more than two levels, when you select the variable you will be asked to select the levels to compare.
If the values for the two groups are stored in two separate columns, use the Separate Ys option. (Note that, in order to conduct a formal test for two means or medians for unpaired data in JMP, the data need to be stacked in one column with the groups labels stored in a second column).

Hypotheses can be tested for the difference in the means or medians.
Resample methods include resampling, shuffling, and shifting.
- Resampling: For each observation in the data table a Y value is randomly selected (with replacement).
- Shuffling: The group levels are shuffled (randomized).
- Shifting: Original values are shifted so that the group centers are equal prior to resampling (with replacement).
Observations in the selected sample will be color-coded according to the original group.

Binary Outcome Event:

If the data are pre-summarized (with a counts or frequency column) use the Summarized option and select an X variable to specify the counts for each category.
If the data are unsummarized, with one row per observation, use the Raw option.

To enter your own values, use Hand Enter Summarized Data to enter the number of trials and the number of successes.

Random resampling with replacement is the available resample method. (When summarized data are used, a table is generated in the background with the number of observations in each group reflecting the original counts, and resamples are drawn from this table.)
Hypotheses can be tested for the proportion of successes or the number of successes.

We appreciate your patience and support as we continue the development of this add-in. We'd like to thank Robin Lock for his guidance and direction in the design and development of the add-in, and Roxy Peck, Chris Malone, and countless others who have provided initial feedback. Thank you all!

We strive to keep improving. For feedback or suggestions, please add a comment below or contact mia.stephens or donMack@jmp.

Revision History:

March 2016: Fixed a bug in the two-unpaired groups, separate Ys option. Thanks to Pat Humphries for pointing this out. Beta 3 posted.