Discussions
Solve problems, and share tips and tricks with other JMP users.
Hello @Alain04,
Welcome in the Community !
I completely agree with @P_Bartell about the size and representativeness of your sample. I suppose the dataset displayed here is just a smaller version of a bigger one, but you should take care about the data collected (and the generation process) to make sure your comparison is reliable and "fair". Start first with visualization before doing tests, the visualization (using box-plots for example) might be sufficient for your needs.
Looking at your topics, there might be several options depending on your objectives :
- As @P_Bartell suggested, you could realize an equivalence test, to analyze if the mean difference stays in a practical equivalence interval or if the two measuring systems are not comparable. However, as the Equivalence test is based on t-tests, there are some assumptions to verify (https://www.jmp.com/en_ch/statistics-knowledge-portal/t-test.html) :
- Continuous data
- The sample data have been randomly sampled from a population (so it goes back to the point of the representativeness of your sample).
- There is homogeneity of variance (the variability of the data in each group is similar).
- The distribution is approximately normal (which might not be the case with percentages...).
As your dataset shown here is too small to verify some of the assumptions, I can't help you on this point.
However, if you want to proceed with this analysis, create a formula column with the difference between the two measures, use the Distribution platform to display the distribution of the Difference column, and from here, you can click on the red triangle and launch an equivalence test with the practical difference of 3 considered as equivalence (and hypothesized mean = 0, confidence level = 0,975 if you want the combined result of the two tests to be displayed at confidence level 0,95). Based on your dataset, here is what you could expect :Remember that this equivalence test only test mean difference, so it won't provide information about a difference of variance between the two measuring systems (which might be very informative to highlight a gain in precision for example).
- Using your datatable in a stacked format, you could have a different analysis option using the platform Fit Y by X. Using your process as X and measure as Y, you can test difference in means (with Student t-test or non-parametric test, depending on the distribution of your data) as well as equality of variance ("unequal variances") between the two measuring processes using options from the red triangle. Here are the type of informations you can collect :
Tests for equality of variance (with Welsh test for difference in means):T-test for difference in means :
Also the visualization displayed in this platform might be sufficient to understand if your results are comparable, in terms of means and variance :
-
You could also use the Measurement Systems Analysis platform to compare the two processes and see how consistents are the results. The visualizations available in this platform are quite helpful to compare the two processes :
Using the Gauge R&R evaluation, you can see (with the small dataset provided) that you may have more variance due to the repeatability than due to a change of measuring process :
These are some options available to analyze your data, but there might be other additional ones. Again, it all depends on your objectives and the data collected.
I join the dataset used to test the three options with the scripts saved so that you can evaluate the different options proposed.
As @P_Bartell, comparing the two processes at a specific time might be informative, but you have to make sure the measuring process is stable and controlled in time.
I hope this response will help you,