topic Re: Selecting the correct sample size in Discussions

Selecting the correct sample size

albiruni81 — Tue, 05 Dec 2017 09:18:43 GMT

Hi All,

Lets assume I have a measurement of 30 location on a particular table top. I would like to sample the measurement of these 30 locations using only 5 samples, is JMP able to assist me on determining which measurement location should i be selecting in order to get the closest representative of the overall 30 measurement location

Irfan

Re: Selecting the correct sample size

Byron_JMP — Tue, 05 Dec 2017 14:53:58 GMT

This is the classic problem that everyone designing experiments (DOE's) is struggling with.

Lets say you have a square with 36 tiles.

Do you know if the plane of the tiles is tilted, up on a side or on a corner? is the plane warped in one dimension? is the plane curved like a bowl? How big is the change across the surface?

How reproducible are your measuments of the tile heights? is the reproducibilty larger less than variation from the plane that is fit through the tiles?

If your plane tilts a lot, and your measurement system is good, then any 5 tiles (selected in a pattern like the 5 on a die) is going to be just fine, and you might not even need the 5th center point.

If you don't know anyting about the tilt or curvature of the surface, and if your measuremnt system is poor, it might be a good idea to select big ranges, and maybe a center points along the edges of plane along with the corners.

JMP can help you pick a good set of experiments to explore your surface, but you have to pick the ranges and decide if you want to be able to detect warp (interactions) and curvature (quadratic terms) in your experment.

What do you want to know? And What can you Measure? Two key questions.

Re: Selecting the correct sample size

Kevin_Anderson — Tue, 05 Dec 2017 15:56:50 GMT

Hi, Irfan!

Just to pile on to Byron_JMP's line of reasoning: remember that effective experimentation is usually iterative...that is, it often pays to ask smaller questions in a number of experiments whose answers can inform the subsequent experiments. Answering the ultimate question in the first and only experiment is rarely as efficient or effective as an iterative method.

The answer to your specific question is "yes"...JMP can assist you. Start with a specific, measureable objective. Continue from there with measurement capability. Then, deal with other questions about your tabletop.

Re: Selecting the correct sample size

albiruni81 — Wed, 06 Dec 2017 01:12:48 GMT

Hi Byron,

Lets assume that all measurement are accurate and repeatable. I would like to select 5 tiles for measurement but the question that I would JMP to answer is which 5 tiles should i be using in order to give a good distribution from the whole population

Is there a plot that I can use in JMP to analyze this

Rgrds

Irfan

Re: Selecting the correct sample size

Kevin_Anderson — Wed, 06 Dec 2017 16:38:40 GMT

Hi, Irfan.

Here's a warning: The road to Grievous Error is paved with assumptions about measurements being accurate and precise.

But, given your assumption, I think we need considerably more information to help effectively. From the information we have so far, you may be better served by a psychic hotline! :)

What is your objective? (It needs to be more specific and measureable than "...give a good distribution from the whole population"!) Why are you interested in doing this sampling? Are you wishing to describe this one tabletop, or do you wish to describe future tabletops? How much data do you have from all 30 locations on your tabletops? (In other words, have you performed a Passive Data Collection experiment?) How were those data gathered? I'm sure there are many more questions that would be helpful...

JMP has many analyses and plots that can help, but it's your responsibility to give it the right data.

Re: Selecting the correct sample size

Peter_Bartell — Wed, 06 Dec 2017 17:12:38 GMT

Irfan:

To pile onto the sage wisdom imparted by my colleague @Byron_JMP and @Kevin_Anderson whether you are gathering samples for what Dr. Deming calls an analytic or enumerative study, recognize that sample size, and strategy (random, stratified, etc.) are only useful for controlling ONE category of risk associated with the study. The category of statistical risk. JMP has a fairly robust set of sample size determination platforms to assist with this issue.

But there is another category of risk associated with these types of studies that you need to consider...I call it representation risk. This is a category of risk that is NOT effected in the least by things like alpha and beta risks, assumed population(s) variability, or difference to detect...which is what you need to determine sample size for many types of sampling plans. Representation risk is ONLY managed and dealt with using domain expertise. Both Byron and Kevin give you many, many issues that are associated with managing representation risk. Keep in the mind the total risk associated with any conclusions you make from the study are is the SUM of both the representation AND statistical risk.

In all my years of using statistical sampling plans to make these sorts of determinations...it's been my experience that representation risk SWAMPS statistical risk. Worse yet, representation risk is very, very difficult to quantify in a meaningful manner. So none of us are saying ignore statistical risk...just recognize it's limitations for studies such as you are contemplating.

Re: Selecting the correct sample size

Kevin_Anderson — Wed, 06 Dec 2017 18:03:59 GMT

Irfan,

Here's another way to think about this...

Suppose these are the patterns of variation that you would like to detect in your tabletops:

How would you sample to detect these patterns? Hint: you may need more than 5 sites! If you can't afford more than 5, some trade-offs will need to be made.

I think this is the representation risk that Peter and Byron are talking about...this is the domain expertise that is required. What are you trying to do with your sample?

By the way, this is an image of different Zernicke polynomials!

Re: Selecting the correct sample size

albiruni81 — Thu, 07 Dec 2017 01:16:02 GMT

Hi All,

Thanks for the reply, yes in any manufacturing plant normally one of the way to improve cycle time is to reduce the number of samples that we have to do. That is the foundation of the question that I have. From a standard of 36 measurement location, how do we statistically determine what is a good reduction of measurement location that we can do which best represent the overall 36 measurment location since the 36 measurement location might be an overkill in some instances. The reduction in the measurement sample will improve the processing time for the measurement hence improve the output of the manufacturing facility

I hope this clarifies the question that i posted earlier

Rgrds

Irfan

Re: Selecting the correct sample size

stephen_pearson — Fri, 08 Dec 2017 09:23:40 GMT

Sounds like for a limited period you are going to need the data from all the sites; until you understand where the variability that most commonly causes issues is located amongst the sites (centre, edges, corners, etc). This may change with time so you might need to revisit extended checks on a periodic basis.
What is your appetite for risk? Do these sampling checks determine if a part is defective, or would you carry on anyway but the result is less profitable?

You might want to consider generating a simple model that weighs up the profit from increased throughput against the risk of less measurements.

Re: Selecting the correct sample size

albiruni81 — Sun, 10 Dec 2017 09:56:03 GMT

DOes it mean that JMP is not able to give me a better way how to sampled my measurement location?

Re: Selecting the correct sample size

stephen_pearson — Mon, 11 Dec 2017 09:01:08 GMT

For a given problem, JMP will be able to choose the most efficient sampling scheme, provided you can adequately describe the limitations. e.g. the samples need to come from the edge, middle and corner plus more than 1 measure is required at each location to understand variability OR more locations is valuable, etc.

The challenge is it will probably require more sampling than you originally considered to have a good chance of success (1, 5 and 10 are very common human choices of sample size). You need to clearly define WHY you are sampling and WHAT you consider to be success.

Re: Selecting the correct sample size

albiruni81 — Mon, 11 Dec 2017 09:32:32 GMT

Hi Stephen,

Above is an example of my measurement location. If I were to select only 5 location to measure, how do I select which 5 location measurement that gives the best statistical representation of the all 23 measurement location. Is there a way to find out ?

Rgrds

Irfan

Re: Selecting the correct sample size

stephen_pearson — Tue, 12 Dec 2017 08:55:12 GMT

Are all locations considered equal? Is number 12 (the centre) special in any way? One sampling scheme would be sampling at random from the different 'zones'. Are the locations rotatable (i.e 1 = 9 = 23)? You can run simulations based on historic data to compare how well any sampling scheme estimates the 'true' value if all locations are considered.