Design of experiments when using commercial samples

[#1]

Hi!

Is there a way to do some version of DOE when relying on commercially available samples?

Since the samples should be purchased from local shops, it is not possible to make a DOE around the production processes, but maybe I could still use DOE to get an idea about how many samples are relevant to include etc.? 

Has anyone experience with that?

Thank you so much!

[#2]

My first reaction is what do you mean by DOE?  Are you asking can you evaluate the variation from commercial samples?  The answer is yes.  More likely this is directed sampling rather than specifically manipulating factors in an experiment.  Could you provide a more specific example of what you are talking about?

[#3 · Accepted solution]

Thank you for your response.

I will try to elaborate:

I want to buy different products in the supermarket, for instance different types of yogurts. In the lab, I will then measure many different parameters related to taste, physical properties as well as chemical properties in these yogurts. I would like find correlations between the measured parameters, such as a high sugar content correlating to a sweet taste or maybe something like high fat and low protein correlating to a smooth mouthfeel (or whatever).

I am struggling with the fact that I cannot "control" the samples, as they need to be bought, and I still want to draw statistically valid conclusions. So I am wondering if there is a way to make a design of experiment for that.

Thank you for your help

[#4 · Accepted solution]

Generally no. You can't design an experiment via the sampling method you are contemplating for say the chemical and physical properties...you are collecting what is often called 'happenstance data'...You get what you get. But that doesn't mean you can't model the responses as a function of the measured characteristics of the samples you purchase. Any number of modeling techniques could be used depending on the factors, their levels, the responses, and what you are trying to accomplish from a practical point of view.

If push came to shove I could see a way to 'design' an experiment using categorical factors like yogurt type (Greek vs. Non Greek) or Fruit type (Blueberry, Peach, Plain) etc...but that may or may not fit with your overall knowledge requirements.

[#5 · Accepted solution]

To add to Pete's comments, I think you can accomplish understanding measurement system variation and other components of variation using sampling.  The sampling can be nested (hierarchical), systematic or crossed depending on what hypothesis you are investigating.  For example the sampling could include: within container, between container, within type, type to type, within brand, brand to brand, etc. (Types may be correlated with fat/protein/sugar content?).

Of particular interest is the measurement system for taste.  This will likely require an ordinal scale which will have to be robust to personal taste preference (this is challenging).

What do you mean by "statistically valid conclusions"?

[#6 · Accepted solution]

Here's a second reply from me that you might find helpful. Occasionally when I was in industry we conducted what we called "Buy Back Studies". Essentially what we did was go to the point of final sale for our products and purchase them just like a normal person who would buy the product. Then we would bring the product back into our analytical labs and evaluate certain characteristics. One example was photographic film. We'd purchase a box of sheet film, then analyze the physical properties of the film. Things like amount of silver/sq. ft., grain size, and binary types of characteristics like physical defects (yes/no, by type, like a line, spot, or some other issue). Then our focus was on just displaying summary data visualizations such as silver/sq. ft variation, grain size distribution, etc. Our goal was to see the variation over time that our customers were experiencing and compare that variation to our specs and what we thought our manufacturing processes were actually doing.

At one point, I discovered something that I thought was troubling...over time and multiple subsequent batches of film, we were picking up a decided downward trend in silver coverage/sq. ft....like we had lost the handle on how much silver to coat for the product. When we sat down with the product and process engineers and shared our findings...they were very pleased. Indeed over time they had INTENTIONALLY been reducing the silver coverage content of this particular product as part of a material cost saving initiative. We just happened to detect this activity in our study. As for the oft asked, "How many samples do I need?"...Well that was an easy one for us. We had a budget to purchase the materials and that was the deciding factor...no statistics involved there.

So the moral of our story was use data visualizations to summarize findings. As Yogi Berra said, "You can see alot by just looking."