Last time, I discussed setting up a new stage of experimentation for dyeing diecast vehicles. Not everything went as planned, but there were some positive results.
I took the alias optimal design from the previous blog post and used a column shuffle to randomize the rows (it was sorted by random blocks originally). Three raters gave a forced ranking as to how well the cars were dyed.
As a quick reminder, these were the factors in the experiment:
Changes during data collection
When I created the alias optimal design, I had a few acetone and vinegar mixes that were close to 25/25 mixes (20/30 and 22.5/27.5, for example). I manually changed these to 25/25 since there was a line on the measuring cup that made me more comfortable with those measurements.
Once we started the experiment, when I began to examine some of the completed cars, I noticed that plastic bases/tires were starting to melt on them – not a good sign! I knew this was a potential problem with the acetone. Upon looking at the design that corresponded to the melting, it was the half water, half acetone mixes that were most noticeably having issues. The remaining 50% acetones were changed to 25% acetone and 25% vinegar – even if the dye was doing a good job, the melting plastic would eliminate 50% acetone from consideration.
After dyeing the 16 cars in the experiment, there were noticeably better results compared to the first experiment (and noticeably worse when you consider the melting of the car…). Before I let the raters look at the cars, I picked out two of the cars that stood out to me and highlighted their rows in the data table to see if anything popped out at me. Sure enough, those two (different cars and colors) had two factor level combinations in common: additional heat and 50% vinegar. Alas, because of the constraints, there were only four runs that had additional heat (and two that had added heat and vinegar, which also happened to be the two I was interested in), which still wasn’t much extra information.
I had a couple of extra cars available, so I decided to do my own augmentation with two cars having additional heat. I dyed one car in blue with 50% vinegar, and one in yellow with no vinegar (both for 30 minutes to see if the color would stick).
I tried a number of different techniques to analyze the average rank, but I’ll go through the basic idea here. My analyses excluded the three runs that had 50% acetone. Not only did I already know that I was not going to be using 50% acetone due to the melting plastic issue, but the raters had a bigger disagreement on those runs because one rater penalized more when he saw issues with the vehicle. However, running the analyses with the excluded runs presented similar results.
Because of the exclusions with the acetone, the acetone*vinegar interaction is not estimable with the main effects. Fitting the main effects model:
Not surprisingly from what I had observed, additional heat was the most important factor. Dye amount and acetone look potentially interesting, but I was also curious in the additional heat * vinegar interaction. Fitting that model:
Note that the additional heat*vinegar interaction is not significant, although the effect itself is large. Most of the main effects are relatively close, but you may notice that the effect of dye amount has dropped. Recall that additional heat*vinegar was not included in the original model. Even in picking the alias optimal design, the -.43 in the alias matrix for the dye amount with the additional heat * vinegar interaction is enough to make a difference with the size of the interaction.
All that said, the one thing that all the modeling attempts I tried agreed with is that additional heat is the biggest driver for the better cars. I usually show Fit Model results at this point in the blog, but I think a picture showing the average rank vs. vinegar, broken up by additional heat is the most compelling:
In particular, the three highest rated cars were 50% vinegar with additional heat. You may notice there is one point that performed reasonably well with 25% vinegar – this car also had 25% acetone, and on closer inspection, the plastic was a bit affected. Recalling that additional heat meant no acetone, I don’t think I’ll be using acetone in the next experiment (my wife didn’t appreciate the acetone smell in the kitchen either, so it would’ve been a hard sell). So vinegar and heat will definitely be in the next experiment. While time and dye amount are not as significant, there may still be some value in carrying them through.
This experiment and the analysis ended up being a lot more involved than I anticipated. As disappointed as I was to see the acetone not work, I was pretty excited to see results better than expected.
I’m certainly intrigued to continue experimenting with keeping heat on during the dyeing process, and will likely limit it to one color, since I think it’s an easier comparison for the forced ranking.
I’ve still been picking up diecast vehicles for the purpose of more experimentation… so stay tuned for Part III. Thanks for reading!
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.