Help performing a median polish

[#1]

Hello,

I have cell-based screening data that is showing a row-wise artifact that causes my odd rows (A, C, etc) to be slightly lower in signal than my even rows (B, D etc). While the perfect solution is to mitigate the source of variance in the equipment, I also would like to smooth my data to salvage and better call hits. I have read in a number of papers about Tukey's median polish as being used for screening data to smooth out spatial bias but was unable to determine how to implement this in JMP or if it was already a built-in tool. I know that I could potentially calculate one iteration of this by finding the row/col medians and performing the math, but what makes this tool powerful is the ability to iterate over n-number of cycles until there is negligible change across row and column medians. I was wondering if someone might help me write a script or point me toward a plug-in that could accomplish this task? It would be nice to have a gui window similar to some of the other analysis tools where I could put in my y-variable, my row and column columns, and then any "by" variables. Then also potentially either a box to enter the number of iterations, or some sort of test for residuals so that it can tell itself when to stop iterating and show me the results of each iteration at the end. I am very new to JMP so this is way over my head, but would appreciate any help! I think this could be a powerful addition to the screening analysis tools also, because this has been shown to better smooth data and make hit calling more reliable.  Thanks in advance! 

-Ani 

Link to info about median polish:

https://en.wikipedia.org/wiki/Median_polish

[#2]

I do not believe there is a Median Polish  provided in JMP, however, the JMP interface to Python or R will allow you to access their Median Polish routines.

[#3]

Hi Jim,

Thanks for the reply. I was hoping to avoid having to build something and instead leverage what others have already made! 

Thanks again,

Ani

[#4]

Also, if you're just looking to de-bias alternate rows, you can do so with a few clicks in the GUI. You could also choose Standardized instead of Centered.

 

It does sound like a fun scripting exercise, though. If I were scripting it in JSL, I'd likely follow the steps in this video.

[#5]

... and then maybe replace Col Mean() with Col Median().

To apply the full Median Polish, the data has to be suitable to do so ...
- otherwise a lot of valuable (essential!) information is removed "accidentally".

[#6]

Another approach ...

Oh, I won't mention this time.

[#7]

I'll mention it for you! @hogi's Normalization GUI can also do this.

[#8]

Thanks both! 

I did try something similar to this with functions and it did improve the data slightly! I took the median of all test wells in each row (because in an HTS under appropriate conditions, most test wells will not be hits) and used this to normalize against. Did a decent job at mitigating the row effect and helping to triage hits. 

-Ani 

[#9]

Hi Jed,

Thank you for your reply. Can you describe a bit more about what you are doing in this screenshot? I am not familiar with these options.

As for scripting, yes I understand the steps of the median polish, but as it is iterative, it would take some time for me to script/build in flexibility. Was hoping someone might help me build it or point me to a script that was already working! 

[#10]

Hi @amichaud,

Here's a quick breakdown of what's happening above:

• Since you mentioned even vs odd rows, I created a column that alternates 0 and 1, called Bias.
• Right-clicking on that Bias column, I selected New Formula Column > Group By. This doesn't create any columns, but sets the Bias column as a grouping column for any future formula columns.
• Right-clicking the Y column, then selecting New Formula Column > Distributional > Center transforms the Y column to be centered around 0, iteratively for any given grouping variable. This means that the 0s and 1s in the Bias column will be transformed separately. This is really just a different way to de-bias the samples based on odd/even rows. You could also choose Distributional > Standardize to do this.
• The graph is just trying to show this visually. The bottom graph has a different fit for each bias group, but the top graph has the same fit for both, since the bias was removed by centering.

Also, depending on what you're trying to do, you could also just include the Bias column as an effect in any model that you're creating. The Profiler will then let you see and interact with the effect, with no data changes needed. I attached an updated data table to show this.