INTERPOLATE spline fit (or my suggested LOESS) fit to desired values

[MarkAldonWeiss]

JMP long ago wrote a White Paper on the procedural steps to go thru to produce interpolations (at the X's I DESIRE/NEED) from a spline fit.  Works fine.  As with Excel files, you keep opening and et cetera, so you don't have to create from scratch the formulas and so on.  Every time though it's still a bit tedious.  The rows adding / the columns creation / whole new data table too and so on.  So even with an already done set of formulas, still tedious.  I'd like to see JMP have a formula or "red triangle" that says "USE THE SPLINE TO INTERPOLATE / ESTIMATE VALUES BASED ON THE SPLINE FOR THE VALUES IN THIS ______ COLUMN" and the red triangle would have a dialog to simply ask you for which column of X's are the spline "responses" to be interpolated.  Once JMP has LOESS regression (per my suggestion), DITTO there.  LOESS interpolate with a red triangle to pop up the dialog, for which column of X values do I want LOESS-Interpolated response estimates.  For the motivation, see my LOESS suggestion.  It's in the realm of metrological statistics.  Fully rigorous, but not fancy for the sake of it.  And a FULLY PRACTICAL NEED.  A NEED unfulfilled (the LOESS) and fulfilled only with a big rigamaroll (interpolation of spline via the White Paper involved procedures).

[XanGregg]

I'm not familiar with the white paper, but just to clarify the request, it sounds like:

• you have 3 columns, x1, y1, x2.
• You use a smoother to fit y1 as a function of x1.
• You want to apply the same spline formula to x2 to get a new column y2.

I'm guessing the white paper says something like:

• Save the smoother formula from Bivariate to get a new column.
• Edit the formula of the new column by replacing x1 with x2.

And the request is for a new red triangle menu item in Bivariate for saving the formula with a different x?

[MarkAldonWeiss]

Thank you for your reply again.

Since you are STAFF, I attached several files. The only thing I will put in this e-mail body text is the description / comments on the files and a brief review of the motivating idea. Sensor A measures property A -- only it doesn't. Depends on temperature too. You've researched that effect. Rather than using AS-OBSERVED temperature values to apply correction to sensor A's response, you've measured so many temperatures that you figure the temperature TREND is closer to the truth. Even if you had NO values for ELAPSED TIME even their ordinal sequence #'s would suffice to time-trend the temperatures. You want SMOOTHED VALUES and additionally INTERPOLATED VALUES of Temperature for those "events" in which sensor did a measurement but you didn't likewise measure temperature.

Now for the Files:

■JMP White Paper Spline Interpolation (1997).pdf -- JMP's white paper (or JMPer Cable article); was already kind of old w.r.t. JMP 4, but more or less has idea / procedure; responses from fit for desired X's need not have a PATTERN to your desires for X values; you could desire response at any arbitrary X values

■ Spline-Interpolation for LOESS Smooth fit -- JMP Formulas.png -- the guts of the matter; two critical formulas

■ VSM Cups Pole Gap Temperatures and Inductions 27-Oct-2017.jmp -- more columns than necessary but see especially Event, LOESS Smoothing, Elapsed Time; LOESS Smoothing was pasted in from the other file...

■ VSM Cups 27-Oct-2017 LOESS smoothed & interpolated pole gap temperatures.JMP -- smoothed & interpolated estimates at desired times (New X) those estimates pasted into other table (you can see the fuss); this file came to be (before the added rows & columns from spline "SAVE COEFFICIENTS" red triangle

■ [SORTED] VSM Cups Pole Gap Temperatures and Inductions 27-Oct-2017.jmp -- need to sort, because I will need smoothed & interpolated temperatures per cup (unit measured) and instead of having the cup ID's in my randomized experiment order want the cups in ID order because I will swap and paste those temperatures into an Excel file

■ The EXCEL file -- has pasted in the temperatures; see column Smoothd VSM Pole Gap Temperatures. The Excel file is for just ONE cup. 20 were run. You can tell how the particular cup was run in two separate sets of 5 replications at different times (and consequently different temperatures). The point is HOW THE SMOOTHED & INTERPOLATED TEMPERATURES ARE USED AS INPUTS TO FURTHER COMPUTATION. They're not about show me pretty plot.

If you like see the plot though, " VSM Cups 27-Oct-2017 Overlay Plot LOESS smoothed & interpolated pole gap temperatures.pdf" To be clear, the JMP SPLINE interpolation was done on a spline tracking nearly exactly the LOESS FIT POINTS OBTAINED FROM S+ Software. The plot shows, overlaid, a JMP spline alternative, but that's not the spline being interpolated, rather the JMP spline that's practically atop the LOESS fit points is the spline of interest -- since it's the LOESS fit.

[XanGregg]

Thanks for the added detail -- no wonder I wasn't familiar with that white paper!

I think the spline and loess in current JMP make for easier application and formula adaption. Looks like your data table attachment didn't make it through the spam filters, yet, but fyi I guessed at the data from your graph and made these fits with loess and spline.

[MarkAldonWeiss]

Thank you for the reply again. I'm not after getting plots though. (That's not a problem.) I'm after getting numbers that will be subsequently used as INPUTS to a further computation. So even if no plot was ever made, I'd still want spline fit numbers at these specific values of elapsed time: at 2.35 minutes, 4.62 minutes, 5.53 minutes,... you get the idea. What where the temperatures at these elapsed times using the spline fit to make the estimates? I may NOT even have measured a temperature, for example at 4.62 minutes. But at 4.62 minutes was an EVENT. My sensor measured the magnetic induction of a cup of ink then, say. But that sensor must be corrected for temperature. (That research has been done.) So at 4.62 minutes when I measured magnetic induction of an ink, I need to know BEST ESTIMATE for what the temperature was then, so I can accurately FIX the affected-by-temperature report of magnetic induction. In some cases, I may have measured the temperature at an event (magnetic induction measurement), but even IF I did do so, I want the trended (spline, LOESS) temperature (smoothed) value, not the observed value. Regardless what elapsed times I DID measure a temperature at, there IS a LOESS or there IS a spline. Now look at my column listing the events' elapsed times and tell me the temperature for each and every one of the events from the spline or LOESS fit. Do this via a "red triangle" in which a pop-up dialog will ask me "for which column (X) do you want spline fit response estimates (Y)?" Or maybe the dialog asks the whole thing in one go: "spline fit which Y? to which X? and give estimates of response for which X2 column?"* Of course I'd prefer LOESS smoothing to spline. In any case, that's what I'm after JMP doing for me (via red triangle rather than all the rigamaroll). I take it you did get the white paper pdf file? Thanks for reading. I'm not trying to send files this time. Besides the JMP files (and the png file that was JMP formula editor picture of formulas), the Excel file I sent (or tried to send) was merely evidence that I did in fact USE the estimated temperature numbers IN AN EXCEL COMPUTATION to FIX the magnetometer reports of magnetic induction based on the temperature. I got a bit specific here, but the situation of needing to know via smoothed trend of a subsidiary (auxilliary) measurement -- temperature most likely -- to apply correction to something that says "I'm giving you the value of ______" where ____ allegedly doesn't require temperature knowledge but ACTUALLY IT DOES --I think this scenario could generally be applicable. Again, thank you for reading all this. *column X2 could be the same as column X. Spline the Y (temperature column) based on my X of elapsed time. Now give me a spline smoothed estimate for every elapsed time listed in that very same X column -- both for the X values in which I supplied a temperature in the temperature column for your spline fitting and also for the elapsed times in which temperature was missing (since I had measured something there [magnetic induction say]; that's why an elapsed time is given; I need temperature there too).

Mark A. Weiss phone: 202-874-2652
Research Chemist,
U.S. BUREAU OF ENGRAVING & PRINTING
Office of Quality Operations DCF
Analytical & Method Development Division

[XanGregg]

Right -- I got off track from your original request. I did get the white paper, and the process is much better now. The formulas saved for Spline and Loess are much simpler now with only a single reference to the independent variable, making it easier to point it to a different column if X2 is different from X. It will be happen for free for the case you mention, where there is a single X column and some of the Y values are missing: they will still get predictions in the saved formula column using the smoother.

[MarkAldonWeiss]

Very cool. I will look into the newer version of JMP. A colleague has started using a much newer version than I am using. Right now is not a good time to persue the matter further however. A couple projects (not involving any smoothing) are in the works and also our Y.E.S. (year end shutdown) is quickly approaching. Now that I know the easier current JMP capacity for my question, like I say, I will look into it some time after our Y.E.S. Thank you for all the time you spent trying to understand what I was getting at. It did sort of make sense that if JMP staff had written that White Paper on the procedures the user should go through to achieve spline interpolation, that at some point you folks would get the software to do it for the user. Thanks again.

[Ryan_Gilmore]

We are archiving this request. If this is still important please comment with additional details and we will reopen. Thank you!