topic What is k when assessing variable importance? in Discussions
https://community.jmp.com/t5/Discussions/What-is-k-when-assessing-variable-importance/m-p/322286#M57226
<P><SPAN>Hi JMP Community, I was running neural networks to construct prediction models by JMP and assessing variable importance by Dependent Resampled Inputs, which using a k-nearest neighbors approach. I had the variable importance for each variable in the model, but how can I know the value of k and the other details of the k-nearest neighbors approach? </SPAN></P>Fri, 16 Oct 2020 05:19:39 GMTSerenitez2020-10-16T05:19:39ZWhat is k when assessing variable importance?
https://community.jmp.com/t5/Discussions/What-is-k-when-assessing-variable-importance/m-p/322286#M57226
<P><SPAN>Hi JMP Community, I was running neural networks to construct prediction models by JMP and assessing variable importance by Dependent Resampled Inputs, which using a k-nearest neighbors approach. I had the variable importance for each variable in the model, but how can I know the value of k and the other details of the k-nearest neighbors approach? </SPAN></P>Fri, 16 Oct 2020 05:19:39 GMThttps://community.jmp.com/t5/Discussions/What-is-k-when-assessing-variable-importance/m-p/322286#M57226Serenitez2020-10-16T05:19:39ZRe: What is k when assessing variable importance?
https://community.jmp.com/t5/Discussions/What-is-k-when-assessing-variable-importance/m-p/322308#M57229
Can u share a quick screenshot of what you're asking?Fri, 16 Oct 2020 07:22:07 GMThttps://community.jmp.com/t5/Discussions/What-is-k-when-assessing-variable-importance/m-p/322308#M57229ThuongLe2020-10-16T07:22:07ZRe: What is k when assessing variable importance?
https://community.jmp.com/t5/Discussions/What-is-k-when-assessing-variable-importance/m-p/322896#M57287
<P>Thank you for replying. I did prediction model of neural network, I selected Profilers>Assess Variable Importance>Dependent Resampled Inputs, and will show a list of variable importance. According to the JMP Help, the importance was calculated<SPAN> using a </SPAN><SPAN class="EquationVariables">k</SPAN><SPAN>-nearest neighbors approach. My question is that if I can know the k value of this <SPAN class="EquationVariables">k</SPAN>-nearest neighbors approach?</SPAN></P>Sat, 17 Oct 2020 05:20:58 GMThttps://community.jmp.com/t5/Discussions/What-is-k-when-assessing-variable-importance/m-p/322896#M57287Serenitez2020-10-17T05:20:58ZRe: What is k when assessing variable importance?
https://community.jmp.com/t5/Discussions/What-is-k-when-assessing-variable-importance/m-p/322899#M57288
<P>Thank you for replying. I did prediction model of neural network, I selected Profilers>Assess Variable Importance>Dependent Resampled Inputs, and will show a list of variable importance. According to the JMP Help, the importance was calculated using a k-nearest neighbors approach. My question is that if I can know the k value of this k-nearest neighbors approach?</P><P><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="20201017 fig1.jpg" style="width: 863px;"><img src="https://community.jmp.com/t5/image/serverpage/image-id/27465i38261BD8F4F88C34/image-size/large?v=1.0&px=999" role="button" title="20201017 fig1.jpg" alt="20201017 fig1.jpg" /></span><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="VarImportanceBoston1.gif" style="width: 401px;"><img src="https://community.jmp.com/t5/image/serverpage/image-id/27466i293623DBD0EA3B72/image-size/large?v=1.0&px=999" role="button" title="VarImportanceBoston1.gif" alt="VarImportanceBoston1.gif" /></span></P>Sat, 17 Oct 2020 05:20:11 GMThttps://community.jmp.com/t5/Discussions/What-is-k-when-assessing-variable-importance/m-p/322899#M57288Serenitez2020-10-17T05:20:11ZRe: What is k when assessing variable importance?
https://community.jmp.com/t5/Discussions/What-is-k-when-assessing-variable-importance/m-p/323024#M57307
<P>I do not think that the importance result is based on k. The help system says:</P>
<P> </P>
<P>Factor values are constructed from observed combinations<BR />using a k-nearest neighbors approach, in order to account for correlation. This option<BR />treats observed variance and covariance as representative of the covariance structure for<BR />your factors. Use this option when you believe that your factors are correlated. Note that<BR />this option is sensitive to the number of rows in the data table. If used with a small number<BR />of rows, the results can be unreliable.</P>
<P> </P>
<P>Further: </P>
<P><SPAN class="Note">Note:<SPAN> </SPAN></SPAN><SPAN>Variable importance indices are constructed using Monte Carlo sampling. For this reason, you can expect some variation in importance index values from one run to another.</SPAN></P>
<P> </P>
<P>In other words, a k-nearest neighbors approach is used to cluster observations so that the covariance structure of the data can be maintained. These observations are put into the model to get predicted values. Now, repeat (this is the Monte Carlo part) by choosing a new set of observations. See how much a change in a single factor made so that you can assess the importance of that variable. </P>
<P> </P>
<P>I do not know all of the details of the k-nearest neighbors approach that is used, but I would guess that if a choice of k is made, it would be the one that describes the data the best. Look at the k-nearest neighbors clustering technique in the help section to see how JMP "optimally" chooses a k in that situation. If this approach is truly followed, I do not know what range of k is explored. Either way, I would bet that a range of k values are used due to the monte carlo simulation that is going on.</P>Sun, 18 Oct 2020 16:09:03 GMThttps://community.jmp.com/t5/Discussions/What-is-k-when-assessing-variable-importance/m-p/323024#M57307Dan_Obermiller2020-10-18T16:09:03Z