cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Have your say in shaping JMP's future by participating in the new JMP Wish List Prioritization Survey
Choose Language Hide Translation Bar
Florent_M
Level III

How to update efficiently a table that contains a prediction model based on DTM (Document Term Matrix)?

Hello!

 

I work on textual data classification. So I made a model that predicts the class by taking as input the DTM. Now, I would like to update, in a regular and efficient way, the data table with new rows. My problem is to be able to easily compute the DTM using a formula so that the update is automatic.

 

Here is a file based on the example of potato chip reviews:
- first, we use the text explorer to save the DTM
- 2nd, we use a neural network to predict the column "Buy again?"

 

From there, I'd like to add rows to the table, and have the model predictions computed automatically, without having to relunch the text explorer.

 

Do you have any suggestions?

 

Thank you,
Florent

3 REPLIES 3

Re: How to update efficiently a table that contains a prediction model based on DTM (Document Term Matrix)?

The DTM is saved as values. Did you try saving it as a formula? Does it update when a new row is added to the table?

Florent_M
Level III

Re: How to update efficiently a table that contains a prediction model based on DTM (Document Term Matrix)?

I tried to save the DTM as Formula. Unfortunatly, the formula return an array, which cannot be used in a predictive model. 
Do you have an idea to use directly the array in a model or to convert an array into columns?

 

Thanks,

Re: How to update efficiently a table that contains a prediction model based on DTM (Document Term Matrix)?

Yes, it can be used as a set of predictors. Did you try?

 

Table with saved formula:

table.PNG

 

Logistic regression with array as only model term, but expands to all terms in the DTM:

model.PNG