topic Re: Missing Y Response Variable Fit Model Proportional Hazards in Discussions

Missing Y Response Variable Fit Model Proportional Hazards

lisamaley — Fri, 09 Jun 2023 16:58:09 GMT

I am trying to do some Survival analysis, where I am combining some analysis into one script. So what I am trying to do, is they enter the columns into the Survival Platform, then I take those columns and use them in the Proportional Hazards Platform. My script works great except for the Y, response, it will not find the Y response variable. If I change Y( As Column( Ycolname ) ), to something like Y("Time to Failure"), it works great. But when I try to use the variable, it does not work.

I have two questions - why does my method work for Censor and the Xs, but not the Ys.

Second question, if they do not enter a by or frequency column, how do I tell the fit model that there is not a column to enter. If I use As Column (variable) it says it cannot find my column, which is right.

Names Default To Here( 1 );
Clear Log();
dt = Current Data Table();

If( Not( Is Scriptable( dt ) ),
	dtsel = Pick File( "Open File", {"JMP|jmp", "All files|*"} );
	If( dtsel == "",
		Stop(),
		Try( dt = Open( dtsel ), Stop() )
	);
);


dtname = Data Table( dt ) << Get Name;
validated = 0;

New Window( "Survival/Reliability",
	<<Modal,
	<<On Validate(
		survival = {};
		Insert Into( survival, Substitute( olb[List Box Box( 2 )] << Get Items, {}, Expr( Y() ) ) );
		Insert Into( survival, Substitute( olb[List Box Box( 3 )] << Get Items, {}, Expr( Grouping() ) ) );
		Insert Into( survival, Substitute( olb[List Box Box( 4 )] << Get Items, {}, Expr( Censor() ) ) );
		Insert Into( survival, Substitute( olb[List Box Box( 5 )] << Get Items, {}, Expr( Freq() ) ) );
		Insert Into( survival, Substitute( olb[List Box Box( 6 )] << Get Items, {}, Expr( By() ) ) );
		Insert Into( survival, Insert( Expr( Censor Code() ), olb[Number Edit Box( 1 )] << Get ) );
		
		Insert Into(
			survival,
			Insert( Expr( Plot failure instead of Survival() ), olb[Check Box Box( 1 )] << Get )
		);
		survival expr = Expr( Send( dt ) );
		Insert Into( survival expr, Substitute( survival, {}, Expr( Survival() ) ) );
		obj = survival expr;
		obj << Save Estimates;
		validated = 1;
	),
	olb = Outline Box( "",
		dt << Survival(
			Failure Plot( 1 ),
			Show Points( 1 ),
			Show Shaded Simultaneous CI( 1 ),
			Show Simultaneous CI( 1 ),
			Weibull Plot( 1 ),
			Weibull Fit( 1 )
			
		)
	)
);

If( !validated, Stop() );
Eval(PH);

dt2 = Data Table( dtname || " Survival" );
columnnamegroup = Column( dt2, 1 ) << Get Name;
GroupID = Associative Array( Column( dt2, 1 ) ) << Get Keys;
Show( GroupID );

n = N Items( GroupID );

Show( n );

Current Data Table( dt2 );
dt2 << select where( As Column( columnnamegroup ) == "Combined" );
dt2 << hide and exclude;
dt2 << Clear Select;

Graph Builder(
	Variables( X( :"log(Time)"n ), Y( :"log(-log(Surv))"n ), Overlay( As Column( columnnamegroup ) ) ),
	Elements( Points( X, Y, Legend( 17 ) ), Line Of Fit( X, Y, Legend( 18 ) ) ),
	SendToReport(
		Dispatch(
			{},
			"400",
			ScaleBox,
			{Legend Model(
				17,
				Base( 0, 0, 0, Item ID( "Op1", 1 ) ),
				Base( 1, 0, 0, Item ID( "Op2", 1 ) ),
				Base( 2, 0, 0, Item ID( "Op3", 1 ) )
			)}
		)
	)
);


PH = Expr(
	Ycolname = Arg( survival[1], 1 );
	Effectname = Arg( survival[2], 1 );
	Censorname = Arg( survival[3], 1 );
	Freqname = Arg( survival[4], 1 );
	Byname = Arg( survival[5], 1 );
	CensorCodename = Arg( survival[6], 1 );

	Show( Ycolname );

	Current Data Table( dt );


	dt << Fit Model(
		//	Freq(),
		//	By(),
		Y( As Column( Ycolname ) ),
		Censor( As Column( Censorname ) ),
		Effects( As Column( Effectname ) ),
		Personality( "Proportional Hazard" ),
		Censor Code( "1" ),
		Run Model( Likelihood Ratio Tests( 1 ), Likelihood Confidence Intervals( 1 ), Risk Ratios( 1 ) ) 

	);
	
);

Re: Missing Y Response Variable Fit Model Proportional Hazards

peng_liu — Sun, 01 May 2022 02:43:01 GMT

If you use Column, instead of As Column, do you get what you want? I don't actually see under what situation, one has to use As Column, rather than Column. Maybe, one of my colleagues who has more knowledge about this function can comment.

Re: Missing Y Response Variable Fit Model Proportional Hazards

lisamaley — Sun, 01 May 2022 22:39:37 GMT

Column does the same thing as As Column, it fills in the information except for the Y:

Re: Missing Y Response Variable Fit Model Proportional Hazards

peng_liu — Mon, 02 May 2022 00:52:08 GMT

Would you please double check your script? I don't see how to get to the dialog that you show here.

I got an error message:

If I move Eval(PH) to the bottom of your script, and change As Column to Column, then I don't see the Fit Model dialog at all.

BTW, I use Rats.jmp to test. "days" to Y, "Censor" to Censor, "Group" to Grouping.

Re: Missing Y Response Variable Fit Model Proportional Hazards

lisamaley — Mon, 02 May 2022 02:27:38 GMT

I know, I got the same thing where the fit model did not come up, if I moved Eval(PH) to the bottom, which is why I moved it where I did. I also got the same error with Rats, but not this error with my files, not sure why. If I take out the Expression and just let it run like this, then the error goes away and I get the same problem as before, without Expr(PH) is pasted below the screen shot, the other columns come through, but not the Y. This script is correct and my most recent. If you have any ideas, please let me know! This is very frustrating for me and I have no idea of why!

Names Default To Here( 1 );
Clear Log();
dt = Current Data Table();

If( Not( Is Scriptable( dt ) ),
	dtsel = Pick File( "Open File", {"JMP|jmp", "All files|*"} );
	If( dtsel == "",
		Stop(),
		Try( dt = Open( dtsel ), Stop() )
	);
);


dtname = Data Table( dt ) << Get Name;
validated = 0;

New Window( "Survival/Reliability",
	<<Modal,
	<<On Validate(
		survival = {};
		Insert Into( survival, Substitute( olb[List Box Box( 2 )] << Get Items, {}, Expr( Y() ) ) );
		Insert Into( survival, Substitute( olb[List Box Box( 3 )] << Get Items, {}, Expr( Grouping() ) ) );
		Insert Into( survival, Substitute( olb[List Box Box( 4 )] << Get Items, {}, Expr( Censor() ) ) );
		Insert Into( survival, Substitute( olb[List Box Box( 5 )] << Get Items, {}, Expr( Freq() ) ) );
		Insert Into( survival, Substitute( olb[List Box Box( 6 )] << Get Items, {}, Expr( By() ) ) );
		Insert Into( survival, Insert( Expr( Censor Code() ), olb[Number Edit Box( 1 )] << Get ) );
		
		Insert Into( survival, Insert( Expr( Plot failure instead of Survival() ), olb[Check Box Box( 1 )] << Get ) );
		survival expr = Expr( Send( dt ) );
		Insert Into( survival expr, Substitute( survival, {}, Expr( Survival() ) ) );
		obj = survival expr;
		obj << Save Estimates;
		validated = 1;
	),
	olb = Outline Box( "",
		dt << Survival(
			Failure Plot( 1 ),
			Show Points( 1 ),
			Show Shaded Simultaneous CI( 1 ),
			Show Simultaneous CI( 1 ),
			Weibull Plot( 1 ),
			Weibull Fit( 1 )
			
		)
	)
);

If( !validated, Stop() );


dt2 = Data Table( dtname || " Survival" );
columnnamegroup = Column( dt2, 1 ) << Get Name;
GroupID = Associative Array( Column( dt2, 1 ) ) << Get Keys;
Show( GroupID );

n = N Items( GroupID );

Show( n );

Current Data Table( dt2 );
dt2 << select where( As Column( columnnamegroup ) == "Combined" );
dt2 << hide and exclude;
dt2 << Clear Select;

Graph Builder(
	Variables( X( :"log(Time)"n ), Y( :"log(-log(Surv))"n ), Overlay( As Column( columnnamegroup ) ) ),
	Elements( Points( X, Y, Legend( 17 ) ), Line Of Fit( X, Y, Legend( 18 ) ) ),
	SendToReport(
		Dispatch(
			{}, "400", ScaleBox, 
			//{Legend Model(
			//	17,
			//	Base( 0, 0, 0, Item ID( "Op1", 1 ) ),
			//	Base( 1, 0, 0, Item ID( "Op2", 1 ) ),
			//	Base( 2, 0, 0, Item ID( "Op3", 1 ) )
			//)}
		)
	)
);



Ycolname = Arg( survival[1], 1 );
Effectname = Arg( survival[2], 1 );
Censorname = Arg( survival[3], 1 );
Freqname = Arg( survival[4], 1 );
Byname = Arg( survival[5], 1 );
CensorCodename = Arg( survival[6], 1 );

Show( Ycolname );

Current Data Table( dt );


Platform = dt << Fit Model(
		//	Freq(),
	//	By(),
	Y( As Column( Ycolname ) ),
	Censor( As Column( Censorname ) ),
	Effects( As Column( Effectname ) ),
	Personality( "Proportional Hazard" ),
	Censor Code( "1" ),
	Run Model( Likelihood Ratio Tests( 1 ), Likelihood Confidence Intervals( 1 ), Risk Ratios( 1 ) ) 

);

Re: Missing Y Response Variable Fit Model Proportional Hazards

peng_liu — Mon, 02 May 2022 13:53:11 GMT

I cannot reproduce the problem if I change As Column to Column. Here is the full codes that I run. I just changed the last three As Column to Column.

Names Default To Here( 1 );
Clear Log();
dt = Current Data Table();

If( Not( Is Scriptable( dt ) ),
	dtsel = Pick File( "Open File", {"JMP|jmp", "All files|*"} );
	If( dtsel == "",
		Stop(),
		Try( dt = Open( dtsel ), Stop() )
	);
);


dtname = Data Table( dt ) << Get Name;
validated = 0;

New Window( "Survival/Reliability",
	<<Modal,
	<<On Validate(
		survival = {};
		Insert Into( survival, Substitute( olb[List Box Box( 2 )] << Get Items, {}, Expr( Y() ) ) );
		Insert Into( survival, Substitute( olb[List Box Box( 3 )] << Get Items, {}, Expr( Grouping() ) ) );
		Insert Into( survival, Substitute( olb[List Box Box( 4 )] << Get Items, {}, Expr( Censor() ) ) );
		Insert Into( survival, Substitute( olb[List Box Box( 5 )] << Get Items, {}, Expr( Freq() ) ) );
		Insert Into( survival, Substitute( olb[List Box Box( 6 )] << Get Items, {}, Expr( By() ) ) );
		Insert Into( survival, Insert( Expr( Censor Code() ), olb[Number Edit Box( 1 )] << Get ) );
		
		Insert Into( survival, Insert( Expr( Plot failure instead of Survival() ), olb[Check Box Box( 1 )] << Get ) );
		survival expr = Expr( Send( dt ) );
		Insert Into( survival expr, Substitute( survival, {}, Expr( Survival() ) ) );
		obj = survival expr;
		obj << Save Estimates;
		validated = 1;
	),
	olb = Outline Box( "",
		dt << Survival(
			Failure Plot( 1 ),
			Show Points( 1 ),
			Show Shaded Simultaneous CI( 1 ),
			Show Simultaneous CI( 1 ),
			Weibull Plot( 1 ),
			Weibull Fit( 1 )
			
		)
	)
);

If( !validated, Stop() );


dt2 = Data Table( dtname || " Survival" );
columnnamegroup = Column( dt2, 1 ) << Get Name;
GroupID = Associative Array( Column( dt2, 1 ) ) << Get Keys;
Show( GroupID );

n = N Items( GroupID );

Show( n );

Current Data Table( dt2 );
dt2 << select where( As Column( columnnamegroup ) == "Combined" );
dt2 << hide and exclude;
dt2 << Clear Select;

Graph Builder(
	Variables( X( :"log(Time)"n ), Y( :"log(-log(Surv))"n ), Overlay( As Column( columnnamegroup ) ) ),
	Elements( Points( X, Y, Legend( 17 ) ), Line Of Fit( X, Y, Legend( 18 ) ) ),
	SendToReport(
		Dispatch(
			{}, "400", ScaleBox, 
			//{Legend Model(
			//	17,
			//	Base( 0, 0, 0, Item ID( "Op1", 1 ) ),
			//	Base( 1, 0, 0, Item ID( "Op2", 1 ) ),
			//	Base( 2, 0, 0, Item ID( "Op3", 1 ) )
			//)}
		)
	)
);



Ycolname = Arg( survival[1], 1 );
Effectname = Arg( survival[2], 1 );
Censorname = Arg( survival[3], 1 );
Freqname = Arg( survival[4], 1 );
Byname = Arg( survival[5], 1 );
CensorCodename = Arg( survival[6], 1 );

Show( Ycolname );

Current Data Table( dt );


Platform = dt << Fit Model(
		//	Freq(),
	//	By(),
	Y(  Column( Ycolname ) ),
	Censor(  Column( Censorname ) ),
	Effects(  Column( Effectname ) ),
	Personality( "Proportional Hazard" ),
	Censor Code( "1" ),
	Run Model( Likelihood Ratio Tests( 1 ), Likelihood Confidence Intervals( 1 ), Risk Ratios( 1 ) ) 

);

Here are my steps and what I saw. I have checked JMP14, JMP15, and JMP16. I believe your version is not further back due to how your codes use new name literals.

Open Rats.jmp
Run the script
Specify the dialog when it pops up, then click OK
Some reports show up. And this is the Fit Model launch dialog looks like:

If you cannot see what I am seeing here, I can only suspect there is something that neither of us are aware of. If so, you may want to contact support@jmp.com to get some help.

Re: Missing Y Response Variable Fit Model Proportional Hazards

lisamaley — Mon, 02 May 2022 17:30:49 GMT

I know I tried this and it did not work for me, but when I pasted your code it worked perfectly. I have no idea of why, but it works so THANK YOU!

Re: Missing Y Response Variable Fit Model Proportional Hazards

lisamaley — Tue, 03 May 2022 18:53:26 GMT

Did you have any suggestions on my second question?

if they do not enter a by or frequency column, how do I tell the fit model that there is not a column to enter. If I use Column (variable) it says it cannot find my column, which is right.

Re: Missing Y Response Variable Fit Model Proportional Hazards

peng_liu — Wed, 04 May 2022 12:23:57 GMT

I am not sure that I understand the problem. Here is what I have tried and seen.

I first modify Rats.jmp to mock up a case so I can test your codes:

I added a column By and evenly split the data into two groups.

I modified the last piece of your codes to the following.


if (isempty(byname), 
	Platform = dt << Fit Model(
			//	Freq(),
			//	By(),
		Y(  Column( Ycolname ) ),
		Censor(  Column( Censorname ) ),
		Effects(  Column( Effectname ) ),
		Personality( "Proportional Hazard" ),
		Censor Code( "1" ),
		Run Model( Likelihood Ratio Tests( 1 ), Likelihood Confidence Intervals( 1 ), Risk Ratios( 1 ) ) 

	),
	Platform = dt << Fit Model(
			//	Freq(),
		By( column(byname) ),
		Y(  Column( Ycolname ) ),
		Censor(  Column( Censorname ) ),
		Effects(  Column( Effectname ) ),
		Personality( "Proportional Hazard" ),
		Censor Code( "1" ),
		Run Model( Likelihood Ratio Tests( 1 ), Likelihood Confidence Intervals( 1 ), Risk Ratios( 1 ) ) 

	)
);

I tried two paths to execute the codes:

First, run the code and enter roles as follows:

No "By". and the behavior of the codes is as what was before.

Second, run the code and enter roles as follows:

With "By", and I see an error message in the end:

I guess the intermediate results are different, and your codes have not handled the situation correctly.

Re: Missing Y Response Variable Fit Model Proportional Hazards

lisamaley — Wed, 04 May 2022 16:48:40 GMT

Thanks for your guidance, you have been extremely helpful!

My problem was - if on the first screen there was no column put into frequency,

Freqname = Arg( survival[4], 1 );

This would would return as empty.

So if I tried to run the script:

Platform = dt << Fit Model(

Freq( Column( Freqname ) ),

By( Column( Byname ) ),

Y( Column( Ycolname ) ),

Censor( Column( Censorname ) ),

Effects( Column( Effectname ) ),

Personality( "Proportional Hazard" ),

Censor Code( "1" ),

Run Model( Likelihood Ratio Tests( 1 ), Likelihood Confidence Intervals( 1 ), Risk Ratios( 1 ) )

)

It would say there is not column for frequency, which was true.

So what I did was to put 8 nested if loops looking to see which columns were empty.

Not exactly the best way maybe, but it worked.