Automate the Testing of JSL Using Hamcrest ( 2019-EU-45MP-061 )

[#1]

Level: Advanced
Job Function: Programmer
Justin Chilton, JMP Senior Associate Test Engineer, SAS
Evan McCorkle, JMP Software Developer, SAS

Have you written some JSL and gotten tired of manually testing after every change? Have you inadvertently broken some piece of your application, or has the fear of doing so prevented you from making the changes you want to make? With automated testing, you can be more confident in your changes. Now available is a set of tools for automating the testing of JSL. This framework includes the creation of tests and tests cases, as well as an implementation of the well-known Hamcrest assertion library. Hamcrest provides flexible ways to assert what you know to be true about the behavior of your JSL. These tools are full-featured and can be specialized for your needs. In fact, JMP development even uses them to test JMP itself. The presentation will cover this framework and its use in testing an example JSL application, from individual functions to the automation of GUI interactions.

[vince_faller]

Is there a best practice for comparing data tables?  There isn't a matcher I can find. 

[nathan-clark]

I was just thinking about this the other day, @vince_faller. Have you found an answer yet? I was looking into using the Compare Data Tables platform, but didn't see an output I could use in a unit test.

[Justin_Chilton]

Hi @nathan-clark,

We have been discussing how a data table matcher might work on GitHub here:  https://github.com/sassoftware/jsl-hamcrest/pull/57

I think @vince_faller might be working on something already.

-Justin

[nathan-clark]

Thanks! I caught up with the conversation and I think you guys are on the right track. I look forward to seeing what is merged in!

[carole]

Hi @Justin_Chilton, thank you for the hamcrest presentation. Could you please share the example you developed into your presentation: JMPMan application and JMPManGuiTests.jsl ? It should be easier for me to reproduce the logic of scripting into other applications. Thanks a lot. Carole

[Justin_Chilton]

Hi @carole, the JMPMan examples are located in the add-in, but I have uploaded them in a zip file to make them easier to access. Let me know if you have any questions!

[carole]

Thank you very much @Justin_Chilton. I found into the add-in of course thanks ! For the questions, I am wondering if using Define Class( )  and Method( ) are essential to use hamcrest tests for an application with a UI ? I am not familiar with such a scripting way ...

[Justin_Chilton]

@carole, as long as you are only writing tests with our existing matchers, you do not need to use classes. You would need to use Define Class() and Method() if you needed to write your own matchers, but we provide a large amount with the library, so you likely would not need more.

[carole]

Ok thanks for the precision it's clear. I am going to test it on a simple application. Maybe I will have some other questions...

[carole]

Sorry for my very simple example @Justin_Chilton. I've tried to simplify my problem. How can I do to avoid clicking on the pick file window during the following test please ?

openfile = function ( {},
	strPathOpen =	PickFile("Select the Data Set Table","$SAMPLE_DATA",{"JMP DATA TABLE|JMP"});
	return(strPathOpen)
);

ut test("Pickfileok", "Open Big Class File", Expr(
	ut assert that(Expr(openfile()), "/C:/Program Files/SAS/JMP/15/Samples/Data/Big Class.jmp");
));

[Justin_Chilton]

Hi @carole, 

Apologies for the delayed response, as I did not see your response. In this scenario, I would add an optional argument to the 'openfile' function that you could pass in while testing to avoid the need for the pop-up dialog.

Below is an update to your script where you can pass in optional args. Another alternative is to have some sort of flag like 'isTest' that you can check in your code to avoid modal dialogs like this.

openfile = function ( {path = Empty()},
    strPathOpen = If(IsEmpty(path),
        PickFile("Select the Data Set Table","$SAMPLE_DATA",{"JMP DATA TABLE|JMP"}),
        path
    );
    If(strPathOpen != "", Open(strPathOpen), Empty());
);

ut test("Pickfileok", "Open Big Class File", Expr(
    dt = openfile("$SAMPLE_DATA/Big Class.jmp");
    ut assert that(Expr(dt), ut n col(5), "opened table should have 5 columns");
));

[carole]

Hi @Justin_Chilton , thank you for your answer. I had found the solution you write as an example before reading you, so you confirm to me that I was on the right way. Since I am getting back to this project I may have new questions in a few days ... Thank you again for your answers.