Hi @Aziza,
I'm glad you were able to create the new paper size! What is "obj" in this situation? Journals and reports can receive a Set Page Setup message, so if you have the result of a platform call as "obj" you will need to wrap it in Report() to send the message to the report layer.
Additionally, as I understand it, Set Page Setup() refers to the settings used when using <<Save PDF. So, your changes won't be reflected in File > Page Setup, but will be used when using << Save PDF. Here's an example of the workflow:
Open("$SAMPLE_DATA\Big Class.jmp");
//Create some output
obj = Distribution( Column( :height ));
//Set the page settings for the report containing the distribution output
report(obj) << Set page setup(margins( 0.2, 0.4, 0, 0 ),scale( 0.75),portrait( 0 ), paper size( "Beamer" ));
//To view the settings use Get Page Setup()
//report(obj) << Get Page Setup
//Save the PDF
report(obj) << Save PDF( "$DOCUMENTS\example.pdf" );