Generally, the 'sense' of the difference matters. For example, if Before = 3 and After = 10, the difference is 7, not -7. The response increased. So, yes, the order of the variables is important - to you.
JMP does not know what your alternative hypothesis is. The null hypothesis is that the difference = 0. Are you testing for any difference, negative or positive change? Then use Prob > |S|. If you are testing for a negative change, then use Prob < S. If you are testing for a positive change, then use Prob > S.
You should have decided the appropriate alternative hypothesis before your analysis. The test is a 'one shot.' The data is good for one test, not for repeatedly testing with different hypotheses until you get a significant decision. You should 'fish' for the hypothesis that you like.