Thank you for posting the pictures.
Your computation is for a two-sided confidence interval with 95% confidence. That is exactly what JMP is reporting in the Summary Statistics in the Distribution platform (fourth picture).
You want 95% confidence between both tails of the distribution so you center the 95% of the t distribution leaving 2.5% on either side. We still label it as Lower 95% Mean and Upper 95% Mean to indication the confidence level. It is a short label meant to convey the lower or upper bound on the location of the population mean with 95% confidence. This kind of labeling is common.
If you actually want 5% in both tails, then you want a two-sided 90% confidence interval. Click the red triangle at the top of the Distribution platform next to the name of your variable and select Confidence Interval > 0.90 and you will get the desired interval.
The numbers from your manual calculation and from JMP might not agree exactly because you are using a t quantile rounded to three decimal places and JMP is using the full precision value, but the values should be close to three or four significant digits.