topic Re: How to include missing values in N using tabulate to build a table in Discussions

How to include missing values in N using tabulate to build a table

LoglinearSpaces — Fri, 10 Nov 2023 09:42:35 GMT

I am trying to create a table showing analysis results and basic statistics for different substances, using tabulate. In the data, there are some missing values coded with *. Those are in fact analysis results below detection limit and should be included in N for correct statistics (i.e."mean" and other)

Is there a way of achieving this without replacing * with 0?

For example: In this table there is a count N=3 for "fosfat" and N=4 missing. I would like N to be equal to 7 here.

jmp file in attachments

Re: How to include missing values in N using tabulate to build a table

jthi — Fri, 10 Nov 2023 09:59:05 GMT

Generally missing values are missing values (and you should treat them like that) and you cannot add them to your N directly. For example if you were to accept that missing values are valid, how would you calculate min, max, mean, median (I would personally calculate them all as missing)?

Re: How to include missing values in N using tabulate to build a table

LoglinearSpaces — Mon, 13 Nov 2023 09:38:16 GMT

That of course is one way of doing it, and by the look of it JMPs prefered solution. But if I would like to include values under LOD in my table when presenting mean and median, is there a simple way of doing that in JMP? Say for example I would like to set the values to LOD/2.

Re: How to include missing values in N using tabulate to build a table

jthi — Mon, 13 Nov 2023 09:55:36 GMT

LOD?

One way of "including" missing values, is to create categorization column for example "MISSING" and then values are either 0 or 1, for example in your case something like

New Column("RESULTAT_MISSING",
	Numeric, "Nominal", Format("Best", 12),
	Formula(Is Missing(:RESULTAT)),
)

and then try to deal with that

Re: How to include missing values in N using tabulate to build a table

Mark_Bailey — Mon, 13 Nov 2023 17:00:36 GMT

Generally, including any values at or below LOD will bias the estimates (statistics) from routines for exact values (e.g., Distribution). The correct way is to use MLE with a distribution model that acknowledges censored values (e.g., Life Distribution). LOD is an example of left-censored data.

Re: How to include missing values in N using tabulate to build a table

LoglinearSpaces — Tue, 14 Nov 2023 08:53:16 GMT

Thank you for your input! Is there a way to use JMP for this and present the results correctly in a table?

Re: How to include missing values in N using tabulate to build a table

Mark_Bailey — Tue, 14 Nov 2023 14:41:43 GMT

Start with the JMP documentation for Life Distribution. The terminology is based on reliability engineering because that is the origin of these methods. Still, in the end, it is just modeling a distribution with a sample of censored data.

Re: How to include missing values in N using tabulate to build a table

MRB3855 — Tue, 14 Nov 2023 15:03:03 GMT

Hi @jthi : FYI, info on LOD (Limit of Detection) here:

https://mpl.loesungsfabrik.de/en/english-blog/method-validation/limit-of-detection-quantifcation