Large language models (LLMs) as ‘almost perfect’ user interfaces

[#1]

Large language models (LLMs) as ‘almost perfect’ user interfaces

I attended a conference recently with, not surprisingly, several talks on LLMs. A couple things became clear:

• Everyone is wondering when they’ll be able to trust LLMs 100% of the time.
• People are using LLMs as a natural language user interface for difficult-to-execute tasks.

In the context of statistical modeling, what’s interesting to me is:

• We already have tools that are deterministic and can be trusted 100% of the time, especially when the tool has thorough testing behind it.
• Software like JMP spends a LOT of time designing easy-to-use interfaces for difficult tasks!

Case in point: Longitudinal analysis in structural equation modeling (SEM)

Correct specification of longitudinal models in SEM has long been known as a difficult task. Workshops, tutorial videos, websites, journal articles, and books on the topic abound! Indeed, strong foundations on SEM would be expected for a researcher to feel confident in writing their own script in traditional software. However, the SEM platform in JMP Pro is designed to eliminate this difficulty.

The Model Shortcuts menu in the SEM platform of JMP Pro houses numerous options for specifying (and fitting) longitudinal (and cross-sectional) models. The goal is for users to focus on the things they care about most: investigating their research questions efficiently and intuitively.

Suppose we have 10 short, stationary, time series measuring 10 constructs of interest. Let’s label each of these constructs A through J, and we’ll use “Time t” to denote the time at which they were measured. We’re assuming our sample was assessed at the same time points on six repeated occasions, and we have some missing data, as is often the case with longitudinal designs.

A screenshot of our data table showing repeated measures of the first two constructs looks like this:

The SEM platform in JMP Pro can be launched by clicking on the Analyze menu, then going to Multivariate Methods > Structural Equation Models. I select all my repeated measures on the launch dialog, click Model Variables, then OK.

The SEM platform report shows up with a default independence model. Notice the red triangle menu next to the Model Shortcuts button; it’s where the easiest specification of complex models happens.

Upon clicking on that red triangle, we can select Longitudinal Analysis > Cross-Lagged Panel Model.

Note that many types of multivariate autoregressive models can be fit within this option, including those with random intercepts and with latent variables to account for measurement error.

A key piece of information is needed at this point: the SEM platform needs to know what the time order of the measures is. Because we included “TimeT” as part of our variable names, we can click the “Guess From Names” button to get this information in an instant!

We scroll down to the end, click OK, and all 60 time values are processed. Now the menu with a variety of useful options is available. For the sake of this example, we select a few options that impose equality constraints on 1) autoregressive effects, 2) cross-lagged effects, 3) residual variances, and 4) residual covariances.

Upon clicking the OK button, a complex SEM is specified. If we hide the equality constraints and covariances from the path diagram (which can be done in the context menu of the diagram), we can see just how elaborate this multivariate system is.

I timed myself while executing, in a leisurely manner, all the steps outlined in this post; it took me 46 seconds. The resulting model will be specified correctly 100% of the time, no matter how many times I follow these steps.

How long would it take us to write a prompt for an LLM to do this?

How accurate would the results of the LLM be if we repeat this task a few times?

How long would it take us to review the results of the LLM to make sure the task was performed correctly?

I’m not “anti-LLMs” by any means, there are great uses for them! However, we should use the best tools for the given scenarios. And when it comes to efficient specification of SEMs, I think JMP Pro is a no brainer. Of course, all of JMP is designed with our users' best experience in mind.

If there’s a model you’d like to specify as quickly as this one and we don’t have it available in our SEM Model Shortcuts yet, let us know!