The Secret is Out! Oh Wait, No It’s Not. (2025-US-30MP-2510)

Good afternoon everyone. My name is Nathan Clark. I've been using JMP for 22 years now, and I'm from IDEXX Laboratories. I'll be presenting on secrets management with my co presenter, Sage Darling.

Hi. I'm Sage. I work over at IDEXX as well with Nathan. Over here, I'm a senior analytics engineer, and we're going to be talking to you about secrets.

Excellent. Let's start just talking a little bit about what is a secret? Typically, at least from a code perspective, a secret is some form of credential that is something that you need to utilize for your code to work so you can access things, but at the same time, other people shouldn't be knowing this. It's very similar to your passwords that you log into your computer with. You have to use it often, but you shouldn't be giving it to other people.

Secrets are ways different platforms have developed to allow these credentials to exist in a way that you cannot see them once they're out there. So they become much more safe to use in background tasks. JMP does not have this inherently built in. But as we start doing more with code and background or behind the curtain stuff in JSL, we've wanted that ability to be there in order to share the things we're doing with others, accessing different things without accidentally giving anyone the keys to the proverbial kingdom.

In a discussion I actually was having with JMP Support a while ago, I actually got this little piece of code written by, I believe it was Evan McCorkle, who would put this together as a way to mask these secrets and something where it could exist like we're seeing here, but then you can encrypt the script and then use it.

Basically what it uses is one function that has the secrets in it, but requires a secondary secret or like an access phrase that you know exists, that all the functions within this encrypted code have it in there, but no one else is going to know what it is. Even if they know that use secret A is the function they want to try to call or something like that, they cannot access these secrets directly without knowing what this phrase is. Once this is encrypted, now you can then include it as a secret script, and then you can actually get to these pieces of information without you knowing. So far, when I was talking with JMP Support, despite trying, they hadn't found a way to expose the secret when testing. With the stuff that I've been working on, I haven't been able to either. It seems like a really nice framework to start with to build upon how you want to be able to have secrets stored in a secure manner, but still allow people to utilize them to be able to do various code activities.

We have two examples today. I'll be talking about an example of how we've put this in place for Snowflake. What I have here is I have, excuse me, the Snowflake connection template that is here. This is basically the function I've written with all of the good bits hidden.

I'll make this a little bit larger here. You see, I have the get_secrets function that has where the secrets live. In my case, for what we're looking to do, I only need a username and a password in order to get into Snowflake with how things are currently configured. This is a relatively easy password array to have in here.

But then I have other support functions to utilize them. This first one creates an actual connection to the database and just returns that new data connector. This is a string that you would put together. This is something where you have the data connector, and it connects to all of the pieces that are specific to your company. The only thing that you need to have is the Snowflake driver installed. This is the name of the Snowflake driver when you install it from the Snowflake website onto your computer. This way you can make the ODBC connection. But your database will be specific to your companies. However, your company servers are connected to Snowflake, et cetera. Then the password is where it grabs from this other array that we've gotten here. It puts it in, then you have your other pieces here that you can connect.

The last function I have is taking it one step further. This actually takes in the string of the query you want to run and does all of the connections here and runs the query in this function. Then what it ends up doing is it gets rid of all of the table scripts that come off that table. I'll talk about that when I do the example as to why that might be useful. But basically it's another layer of protection so that there's no way the credentials accidentally get shared or exposed.

With that, I have in a really simple example here. In the first example, this is just basically doing a straight-up raw query, and I'm doing custom SQL and just inserting the string version of what the query would be. If I run this, I get a table from Snowflake. It looks an awful lot like Big Class because it is. I just uploaded a copy of it to Snowflake. This gives you the table exactly as you want. It includes some source pieces of information here. This can be nice. I noticed that when I do this, it does mask the username and the password, which is what I would expect. I did notice that when I was poking around a lot of different ways at this, I had a couple examples, whether they were flukes or not, that the password wasn't exposed, but the user ID was. Depending on how tight you need to monitor these credentials, that's why I made the additional function that removes these pieces of information.

If I comment out this section here, and now I run this, this is just creating a new table with this other function where I'm inputting this as the query, and now it creates the same thing. Only now you're in a situation where the table doesn't have any of these table scripts. That's nice. To further go into this with how this works, for instance, if I want to delete this and just select this star graph, all the columns, this should now work and brings them all in. The same thing if you wanted to add some of the filtering capacities. You can do these here. It works just like standard SQL language does when you're doing these queries. You just pass the string in to this and get your scrub data table out. That's a relatively simple example. I'd like to pass this off now to Sage who has a more complicated version of how this can work in together with the layering of your function. I'll pass off to Sage now.

Fox Drive is a online folder saving location where you can put information. For us, we use that for a lot of different things and so because of that that's what I wanted to start with, was how do I get myself to authenticate in Box in order to pull files locally from Box to my computer and so that requires building an app through Box Developers that I then can authenticate. Then by doing that, that app requires giving you what's called a JWT, which is a JSON web token, and that JSON web token pipeline wise can then be used to get an access token that I then can use to actually access Box. That's where my situation was a little bit more complex because I had never messed with JSON or JSON web tokens before, JWTs, and so that's where I ended up having to implement here.

As Nathan was describing, this is my example on the right of what my secrets.jsl looks like. This is not encrypted right now, so you can see what the insides of it look like. Mine starts with also a get secrets or has a function for get secrets. This is where my entire JWT information goes. The JWT contains information like client ID, client secret, things that are unique to that application. I put that in this associative array in order to be the secrets that are then going to be pulled. Then in order to access each of these pieces, I have other functions that access pieces within those associative arrays starting with the client ID.

I'm going show just one of these because if I went through all of them you would see all the pieces of the associative array and that would probably be boring. This is an example of getting just the client ID. Inside of this, I use the secrets to then get secrets which is my Box access key which then gives me the client ID. Again, this stuff keeps everything secret in order to pull from that one function because it's encrypted.

The thing I had to add that was a little bit different is related to the get box JWT token. This requires a little bit of a different finesse because JWTs requires assertions in order to basically get a new access token which requires a signature component. Because of the fact that that's a signature component, it's actually something that's considered sensitive material. My secrets.jsl includes functions that also process a little bit further in order to get that information from Box. In this case, my JWT function... I'm going to just wrap this so it's easier for me to see. My JWT function goes and creates that assertion with Box in order to be able to get the effort to send to get the access token. In order to do that I have to get the explicit information that I also have above for my secrets, which is my get box enterprise ID, my get box client ID, my box public key, my private key, and my paraphrase that's unique to my app.

From there I've got some, I've got some troubleshooting stuff for like if keys are missing, what's really important, and then in order to do the JWT assertion I actually had to do this via Python. This also includes another example of using another language inside of the secrets.jsl, which was also something I had to maneuver in terms of making sure things could be printed. That's why it lives in the secrets.jsl in terms of being encrypted and not in just the script that I'm using to use the information that's my JWT information. This is also a unique opportunity to see Python integrated inside of that JSL, which is pretty cool.

I then send that information to the Python script, which then pulls to get a specific signing for that JWT, and then from there it creates those credentials and then returns that JWT assertion so I can use it further down the line. Now that I have that JWT assertion, I can then get my access token. From there, I basically get my box or I get my box JWT token, which you just saw above, and then I use my client_ID and my client_secret to then create a token by taking that JWT_assertion, taking my client_ID, taking my client secrets, and using an HTTP request to then send that to Box. Then from there I'm able to get my new access token.

I also have some air-catching in here for individuals who are interested in digging further, but then it'll send my information relative to whether or not I actually got the access token and then returns the access token to the user.

Where is this used? How is this used? I'm going to show my example because there's some interesting conversations we can further have about access tokens themselves. In order to integrate this I have a generic folder that I'm going to show you for an example, but for my actual code here I'm just going to show some inputs.

This is me including that Box connection that's encrypted in my case because that's what you want to do after you're done with that secrets file. Then Box has a location or Box has a way of doing their folders such that they have ID codes and that's a way to uniquely know what folder you're looking at for files to search through. That's where you would put your unique folder ID here. I am specifically pulling just JMP files. This could be way more intricate in terms of what you're looking to do, but my example was just JMP files.

I also included a parameter here where I could look at recursion depth. Box is interesting because the way it works is that it's object-oriented. You have a folder ID, then you can have IDs within that folder ID, so you have to recursively search the folders. I allow my script to go as many layers as I feel fit, which in this case is five. Then I have a location to download where I want to put my files, and then I have a way I want to name them. What I'm going to do here in this example after I talk through how this actually works and some of the intricacies with access token that for security purposes, I'm going to show you my example which is that I'm going to pull some JMP files from Box.

There's some functions in here about recursively searching through JSLs. I'm not going to go through those or Python downloads, but what I will do is I will talk about the main Box connection. In this case, this is where we were talking about in terms of some security things that could be a little bit interesting in terms of where is the best place to put this for every company/every person where you might put this is a little bit different and in terms of security it's of different depending on which place you're talking about, but in my case this is where I left it. This main function, to start right off, I need to get the information from Box. In order to do that I have to get my access token. Getting my access token is part of that secrets.jsl that you saw that was encrypted. In this case, what it does is it gives me this access token. This is a variable that exists in this JSL.

In this JSL, because I'm getting this access token here, if I were to add a print statement on say line 583, you would be able to see the access token. Now the nice thing that's interesting or the nice thing about Box is that those access tokens expire. There's really no risk of somebody being able to use them later down the line. Gets smaller, but it is still potentially a security risk if you were to print this access token or get a hold of the script, run it, and print at that location.

Ideally or potentially, what you could do is instead of what I've done here where I put the Box access token here, I could instead move the authentication pieces, both the testing and the searching to my secret.jsl to then encrypt them. In that case, it would no longer be accessible and others wouldn't be able to see it. A little bit of a different use case, little bit of a different, you can use at your own leisure in terms of which way you want to put it. The safest way would be to put the Box access token inside the secrets.jsl and encrypt it so nobody can see it. But in this case, this is a good talking point, this is how I left it, so this is what it looks like. We can chat about it from there.

Then it goes through and gets the Box access token and then from there I can actually search Box access files because I now have my authentication piece that says I am who I am, and you should let me in. In my case, I have a demo, and in my demo I have an example where I am pulling from a file I have on my Box drive. That file area looks like this, and it's Secrets_JMP Discovery, File Examples_Secrets and I have some JMP tables that I'm going to pull and I'm going to concatenate from this location. In order to do that, I'm going to run this. Then we're going to do the tragic wait music because that's what happens when you run a script sometimes. It sometimes takes a second.

Authentication, tell me who I am, tell me who you are. Here we are. This is the file that I got. You can see my source tables, those were the names of the JMP tables I had in that Box location and these are the tables that have been concatenated together. Just a bunch of fun jokes.

That is what I've done with my Box access tokens and integrating with JWTs. I'm going to pass it back to Nathan.

With all of this that Sage has shown, and I showed, how is this really going to be potentially implemented and a way to get it so that now that we've done this, more than just me and Sage can do some of this work? What we've done is we've put this onto GitHub. Here is a look at GitHub. We have the repository. This is also linked with the presentation, and we have our examples in here.

Now this is something where, depending on how you want to organize it, there are a lot of options you have here. But the main thing that we've put together is, here's the secrets template. This is what I had showed you earlier with that really base value. Then we have our different examples here. Within these examples, we have basically three main files where we have, here's our template that we would show that has all of our pieces of information in it. This is where you can fill this out to be your own personal or company information, and this is what gets what you do for the encryption. What makes this safe, is what we've recommended you do is you take this template, and you save this file in your repository or your clone and replace template with the word raw. That way, that is where all of the really important things are hidden. What we've done is in this gitignore, we will ignore, the raw.jsl. That way, those stayed local for you to work on. You have your credentials. You keep your credentials. Then you encrypt your raw file, and you upload it as the encrypted version of that file.

Now the only thing that's in GitHub should be your encrypted version that is workable. In each of these folders, we have docs that are specific to their usage. How much you want to share both in your main README or your docs is really dependent on what you want people to know in terms of how to use this information. If you have something, or you're utilizing something like Project Jab, which Sage and I presented on last year, that's a way to build add-ins and allows you to potentially pull files from individual repositories. That's a way that you can just pull in the secrets files you need to utilize to be able to do what you need to do. That's a nice way to keep this available, but also maintain various levels of secrecy.

As Sage alluded to, if you're in a situation where your code is getting a little bit more complicated, you may be put in a situation where in order to maintain that level of secrecy, you end up putting more and more information into this encrypted file.

Like Sage was talking about in her template, she has all of these functions here that do a lot of work, but it's when she went into her example, she pointed out that very specific access token. I can't remember exactly where it is in the function, but that is something where if you didn't want that in your main example for people to see, you would just add that function into your template or your templated raw file and then encrypt that, and then it's all hidden. The layers you have to go through, I think will be dependent on your use cases and what you're looking to do. In Snowflake, it turned out to be relatively easy because I was able to use the connection directly or simply scrub a table. So your mileage may vary and what you need to connect to, whether it be APIs or things like that.

But I think that is a relatively easy-to-digest and utilize manner to go through your secrets and be able to offer many layers for those to improve on their background coding while still maintaining solid cybersecurity practices to keep things clean and kosher.

I think that ends everything we wanted to talk about today. Thank everyone at JMP very much for giving us this opportunity and thank all of the watchers for giving this a look-see.

Totally agree. I hope you are keeping your secrets safe. If you want, you can come find us at JMP Discovery this year because we'll be there.

Yes, we will. Thank you.