Thank you all for clicking on this talk and coming to watch it.
This is really about bias in data.
Every analyst that works on an project understands
the importance of ensuring that the data that they collect is unbiased.
The steps are taken to avoid that at the start of the data collection,
before the projects even really begun,
there are numerous checks that points along the analysis,
and then at the end, any conclusion reached is taken
in the context of potential additional sources of bias.
But this same level of care isn't applied to online searches on topics of interest.
Search engines use algorithms that are designed to deliver
personalized content that is relevant for us as individuals.
Now, this has advantages.
It means that return search hits
are more likely to be relevant and interesting,
but it also has disadvantages.
By definition, these are not unbiased.
We have an example.
Yeah, I heard they brought up a great point there,
in science and engineering,
we take a great care to make sure that our samples are unbiased.
But let's think of a library.
We walk into the library and there's two people interested
in informing themselves on vaccine safety.
Let's say they walk into a library
and ask a librarian for these books on vaccines,
so the first person receives three books.
These are actual books.
Smallp ox: A Vaccine Success ,
Anti-vaxxers: H ow to Challenge the Misinformed Movement,
and Stuck: How Vaccine Rumors Start and Why They Don't Go Away.
Now, let's say a different person walks in
and receives three completely different books.
THE COVID VACCINE: A nd silencing of our
doctors and scientists,
Jabbed: H ow the Vaccine Industry,
Medical Establishment, and Government Stick it to You and Your Family,
Anyone Who Tells You Vaccines Are Safe and Effective is Lying.
These are actual book titles.
Let's say that looking at who you are,
so where you live, how old you are, your gender,
maybe even your browser history
determines which of these sets of books you get.
This is essentially some of the problem with the bias
as you go in to search for things online.
It may be that before we even start
looking at our browser search, we've already got bias in there
and we want to understand if that's the case or not.
That's what motivated this.
You got any thoughts on that, H adley?
Well, I think that the thought that I'd like to express right now
is that the purpose of this presentation isn't to judge or to opine
on the advantages or disadvantages of the search algorithms
that may or may not be used.
The purpose here really is just to take an example
of complex/ unstructured data and complex because it is unstructured
and this was search results.
Then to use some of the exploratory visual
and analytic capabilities found in JMP Pro 17 to try to understand
what we were seeing and to present it in such a way to help you to understand.
The purpose of this presentation is to inspire you to try these techniques
for yourselves and others like them on your own data.
Let's go through briefly the methodology.
What Pete and I did was we came up
with a few search terms we thought would lead to interesting results.
You can see those terms here.
We define some potential input variables
which may or may not be affecting the results of the search.
We know that there are very likely others as well that we didn't include.
This is true with any designed experiment.
We can't capture every variable, but we took a few.
We'll see whether these are significant or not.
We developed a data collection procedure
whereby we use the MSA design in the DOE menu.
This is a convenient way to create these tables that we can then send
to JMP SEs and friends of SEs.
Now, right away this isn't an unbiased random assortment
of people we've asked to fill out these.
They're all people that work for the same company and have the same job title.
As we said, the purpose is really
to understand the techniques and methods that we use to try to understand the data,
and then to think about how you can apply it yourself.
We explored the results which we'll show you.
Then finally we presented the findings at the JMP Discovery Summit America 2022,
which is what you are watching right now.
Without further ado, let's jump into the data.
I'll start out by talking just briefly about the MSA design that you see here.
What we've done is we've added the factors of interest,
we've added the terms of interest that we were looking at.
then the nice thing about this is that when we make the table,
what we could always do is press this button,
send these out to everybody that needed to complete the results for us,
send them back, catenate them
and then we're ready to start beginning our analysis.
But as any analyst who's ever collected data
and tried to analyze knows the data very often isn't in a format
where you can immediately start with your analysis,
some cleaning needs to be done.
I'll pass things over to Pete to talk about that.
Pete?
Yes, great point.
I think everybody has gone through this.
Even with a well- designed DOE,
you oftentimes have to make some adjustments to do the analysis.
Hadley showed those operator worksheets
that came out, and here is one that I filled out.
I'm not going to keep myself anonymous,
but I didn't want to share someone else's results.
But just to give you an idea, we had folks answer
a few demographic questions that hopefully weren't too revealing,
but basically where you were located,
how old you are, and then the search term you used.
Like Hadley showed, there was three responses.
We had people do this search and then show the top three responses
that that search engine recommended.
To do the analysis,
the first thing we had to do is just take these three and bring them together.
This is a nice easy way to do this,
is to go under the columns, utilities and just combined columns.
Now I just called these responses and made a little delimiter.
I unchecked that multiple response 'cause we're going to just do
text analytics on this.
Then you get this,
which is the table that we, excuse me, the column we're using for Text Explorer.
Now, I did this and then I brought in all of the results
from all the different people who took the survey
and tweaked a little bit more like combined whether you were in the US,
which state you were in, and then summarize that into region
between America's and Europe,
'cause we didn't have enough respondents to break it up by state.
But in the end, we end up with a table that looks like this.
We had to do a little bit of recoding,
we had to do a little bit of filling this in
and then anonymize that search engine.
The folks that got the survey knew which search engine to use,
but we're not sharing that here.
Hadley is going to now talk about some
of the results we saw out of this once we had it in this form.
Let's open up this dashboard right here.
What you're seeing are the most popular terms
in order of popularity from left to right, descending order
for the first response, second response, and third response for every one
of the search terms, for every gender and age and all of the other factors.
W can use this
and this hierarchical filtering on the dashboard to explore
this a little closer, see if we can learn anything.
One thing I happen to notice if we look
at the world is and we click on male, you'll see that for most people,
or for many people, the first search they found was that the world
is not enough if you're female, equally likely is to find it your oyster.
Interestingly, if you're less than 40 years old,
that's when the world is not enough, suddenly becomes the world is yours.
I think we could probably agree with it's probably true for people
under 40, isn't it?
What else have we got here?
If I look at climate change,
so another of course hot topic of interest these days,
as well it should be.
If I were to look at people over 50,
apparently a huge concern for them is whether climate change
is changing babies in the womb,
which, interestingly, isn't a concern for people below 40.
I wondered whether
this is a valid concern for people over 50,
whether they're more likely to have their babies change
in their wombs or not.
But aside from that,
let's take a step back and see how we can go about creating this dashboard.
It's quite simple.
The first thing we need to do is to create our filter variables.
I've done that here.
Here's our search terms and our distributions.
What I'll do is I'll go through how to create the graph builder report,
because that's something that you may not be familiar with,
that you might be interested in doing.
I'm going to take my first response,
put it here, and simply choose the number that those occur.
Then I can right click and order by count descending.
T hat's it.
I've done the same thing for my second response
and my third response as well.
Now we can go ahead putting together the pieces of the dashboard.
We'll click on new dashboard,
we'll choose the hierarchical filter plus one.
I'll take my distribution results put them there,
my input parameters, put them there,
and then my graphs.
Let's see, is this one first?
Well, I cant tell.
We'll just put them in order like this,
and I can always change the order if I want.
All right.
I'll run the dashboard,
and there we have it.
It really is as simple as that.
Then I can go ahead and save it to the table.
That was one use of a dashboard.
I'll show you another use of a dashboard,
which was to use it with a Text Explorer Word Cloud.
This is the most common words,
not just entire phrases or entries, but individual words.
You can see the word design seem to be used a lot.
If I were to look at, for example, statistics,
so it looks like everybody can agree that statistics is a science.
Interestingly, if you're in Europe, apparently you find it harder than you do
if you're in America, where that doesn't come up,
so something I happen to notice there.
To create this dashboard, it's very much the same as the other one.
We'll add our distribution items.
Here's the first one, here's the second one.
We'll add our Text Explorer Word cloud,
and then we'll simply put this one together
just as we did the previous one.
With that, I'd like to thank you
for this part of the presentation about the exploratory visual analysis.
I've shown you how you can go about
doing this using the hierarchical dashboards.
Now I'll turn things back over to Pete,
who will take us through some more in depth use of the Text Explorer.
Perfect. Thanks, Hadley.
Like Hadley mentioned this is
a different way to display this, but this is the end result of using
the Text Explorer and looking just at the Word Cloud here.
He had made this a dashboard
and used filters that were graphical of nature, which is great.
You could do this also with a local data filter.
But this is basically the end result we're going for.
Let's now back up and talk about how we got here.
With our data set over here, we just launched the Text Explorer
under analyze menu,
we put in our column that we're interested in.
In this case, all three responses combined into one column.
We have a bunch of options we can use to tweak this,
including language and how we tokenize the words.
But we're going to go ahead and just use the defaults.
Here you can see since we have different responses
to different search terms,
the overall term and phrase list by itself is not super informative.
What we would want to do is apply that local data filter
and the first thing we'll look at is that search term.
Now we can do something like the economy or coronavirus
or climate change and go from there.
Let's focus in on climate change here.
One thing that I wanted to do was add some sentiment analysis.
The first thing I'm going to go ahead
and turn on this Word Cloud so it looks like it had before.
Now we can display it this way where you
have the most common term in there and you can see it's climate and change.
We know that we're searching that,
so I could go in here and add these as stop words
and now see which ones come up
the most frequently when we're mentioning climate change.
This is one way to display the Word Cloud.
I can also go through here and maybe change this to something
that is a little more appealing to the eye,
but maybe less useful from a quantitative standpoint.
You can always add some arbitrary colors
if you like that as well.
All right, so I've done this to this point,
but now I want to add some sentiment analysis to this.
Are people thinking climate change
is natural, a good thing or is it a bad thing?
You can see some things in here that maybe indicate that,
but I wasn't quite sure where to find sentiment analysis.
With JMP 17, we have this new feature called Search JMP.
If you're ever looking for an analysis in JMP,
this is a great way to find that.
If I just start typing in sentiment,
you can see right here that it tells me how to find this,
I can do the help, but I can also just hit show me and it launches it right there.
If I'm ever wondering, hey, how can I do this, this gives me the option.
Now, a couple of things you see here
it's identified some of these default terms
that are providing sentiment.
Things like good.
If I click on good, I get a little summary.
It looks like when people are saying good,
that is actually a positive sentiment.
Now, what about greatest?
Oh, boy, almost everything that says greatest is a greatest threat.
Maybe that's not actually a positive sentiment there.
We might need to do a little bit of tweaking.
First let's go in here and say, okay, well, greatest threat is a phrase
that we're seeing commonly.
I'm going to just add that phrase.
Again, you would do this in your curation process,
and now you see that that goes away.
But I think greatest threat is actually a negative thing.
Let's look at those sentiment terms.
You can see JMPs identified that as something that maybe has sentiment.
I'm going to just say, you know what?
That's a really negative sentiment.
Now when we go down here,
you can see that it's flagged those seven occurrences
where they mentioned greatest threat,
and it said that those are a very negative.
That's changed our overall impression of, do most of these search terms
think this is negative or positive?
That's just an example of how you can
walk through that flow and come up with the end sentiment analysis.
I'm going to pass it back over to Hadley
and let him wrap things up here.
What I'd like to say is that we showed you,
first of all, how we went about using
the MSA Design to help with the data collection.
We use Recode and other items
in the Tables menu to help with the data clean up.
We then used Distribution, Graph Builder,
Text Explorer, and combinations, all of them together to help
with the data exploration,
see if we can uncover anything interesting.
Then Pete used Sentiment Analysis together with the Search and JMP 17
to see what else we can learn about the data as a whole.
With that, I hope you found this useful.
I hope it's given you some ideas about how
you can do this on your own data for yourselves.
I'd like to thank you all for listening,
and I hope you enjoy the rest of the JMP Discovery Conference.
Thank you.
