My name is Kemp Wu. I come from Taiwan,
and I work in Applied Material for Continuous Improvement Group.
Today, I will present BB JMP Exam Item Analysis.
First, let's do the project adjusted in 2023.
The entities sub way used in GB/BB training course
is changed from mete r to JMP.
To modify the AGU,
GB/BB exam questions based on JMP survey is necessary.
In this project, we use Item Analysis to study the 2022
BB JMP exam questions to support this GB/BB exam questions change .
Item analysis will help us to identify the difficulty and the discrimination
of each questions. For GB we need easier questions
because it is data label requirement for the beginner.
We won't want too many hard questions inside GB exam.
When we move to BB exam, the questions should be harder and better discriminated.
For the last year,
for the three chapter of the BB JMP training programs,
we have, step one, is the Intermediate Statistics
and step two, for the Graphical Entity and step three is MSA, PCA and SPC.
Based on that, we would like to identify each questions difficulty
and discriminations and by day, if any opportunity to modify
the training material is sent questions based on the NAC results.
Also the capability of the questions to judge the ability of the trainees.
This is the last year field JMP insignation results.
For the step two, is the Graphical Entities.
It's very basically for the reading, also the drawing for the users.
For the step three, is the MSA, PCA and SPC.
This is more advantaged.
For the best one, we got a more higher score for the average.
Then the advantage f or the statue, the score is more lower.
If we want to be used the based on the score,
it's very difficult to separate for the GB or the BB, the questions.
That's why we come out. We need to use the JMP Item Analysis.
They can deliver more information for us to do the data analysis
for such as the questions' difficulty or the discriminations.
How to use for the JMP Item Analysis
application on the GB and the BB installations.
F irst, let's look at the characteristics curve.
Characteristics curve control the relationship
between the probability of quality answers.
The item and the ability for different probability labels
on the ability scope also show
correlation between for the difficulty and the discrimination attributes.
There are two parameters in the used parameters curve.
First is the a parameter, is about for the discriminating power.
Another one is the b parameter,
is about the item difficulty for the left button of the chart.
For the y-axis is the probability
of the quality of the answers from 0-1. One is meaning for the 100%.
For the x-axis, is the abilities from minus 4-plus 4.
More higher number is being more difficulty.
Also more ability for the center.
For the periphery,
if the curve is more steepness of the curve it mean is more powerful
for the discriminations.
For the parameter b is above the difficulty.
We can see for this chart, the right hand side of t his curve
is more difficulty than the left hand side.
For example, the ability for the deal
to go up to match for the right side of the curve is about just only
about the 10% the corrected. If compared with the left side
of the curve, it can go to about 90% can be corrected.
Also we look at the example for the question 24,
b is the vertical line of the preferred of the discriminations.
And the right hand side for the question 12 is for the difficulty,
for the terratinous curve [inaudible 00:06:28] curve
from the bottom left to the up of the right side.
Next we do add the parameter estimated.
The parameter estimated, it show the item
which is, each of the questions showed by the difficulty
and the discriminations.
It show by the numbers also by the histogram.
For the difficulty, easy one is less than zero
and the hard questions is over zero.
With the discrimination, it is over plus and minus three.
The up is, there's no discriminations.
This example for our result for the step one, we have nine questions
to show by the parameter estimated for each of the number.
Also for this parameter estimated,
the number also will be showing in the characteristic curve.
We separate for the three group
of the data based on the difficulty and the discriminations.
For the group 1 is the green zone. It is the easy with no discriminations.
We can see for the red virtual line , it is less than zero .
Even the question is easy.
Also with the fake curve,
for this blue curve it means is no discriminations.
For another one is the group 2 is the easy with the discriminations.
We can see the grey line is less than the zero
or another one is a vertical line.
This show for the over the blue line and here it can be 100% correct.
If left lower than this line, it mean is probably very hard to have the correct.
Next one is the group four. Group four is the hard questions
with the discriminations.
We can see the ability is over the needle.
It also showed the virtual line for the blue curve.
It says group 1 have no sample so we didn't find out there's any show
by the glossary.
Next we will go for the Dual Plot. The Dual Plot report contain a plot
that show question difficulty and the student ability in the same scale.
I t will show on the Y scale for the ability from minus 4- four.
There are some three zones.
The first zone is this area. This is about the question difficult.
More higher it being the question is more difficult
and more lower it being the question is more easier.
Another one in the middle zone is for the student ability.
More higher it being the student have more ability
and more lower it being the student has low ability.
How to compare with the question difficulty and the student ability?
For example, the student 2 to compare with the question 28
because the student is more higher than the question 28.
The student 2 he have more probability to answer correct questions.
The obvious is for the student 1. Student 1 compared with the question 15.
It means the student 1 is very hard to correct the answers.
A lso for this one is, the right zone is for the group shown by the histogram
is shown for the two group of the student.
From minus 2-0 is one group and zero -2 is another group.
For this example, it show for the two bimodal more
the upper zero is for the G B and the lower zero is for the BB exam .
Another one is the group three STIPS descent result
to show by the dual plot.
First we go for the cell plot to see the reference for each questions.
For x-axis is for each questions and y-axis for each of the students.
For the red color, it means the answer is right,
and for the blue color the answer is wrong.
We come to see the dual part.
For the step one you show the pattern is…
We see the student number 5 is more high performance than others.
Step two they have too good of the students.
Step three,
this group of students all better than zero and also show
the students six is not that good,
but if compared with the student 1 you have very long difference.
It means the student 1 is the outlier.
From this we have four way for the improvements.
The first is we can base on the ability zero to separate the upper side
is for the GB and the lower list zero is for the BB itself.
Second, we can base on the discrimination to show a different kind of happen
like state 1 we can find the most performance student.
For the state 2 we can separate from the people have good study
or no study for the same.
The third one is the question difficulty such as this question.
None of the students can be correctly to answer this question.
Maybe there's a material need to be improved
or some area we didn't teach for the students.
The last one is we also can find
who is the student need to have more extra coach such as the student 1,
is [inaudible 00:15:05] .
Maybe we need to pay more attention on him.
All of the essential can be decided by you.
You can see from the door part to see if the ascent achieve your expectations.
Now I will show you the real case on the issue of the group of the questions.
The group 1 is the easy with no discrimination.
This is the question of this
and this the issue of the session or the answers?
For this question the answer is D.
It is very easy and simple for the JMP operations
and it can be found in the training material.
You can see for the cell plot, most of people are say correct
and another one is the group 2 is the easy with these questions.
Is about for the parallel part and the answer is B, negative correlation.
Even this parallel part is easy but if the student,
they don't really understand how to read the parallel part,
they cannot understand how to say the correct answers.
We are going to see the group 3 with no discriminations.
This question is hard also in the training material
is the guide the way let you to say the answer D.
We can see most of people are saying wrong for these questions.
Maybe we need to reconsider is there any training material need to be corrected?
For another one, the group 4, hard with the discrimination questions.
Even that's all the question.
It looks very similar and hard to decide.
But if we pay attention on the case or making the nose,
you will understand the answer is the B.
From the item analysis we can be analyzed for the issue of question
for the difficulty and discriminations to dive into for good of the questions
for the continuing improvement opportunity.
According for good of the questions,
we can deliver for the GB and the BB proposal.
For the GB we will consider for the 30% for the easy
and the no discrimination for basically questions for the students
and for 60% for the easy and with the discrimination to separate
the student who has studied and not studied.
Also, we consider 10% for the high- performance student
and for the BB we are not considered for the best questions
for the BB due to you need to be more
difficulty and we consider 30% for the easy with discrimination,
not every question need to be hard and also for the 70% for the hard
with the discriminations but we not consider
for the difficulty and the discrimination
and the duration of the questions is for the reference.
Of course, it depends on the ascent requirement
to adjust the percentage for the ascent.
Last I will be talking about for the takeaway learning
the JMP item analyze can help to organize GB and BB exam questions based on JMP.
The item analyze help to detect difficulty and discrimination of the same questions
for instructor identify any gap in teaching efficiency.
Also modify GB /BB training material based on JMP item analysis models.
Last one it can be proliferate to the internal training programs such as,
8D program or the Lean training.
Thank you very much.
