Good morning , everyone . Good evening , everyone .
I 'm Sukti Chatterjee .
Before starting my presentation ,
I would like to introduce myself with few words .
I 'm Sukti Chatterjee from CTO team of Applied Materials .
It is advanced technology team , and our team goal is to develop product
adjacent to the semi industry or outside the semi industry .
For example , we are working for aerospace industry ,
pharmaceutical industry , or industrial coating .
This example , present example ,
we are taking from the pharmaceutical industry .
My topic of the presentation today.
Gauge R&R of X -ray photoelectron spectroscopy to monitor a coating process .
Agenda of my talk today.
Fi,rst, we will talk about the background and problem statement ,
then we will discuss about the operation definition and data collection plan .
Next , MSA component analysis .
Finally we 'll talk about the plan for MSA component improvement .
Let 's start with the background .
In the pharmaceutical industry , in therapeutic windows , therapeutic areas ,
there are , for example , antibiotic drug , alcohol addiction or cancer patient ,
they need everyday injection because drug level in the blood
is certainly increasing , spiking in the blood ,
and then within short time , within a few hours ,
it is going beyond the therapeutic window limit .
That 's why they need everyday injection and it is painful .
It causes some side effect ,
that 's why patient skipping the medication or stop the medications .
To solve this problem ,
our customer needs some approach to tailor the release of drug .
Our team developed a barrier layer , aluminum oxide barrier layer ,
that forms a shell around the pharmaceutical particles .
Properties of this barrier layer
can control the release of the drug in the blood .
Even it is possible like that ,
it can release few weeks instead of few hours .
Here , we will talk about the composition analysis ,
and what is the noise analysis of this composition measurement ?
That we will talk here .
Our problem is measurement of AlOx coating composition .
Our spec limit , customer spec limit , is O /Al ratio
in the aluminum oxide film is 1 .2 -2 .3 .
Our objective here to determine the XPS method if it is adequate
to differentiate AlOx process variation .
We will determine here , gauge R&R measurement error
of XPS for AlOx composition analysis .
X -ray photoelectron spectroscopy can measure quantitatively
atomic percentage of composition .
It can measure aluminum and oxygen percentage .
XPS actually measure the kinetic energy
of photoelectrons emitted from the elements
and it counts the electrons .
Whenever it is counting the electrons , it can count the presence of elements
and also it counts the element which is bond to it .
That 's why we can get the information
about aluminum and oxygen in the aluminum oxide film .
Most of the source of error for XPS , it can add it in gauge R&R .
It can reproducibility , it 's coming from the calibration electron count .
It can add repeatability and reproducibility error .
Analysis can add reproducibility error .
We will talk more details this one in the next slide .
In our operation definition , we 'll talk about the different steps
of the XPS measurement and how it can introduce
the error in the measurement error GRR , gauge R&R error .
Our objective measure aluminum oxide coating composition ,
and to measure it in XPS , first we need to do baseline correction .
It is automatic , and then we need to go to the calibration .
In calibration , normally applied materials have calibration sample ,
especially whenever we have some
developed technologies like aluminum oxide .
But in our cases , we are coating pharma particles
and our process window is totally different
from our applied materials core technology process window for aluminum oxide ,
because coating need to be compatible with the pharma particles .
We are coating this particle , at the same time ,
we are coating also silicon wafer and API pallet
because XPS cannot measure particles .
It needs some planar substrate .
That 's why we are depositing on silicon wafer and API pallet .
Since we don 't have calibration sample , we are using the second option
for calibration like carbon peak calibration .
Left -hand side picture , you can see carbon peak calibration
and it is manually need to do it,
and that 's why it impact on the reproducibility .
Then after calibration , we need to do XPS survey
or high resolution scan to get the spectra.
In the spectra , you can see oxygen peak, aluminum peak .
Since we need to do manual calibration and we have automatic baseline correction ,
this can impact error on repeatability and reproducibility .
Next , we need to do analysis . Analysis is peak fitting .
We need to fit this peak and then we can gauge .
From peak area , we can calculate the oxygen aluminum percentage .
Since it is semi automatic , it can add error in the reproducibility .
By XPS measurement , we are calculating O /Al ratio
and our customer spec limit is 1 .2 -2 .3 .
Next , we will talk about the cause and effect diagram ,
MSA cause and effect diagram .
In MSA cause and effect diagram , we did some detailed analysis,
and we found several one it can impact on the gauge R&R .
We highlighted also major ones like electron counts ,
calibration analysis , we talked in the earlier slide .
Now we 're adding another one , it 's sample loading ,
how it is added error in the gauge R&R .
Sample loading , we need to do it . It 's not automatic , it is not full wafer .
We are doing with coupon wafer , so we need to place the coupon .
If it is location a little bit different or angle is little bit different ,
then it can impact on the measurement .
This is impacting on reproducibility .
All other major one impact we already discussed in the previous slides .
Other one is the sample . It depends on the process .
For this presentation , it is out of scope .
We will talk about these four in this presentation .
Next , our sample collection plan .
For our sample collection , we use six samples for MSA analysis ,
and for these six samples we have four replicates .
Here you can see these four replicates .
We measure those samples in two sites .
Since we have the four replicates , we are measuring those sequentially .
Is it possible that if samples are degraded
then sample degradation could be a risk ?
We will talk about this risk later on more details .
Our expected outcomes like that we need to find out XPS method
is adequate to differentiate process variation .
Also , you like to gauge like that
whenever we are measuring two sites that have similar result .
Also , we like to gauge like that sample or part is not interacting with the site .
Now we need to do the MSA design .
In MSA design , we are using substrate and site at the cost factor .
This is the site , this is the part , and we have also two different substrate .
We mentioned it before , API palette and silicon wafer .
S numbers are silicon wafer , A numbers are API palette .
We 'll not be able to use actually completely randomized option
and we use first repeat .
For first repeat option ,
here we are not changing the sample replicate number .
That could impact on sample degradation problem .
That 's why later on we will compare first and fourth replicate
to check this sampling risk .
For this MSA analysis ,
we sequentially use several JMP applications from JMP platform .
We use data distribution of MSA samples .
That is from descriptive inferential statistics application .
It is from distribution fit Y by X .
Then we check the data variability using control chart and one -way ANOVA .
Then we analyze gauge R&R components .
It is from the variability chart .
Then we 'd like to gauge like that what is the relation
with process capability with gauge R&R .
That we can find out like that , interclass correlation versus P /T plot .
Next , we did the root cause analysis to plan for improving the GRR .
We will find out what is the GRR major error
and how we can find out .
That we are using for box plot , density ellipse ,
matched pairs , and fit line
that are different platform of JMP platform .
Let 's start with the data distribution .
We developed process initially at the two spec limit ,
upper spec limit and lower spec limit .
In upper spec limit , we have two samples , two parts and four replicates .
All are measured two sites ,
and we already mentioned we did the first repeat .
Similarly , at the lower spec limit , also we have four parts ,
four replicates and two sites .
Since we did the process development at the two end of the spec limit ,
that 's why we can see that our distribution is bimodal .
It's completely bimodal distribution .
Problem of bimodal distribution , it can impact on the GRR components .
It can impact on P /TV ratio ,
it can impact on P /PV ratio and misclassification .
Since P /T ratio is not related with ...
It is not dependent with the part , that is the reason P /T ratio
it 's not impacting by the sample distribution .
That 's why we will be used in our following slides .
Our figure of merits we are using as a P /T ratio .
For misclassification probabilities , there is five probabilities .
Last three , it could be impacted by the sample distribution more ,
and first two is less impacted .
To minimize the risk , again ,
we are focusing on the P /T ratio as a figure of merit .
In the next time , our plan to do MSA analysis
using uniform sample distribution .
Let 's check now the variability of data .
Here we can see that we use I -MR chart ,
individual moving range chart , and we saw that many data points
are outside the control limit in the upper chart ,
and in the lower moving range chart ,
we saw that three data point is outside the control limit ,
and that these three data points , it is sudden shift .
It is sudden shift , it 's not staying there , it is going back .
It means it is the type II shift
and there is a mixture of common cause variation
and special cause variation in the control chart .
That 's the reason here control limits are meaningless .
We need to subgrouping with special cause
and then only we can consider the control limits .
Now we like to find out what are the special cause .
First we will check if part variation could be a special cause .
We did it using the one -way ANOVA and in one -way ANOVA ,
we can see there is a variation of the samples .
We did the process near upper spec limit and we did the process lower spec limit .
That 's why samples are different .
That also we found by one -way ANOVA , and here we can see that
within variation is very small compared to part variation ,
and also by analysis of variance is showing like that .
Here our hypothesis is all parts are same ,
but it is rejecting the hypothesis because P -value is less than 0 .05 .
It 's telling us it is significantly different .
That means part variation is a special cause ,
so we can use as a candidate for subgrouping .
Again , similarly we check with the site variation if it is a special cause or not .
We considering two sites measurement near upper spec limit
as well as near lower spec limit .
We saw that here our hypothesis is two sites are measurement similar,
and we found that its P -value is higher than 0 .05 .
For upper spec limit ,
there is no evidence that we can reject the hypothesis .
It is similar , on the other hand , for lower specs limit .
It is marginally rejected because it is less than 0 .05 .
For site variation , either it is marginally rejected
or there is no evidence to reject .
That 's why site variation is not a good candidate
and part variation is the better candidate .
What we did next , we make our control chart again with phase option and A here ,
sampled part at a different phase .
When we do it , we saw that in a moving range chart ,
we found change in the variation in the measurement in the moving range ,
and that calculated the control limits for the bottom chart and the upper chart .
Now we saw that all the points ,
all the measurement points are inside the control limit .
These is the variations of each sample . It is the repeatability .
When we consider site A and site B ,
and we saw also site B has also repeatability .
But compared to site A and site B , there is some variation of repeatability .
That is called reproducibility .
Now we calculate the gauge R&R , all the components in the next slide ,
and we 'll find out what is the dominating error in gauge R&R .
First , we did main effect .
We didn 't consider for the main effect part and site variation interaction ,
so only the main effect .
Here , we saw the repeatability , reproducibility .
Repeatability is 22 % and reproducibility is 15 %.
I already mentioned as a gauge R&R , we are considering P /T ratio
because our sample distribution is bimodal ,
and we saw that P /T ratio is 26 %.
It is passed , it is less than 30 %.
It is marginally passed , and major error is 22 % repeatability .
One more thing I should mention here , we are considering P /T ratio
but P /TV or P /PV ratio is very close for our measurement cases
because our sample distribution is bimodal and at the two end of the spec limit .
That is the reason this ratio T or TV are very close or PV is very close .
That is the reason we have this gauge R&R .
This figure of merits is very close .
Also , I should mention here type I error alpha and type II error beta .
Type I error , all our data points within the control limit .
That 's the reason our type I error good part is falsely rejected .
It 's very small . It is less than 6 %.
On the other hand , type II error , it is 6 %, it is failed .
It is more than 10 %.
Why type II error is higher ?
Our repeatability is the major issue .
Whenever we are measuring the samples , it is within the spec limit .
But it is possible like that whenever a customer is measuring it .
It could be beyond the spec limit because repeatability is high here .
At this point , since we are developing the product ,
we are in the initial feasibility check phase .
Customer is happy with this beta type II error ,
but we have option .
If we can improve the repeatability , then it can improve this part also .
On the other hand , if we can consider that part and site interaction ,
then we saw that part and site interaction is 6 %,
not that much ,
but there is a little bit interaction.
And when we didn 't consider the interaction in the main effect mode ,
then this interaction is added in the repeatability .
That 's why we found that whenever we are considering the crossed effect ,
we saw repeatability little bit decreasing
because our interaction is very small , not that much decreasing .
Since this interaction is very small ,
our figure of merits are not changing that much .
It is changing from little bit .
Now from here ,
we know that our dominating error is repeatability .
Before going about the more discussion with repeatability ,
first another thing I would like to mention ,
process capability with gauge R&R .
Effect of gauge R&R on the process capability .
Here , process capability we are plotting in ICC versus P /T plot .
ICC is the part variation to total variation
and P /T is the six sigma gauge , and USL minus LSL .
We calculated from here Cₚ , and in our cases ,
in our process current condition , Cₚ is 0 .93 .
It is less than one .
It is in the red zone , and we need to go Cₚ ...
For a good process capability , we need to go between Cₚ 1 .33 -2 .
It is the yellow zone .
To improve this Cₚ , what we need to do ?
In this part , this is the process part
and in this direction , it is the measurement part .
Process variability or part variability is very high .
For our measurement , we saw that our P /T is 24 %.
If we would like to increase , if we would like to improve the P /T
from 24 to suppose 15 % or 10 %, then we have to improve 30 % -50 %,
and within that , our repeatability is the main issue .
That is the reason we need to improve the repeatability .
Now it is question .
If we need to improve the repeatability , do we need to change our measurement tool ?
That is again depending on the ROI that is question to our managing level ,
or we can address the repeatability in different way .
That 's why we 'd like to find out the root cause why repeatability is higher .
Here we are considering variability chart with analysis of variance .
Here we can see that we plot all the samples variability together
with site A and site B measurement .
You can see that suppose , for a sample A0 ,
this is the measurement repeatability , and it is changing .
This repeatability is changing for all the parts .
Also repeatability is changing with the site to site
because here you can see repeatability is 0 .06 standard deviation ,
but in these cases when they measure their repeatability is 0 .03 .
That is the reason this repeatability is changing
with part to part also site to site .
Whenever it is changing with site to site ,
it 's called reproducibility .
Here if you can consider the analysis of variance ,
then we can see that site to site variation is much smaller
than within variation .
This is the repeatability , within variation , and site to site .
Site to site variation , it is reproducibility , it 's much smaller .
Repeatability again from here also we find out that it is the bigger problem .
Now in the next to find out the root cause ,
we plotted all the repeatability side by side together ,
and for both the cases , USL , upper spec limit and lower spec limit ,
and all the cases we found that its repeatability is different .
Next we like to correlate or find out any relationship
if it is present site A and site B measurement .
Ideally , site A measurement will equal to site B measurement should be .
But in our cases , we did some linear fit and we found that we have intercept
as well as we have linear fit slope , it is not one , it is not zero .
Here we found that linear slope is less than 0 .4
and intercept is higher than 0 .9 .
Our fitting points are distributed widely .
That 's the reason our R -squared is also poor .
We also did the density ellipse
and density ellipse also telling that this correlation is less than 0 .5 .
If they have a very good correlation relationship ,
then it should be 0 .9 .
If it is 0 .6 , then it will be moderately correlated .
But in our cases , it 's not that .
That 's why we know that site A measurement is not site B measurement .
It 's the repeatability impacted on the reproducibility .
Problem of repeatability is impacted on the reproducibility .
Now we check more closely how it is different .
We are comparing by match pair the site A and site B variation .
Here our hypothesis is site A equal to site B ,
that means site A minus site B equal to zero .
We saw that our probability for this hypothesis ,
site A minus site B equal to zero , is less than 0 .05 in both the cases .
It is upper spec limit and lower spec .
Both the cases you can see that it is probability is less than 0 .05 .
That means site A and site B measurement is different ,
and you can see our difference of mean value
and confidence interval is above the zero point line .
That means though this is
site A measurement is always higher for site B measurement .
Now from here , our question appears , since we did the first repeat analysis
for our MSA design is first repeat , it could be possible like that
if samples are degraded , like O /Al composition is degraded .
That 's why we did again match pair test
with first and fourth measurement
both in site A and site B for all six samples ,
and we found that here ,
first measurement minus fourth measurement equals zero .
That is our hypothesis .
We saw that P -value is higher than 0 .05 both the cases .
That means our sample degradation is not an issue .
First sample , there is no evidence .
First measurement and fourth measurement is dissimilar .
That means it is the measurement issue .
For that , this is summarized in the dashboard table ,
in the dashboard , like our figure of merit
for gauge R&R 24 % and repeatability is 21 %
and that repeatability is changing from part to part and site to site ,
and we have always higher repeatability for site A compared to the site B .
Now for our next plan ,
we plan for a discussion each site as well as with the process team .
Site has a problem like repeatability as well part -site interaction .
We know that what error could be introduced in the measurement
like background /baseline correction , electron counts , peak deconvolution .
We 'll discuss those methods source of error with site A person ,
site A facility , and we will find out
how we can do the streamlining process for improve our MSA .
Also we have a plan set up a calibration sample
or we can set up a set up sample
that we can measure in regular interval in the both sites .
On the other hand , with the process team , we 'll talk to improve MSA next time
to MSA data collection uniform .
Instead of bimodal , we should collect the data uniformly .
Then also we saw part to part repeatability variation .
There is one reason it could be measurement issue .
Another reason could be process is not uniform .
We need to validate our thermal math to check our process uniformity .
Finally , I would like to mention that what is the impact
on my learning for this MSA analysis .
Now we know that several JMP platform or JMP application can help me to know
what is the signal variation from the noise variation ,
and then we can identify what figure of merit we can use
to justify our measurement method .
In our cases , we found P /T is the best method ,
best figure of merit to analyze it .
Then how misclassification risk can relate to the MSA component
as well as sample distribution that we learn .
Root cause analysis , we did several JMP application
that can help us to plan to improving MSA .
Since it is very helpful for particular program application ,
that 's why I would like to introduce
this data driven decision making for all the programs I involve in
to improve the project quality , cost , and time .
Finally , I would like to promote data driven decision using JMP
in our advanced technology group like CTO team ,
or other different projects .
This is my final slide .
I would like to mention my journey .
I started JMP learning beginning of the year ,
and that time we did A0 , A1 , A2 . This is my foundation .
Then after I work with MSA analysis and SPC .
I also got my certificate , JMP STIPS certificate May 2023 .
Now I am instructor at AMAT JMP instructor .
I 'm planning to in person presentation in October 2023 ,
and also I am working for my Black Belt on 2024 .
Thank you for listening .
