Hello everyone.
My name is Harry Dong.
I'm the director of the Optical Process Engineering Group at Magic Leap.
I'm so excited to be here to present my topic:
Factory Yield Ramp- up Approach
through P rocess Performance Metrics Guided Improvement Activities.
Today I'm going to cover Factory Process Performance Overview,
the Process Screening Tool and the Statistical Process Control
using JMP Script to automate the process capability analysis and the conclusions.
People have been talking about process performance,
S uch as capability stability, but what are they?
Basically, process capability is a measure
of the ability of the process to meet the specifications.
While, the process stability refers to the consistency of KPIs over time.
It's very important to realize there is no inherent relationship
between process stability and process capability.
Thus it is both extremely important aspects of any manufacturing process.
As you can see from the bottom- left picture.
It's showing,
process can be both capable and stable meantime,
this is a perfect world.
Basically your process is super tight
against your lower spec limit,
upper spec limit.
But on the other side.
When you started seeing instability,
when your variation is small,
you'll still be able to meet the process specifications.
But over time,
you will see a lot of variations.
The other scenarios can be your process is super stable.
Over time you don't see a lot of up and downs,
But because it could be like a small
a three Sigma, a lower spec limit, upper spec limit,
you have large variation,
then you'll not be able to meet your specification
against a high yield target, high process capability target.
The worst case can be your process is not capable,
which means you have large variation
and over time you also see a lot of stability issues.
It's everywhere,
Across the whole industry,
monitoring factory process using the process performance plot
becomes very useful.
It's including capability, stability
in the combined metrics.
As you can see,
this is a JMP- generated process performance plot.
On the x- axis you see the stability index
and the y- axis is indicating the capability overall which is a Ppk.
Eventually this is a four quadrant plot.
We want to push everything low stability
and high process capability which is in the green zone.
Often you'll see some process not capable which is under this line
or not stable, which is across the vertical line showing in this graph.
I'm going to talk about
the Process Screening function,
JMP provided.
This is a very powerful tool,
if you are talking about quickly identify unstable process
or some incapable process, meanwhile.
Basically this is a JMP- generated report,
as you can see,
I have 24 processes listed in this report.
Very quickly you can see stability index is showing up.
How do you define the process stability?
It's calculated using the Within Sigma and Overall Sigma,
This is the ratio using the Overall Sigma
divided by Within Sigma,
What do they mean?
Overall Sigma usually treated as a longterm process variation
and the Within Sigma treated as a shortened process variation.
So JMP has certain rules that you can refer to
for this calculation.
But basically you can define color codes.
How do you see your process stable?
You can use 1, 1.3, whatever the number you want,
to color code that.
For me, as you can see,
I color coded process greater than 1.7
as a process not being stable.
Yellow as a process is kind of marginal,
And the green zone meaning the process is super stable.
This is telling us some process can be stable but not capable,
Because you can see the Ppk Cpk are kind of low,
But on the other side, you can see a bunch of process,
they were not stable, but some of them are very capable,
Again, this is aligned with what I went through earlier.
Basically, utilizing the Process Screening tool,
you can quickly identify the unstable processes
as I marked in this graph.
Meanwhile, as you can see in this Process Screening tool,
you can also see the control chart alarms
based on the samples that the raw data you put in the report.
We finally deploy the control chart,
called statistical process control chart,
be able to monitor and improve process capability.
Stability is very important,
This is a JMP- generated control chart,
You can use Windows Scheduler or JMP Live
to automate those charts to be able to pull the data real time.
Meanwhile, you can also use certain web API
to generate the control chart.
JMP is very useful in this case.
It can send out email notification to the group,
to individual engineers.
Meanwhile,
if you can connect us with your internal system ,
I think that's going to give you additional power
to be able to communicate with your process tool,
be able to pause the tool,
put it on hold for the engineers to react to the variations
either process shift or out of control data points.
Process Capability Analysis is a standard,
Basically a few rules that we need to follow,
First of all, you want to make sure
your data set is following the normal distribution.
This can be done using JMP tool,
I won't go through.
But on the other side we realize Cpk calculation changes dramatically
due to the outliers, especially when sample size is small.
The outliers can be driven by special causes,
excursions during the process.
So this can add bias to Process Capability Analysis.
Sometimes if you do see some outliers,
you can drag your Cpk down,
but it's not representative to your standard process variation.
Apply outlier removal method to remove outliers.
We'll help you get rid of those noises
to better understand your true process capability.
Within the JMP,
they offer different methodology
to remove outliers.
I won't go through each of them, but they're all very powerful.
You can read through the instructions which method is the best in your case.
Basically, they can be found under Analyze/S creening /E xplore Outliers.
Basically, I'm going to show the basic method we are using
to exclude the extreme values for our process,
which is a quantile range outlier removal.
Basically this tool we found is very useful,
Because when you try to pull the data through the database,
often you'll find some outliers.
Some data are very extreme,
you know they're outliers, some are not as obvious,
So basically this quantile range outlier method
offer you flexibility.
Basically, as you can see, this is the distribution of our process.
We have upper spec limit, target and you also have the mean.
And follow this Box Plot,
you can find the 10th percentile value,
or the 90th percentile value.
Basically the inter-percentile range
is calculated using 90th percentile minus 10th percentile value
and this is defined as your inter-percentile range.
The lower threshold value is calculated,
using the 10th percentile value
minus three times the calculated inter-percentile range.
So this becomes your low threshold for each individual processes.
On the other side the high threshold
is defined by 90th percentile plus three times inter-percentile range.
This becomes your high threshold.
Basically all the extreme value outside of this range
will be treated as outliers and they can be colored.
They can be marked as missing .
They can be excluded from your data analysis.
As you can see, JMP did provide the flexibility.
How do you define the inter-percentile range?
You can do 0.1, you can do other values as needed.
And also the Q value,
which is this value I highlighted here,
can be changed as well.
It really depending on how much noise
you want to get rid of from your data analysis.
A s you can see,
This is an example that showing some value is being colored
and also being changed to missing value,
so they will be excluded from the data analysis.
This is a quick demonstration to show you how the outlier
is going to impact your Cpk calculation.
As you can see, Cpk value remains equal or better
post outlier removal.
It really depends on your sample size.
Sometimes if your sample size is small, the change can be more dramatic.
But in my case, I believe my sample size is quite large.
This is why they're not showing very big differences.
Yeah, the quantile range o utliers
parameters can be tuned if necessary as I mentioned earlier.
Other things I want to highlight in this page,
it's the process capability box plot.
We figure out this is very useful tool
because you can be monitoring many process parameters.
So be able to put them together
to visualize how tight they are, which direction they're shifting
and how much variation is being counted
to calculate our process capability.
Process stability is very useful,
as you can see.
They use a standardize s pec limit to be able to combine everything together.
It's super useful for data visualization.
I want to quickly show you,
before and after we automate the process performance analysis.
At the beginning team are not using
the JMP scripting to automate this process,
as you can see.
We have to collect the quality data by individual process owners per module,
so data, sometimes, becomes not standard.
They use different formats
and then they often do manual outlier removal
and then they have to grab all the data together
to be able to merge them.
It's also a manual process, very tedious.
After that they have to do the manual process performance
because the variation they see through different people,
different format and then generate report.
They have to eventually repeat everything they did
on like weekly basis, monthly basis or per PEQ build.
But on the other side for the automated process flow,
basically the quality data can be queried all in one step.
All the raw data, all the specification data
can be put from the database
using the SQL JMP scripting,
Of course we apply standard outlier removal methodology across all the data
and then all the sorting, split, spec assignment,
all the different visualization
can be done automatically.
Standard report will be generated.
Then, when we are talking about over time
or per PEQ summary ,
you can simply modify your SQL query filter to update the data
and on top of that, you have basically all the raw data.
You can apply local data filter.
You can add additional functions to make your filter data analysis easy.
Basically, we figure out manual process
is very time consuming,
the feedback is slow and it's not very efficient regarding
the yield improvement.
On the other side,
we figure out the automated scripting process using JMP.
Anybody can perform this complicated process performance analysis in minutes.
Some highlights I want to share for the process capability analysis.
As you can see,
I didn't include the JMP query portion.
But I basically put different process name,
different test label,
all the raw data into the JMP table
and after that we figure out to split this d ata table
because table comes in everything combined
process, part ID, process time, test label.
So we have to eventually split the data table
to be able to perform this process capability analysis,
the box plots or the Cpk Ppk analysis.
So very useful function for JMP is after we split, group them.
So we have some missing value because we sort the data by data time.
Some process leave blank but JMP is smart enough
not counting those missing value
into the capability analysis
which is very useful,
So I do want to mention that.
Please don't forget to sort your data by date time
because that's super important
because sometimes the Cpk Ppk calculation
is really depending on the process sequencing.
If you are not sorting the data by date time,
then the results can be biased,
So we made some mistake earlier, figure out this is useful tip.
The other one is automated outlier removal
using quantile range outlier.
As you can see, b asically, very simple process.
You get all the column names
using this scripting.
Condition can be numeric, continuous
and then you start at launching
the quantile range outlier platform
basically assign them into the report and then go through the report.
Use this for loop function
to be able to exclude all the outliers identified,
This can be repeated as needed .
Then you can also launch the process capability analysis.
If you don't know how to do it, you can basically manually do it
and then grab the code from the log,
This is a new function JMP provided which we figure out is super helpful.
Something I want to highlight here is the spec limit assignment,
This is a super powerful, very useful.
Basically you can assign specification for multiple process variables,
using another data table
that's generated using t he SQL query as well.
Conclusions.
Analyzing process performance data
using JMP is super critical to drive the yield improvement
in modern factories,
especially with many process and methodology stacks
to ensure healthy production lines and deliver high quality products.
Analysis of performance data including long term, short term process capability,
stability and statistical process control
is particularly useful when monitoring
hundreds of process KPIs.
During the production ramp up phase.
Identifying the processes of most concern is highly challenging
and using JMP scripting and quality data analysis platforms ,
our Eyepieces factory implemented an automatic processor
which can pull, analyze, visualize, correlate, predict and verify
factory yield improvement based on a variety of performance matrix.
Magic Leap's Eyepiece Factory,
we demonstrated greater than 90% RTY, which includes hundreds of process KPIs.
Eventually, you have to multiply them together
to get this RTY number, no throughput yield.
We were able to demonstrate greater than 90%e RTY
six months ahead of our next generation product launch,
driven by continuous process improvement activities,
guided by automated process performance analysis,
using JMP scripting, and quality platform tools.
And thank you for your time.