Established a Systematic JMP 19 MSA Workflow and Scorecard Methodology (2026-US-PO-2960)

Hello, everyone. This is RuiZe Qi. I'm a graduate student from UVA, and also a employee in Applied Materials. Today, I would like to introduce a systematic MSA workflow and scorecard based on my recent project on the 3 Micron Aluminum R&D Metro System CIPs.

Before we kick into each module, I just would like to introduce the motivation for this project. This is to ensure our measurement system provide an accurate and reliable data for the manufacturing decisions, especially in the semiconductor for the automobiles.

From the quality perspective, we can only improve the process if we can measure it correctly. If the measurement system is not reliable, we may spend resources solving the problem that do not exist or missing the real problems that truly matter.

The purpose of the whole project was to build a confidence in the metrology system, improve the data integrity, and to ensure the engineering and production decisions are supported by trustworthy measurements.

The full presentation include three sections: Module A, from the define phase to the data collection plans, and Module B, the MSA performance assessment and root cause analysis combined, and Module E, the CIP plan and outcome, in which all the eight core steps highlighted here will be included.

Let us just first start by introducing the project chapters. Recently, the main challenge in the semiconductor field I work on is the excessive metrology variations observed in the 3 Micron Back-End-of-Line Aluminum etch process. This measurement variation will impact our decisions into the product qualities, making the process monitor less effective. Therefore, the goal for this project is to validate our measurement system, and improve the system if it is necessary, and finally achieve our CTQs, the measurement system below 20%.

To achieve the GRR target discussed in the previous slide, we first need to clarify what we measured in our system. In this slide, it tried to tell you the measurement bottom CD here from E1 to E4 is the measurement that is coming from our interest.

The measurement is performed by using a top-down CD-SEM with where the electron beam scan through across the feature and image is processed to calculate the dimensions. The output is performed in the bottom CD in nanometers and the measurement system consists of the VeritySEM platforms that the Applied Materials provide and its staging positioning capabilities. All these definitions establish the foundations of how the MSA evaluations in the following sections.

Now we know the measurement system has been defined. The next step is to establish how the repeatability and reproducibility will be evaluated. For the repeatability, the same features will be measured four times on the same die without touching or changing the samples.

For the reproducibility, the measurements are performed across four VeritySEM tools while controlling the key variables such as the measurement method, recipe, feature location, etc. By keeping those conditions constant, any observed variation can be attributed to the measurement system itself rather than the extra human errors.

After defining the GRR study method, the next step is to select the appropriate parts for the evaluations. As we can see from the left side, the plasma distribution interactions in our semiconductor field and the pumping effect create a wafer-level bottom CD full signal range.

As shown here, the selected part exhibits a near-uniform distribution with a kurtosis close to -1.2, which is the target for the MSA study. This try to ensure the sample site adequately correspond to the process variation and provide sufficient coverage across the specification range, also including the risk of misclassification near the specification limits.

With the part selected, the next step is to define the GRR study model. Since the CD-SEM measurement is non-destructive, so the same die can be measured by multiple VeritySEM tools. Therefore, we use the crossed GRR models where every selected die is measured on every SEM tool. This approach allows us to separate the part-to-part variation from tool-to-tool variation and can also evaluate any interactions between the specific part and SEM tools.

Before executing the full GRR study, we need to verify that repeated SEM measurements would not alter the feature being measured. To assess this risk, the paired t-test was performed by comparing the measurement taken before and after 48 hours queue time.

The normality assumptions were satisfied and the t-test show that no statistically significant difference between the two measurement sets. In addition, the correlation remained very high at approximately 0.97. Therefore, the part degradation risk is considered low, and repeated measurement can be used confidently in the GRR study.

After confirming the part degradation is not a concern, we are ready to finalize the MSA study design. The response variable is the bottom CD, while the two study factors are the VeritySEM tools and selected dies. The four SEM tools and five representative dies were included in the full crossed design. Each die was measured repeatedly on every SEM tool, allowing us to quantify repeatability and reproducibility separately.

To minimize the tool downtime and improve the execution efficiency, a fast repeat strategy was used. Although this approach may slightly underestimate long-term variations such as drifting effects, it is still the best and appropriate ways for us now to evaluate intrinsic performance of the measurement system.

Although the operators were not included as a formal factor because all these measurements are automated by the machines, we still want to verify if the operator has some operating differences during the automation operation that could introduce additional measurement variations.

To evaluate the risk, a paired t-test was performed again using the measurement collected by two operators on the same samples, and the result shows no statistically significant differences between operators, and the measurements were highly correlated. Therefore, the operator-to-operator variation is considered negligible compared to the overall measurement system variation and was excluded from the final GRR model.

After finalizing the study design, the diagnostic evaluation was performed to verify whether the selected data collection strategy could reliably estimate GRR components. Here we compare the fast repeat with completely randomized method using the simulation-based metrics.

The results show that both approaches has a low relative bias and mean absolute relative error, the MARE, below one for the major variance components. This indicates acceptable estimation performance.

With the MSA design validated, we proceed to the data collection plans. The study included four VeritySEM tools and five representative dies selected from the wafer. Each die was measured four times on every SEM tool, resulting in a full crossed dataset. The run order was randomized to minimize the bias and ensure the collected data accurately represented the measurement system variation.

Before analyzing the GRR results, we first need to define the acceptance criteria. Since our selected part intentionally covers the full specification range and include the measurement near the specification limit, the primary metric chosen for this study is the P/T, or the precision to tolerance.

In addition, the misclassification analysis is also included because the decision risk becomes critical near the specification limits. Based on the standard MSA guidelines, our P/T value below 20%, the CTQ as defined before, to be considered acceptable and align with the project objective.

Before analyzing the GRR results, we first need to define acceptance criteria, since our selected part intentionally covers the full specification range and including the measurement near the specification limits. The primary metric chosen is P/T, and also the misclassification analysis is included because decision risk become critical near the specification limits. These allow us, the GRR study, not only evaluate the measurement precision, but also its impact on the manufacturing decisions.

This is the first chapter of the scorecard assessment. Basically, all the scorecards are above seven scores, which means we can proceed with the next module, the GRR's data analysis and the root cause analysis.

Before analyzing the repeatability issues here, we firstly identify the major factors that could potentially introduce the measurement variations. The first category is the image quality. As we show here, this included the contamination build up, the feature damage, or the resolution degradation during the SEM measurements.

The second category is the image signal processing, where the edge detection sensitivity could lead to the measurement shift if we have the image quality issues. The third category is more to the tool's mechanism side. The stage and wafer positioning stability, including the vibration effects or the Echuck behavior during the measurement. All these factors were defined as primary repeatability risk candidates and will be evaluated further in the subsequent analysis.

The same for the reproducibility, which evaluate the variation between different SEM tools. From this study, we identified the two major risk categories. The first is the tool-to-tool process consistencies, including factors such as the pixel calibration, autofocus performance, charging behavior, etc.

The second category is the algorithm stability itself, including the potential risk caused by the beam current, dwell time, scanning speed settings, etc. All these factors, again, defined as the potential sources for reproducibility variations and later on will be systematically discussed.

Here, the JMP gives you a platform to load your different sessions, the part, the metrological, and also the measurement of interest into this screen, so we can start the data analysis. This slide is the detailed analysis of our repeatabilities. Within the potential repeatability and reproducibility identify, we load the collected GRR datasets into JMP, and this creates the crossed variability models.

This shows three potential repeatability concerns labeled here in R1, R2, and R3, with R1 showing the largest variations. However, this is assuming our data is following the normal distributions, but we fail these normal assumptions with both Shapiro-Wilk and Anderson-Darling below the significance level.

Here on the right side, we perform a non-parametric variance test, which is ANOVM, that was used for the conservative perspective of validations. From both left and right, the R1 is considered consistently high as the most significant repeatability issue, followed by the R2 and R3. Based on these results, R1 was prioritized for the further root cause analysis for investigations.

Earlier in the design MSA section, we identified several potential repeatability risk factors, including the sidewall damage, edge contamination, image resolution, and so on and so forth. After identify the R1 as a primary repeatability concern, we correlate those risk candidates against repeatability performance.

This multivariate analysis on the right side shows that the edge pollution has the strongest main effect on the repeatability variance. In addition, several interaction effects, such as the sidewall damage combined with the edge contamination, also appear significant. These results helped us narrow down the troubleshooting space and guided the detailed root cause investigations for the further slides here.

By using the same approaches for the repeatability, we can also try to narrow down and find what is the most significant contributor to our reproducibility issues. We did find that the pixel calibration is the strongest main effect, while interactions involving the pixel calibration and the FOV charging effects may also contribute. This will help us do a better root cause investigation in the later slides also.

After narrowing down the investigation to the key repeatability outliers, we will review the corresponding SEM images, the R1 through R3. As we can see here, the R1 and R3 are associated with the edge contamination, where the R2 is more related with the feature of sidewall damage here. These image quality issues can affect edge detection consistency and are likely contributor to the repeatability variation observed in the GRR study.

After identifying the edge contamination as one of the primary repeatability issue, we investigate its impact on edge detection algorithms. The key observation is that the left side edge of the feature E1 remains relatively clean and stable, resulting in a stable signal profile and consistent E1 edge detection.

However, the contamination on the right side introduced additional noise pattern. Depending on the contamination levels, the algorithm may detect the E1, E4, either near the true feature of the edge or near the noise staircase-like signal. This will cause the detected edge location to shift from measurement to measurement. As a result, the right edge exhibits a much larger detection variability, which will directly translate into this repeatability big noise in the measurement bottom CDs here.

Next, we evaluate the impact from the feature damage on the repeatability performance. Unlike the contamination issue we discussed in the previous slide, the feature damage physically alters the sidewall profiles.

When localized damage create a trench-like signal drop, the edge detection algorithm may incorrectly identify this damaged region as the actual feature boundary. For undamaged structures, the algorithm can consistently detect the edge near its true location. However, for the damaged structure, both the left and right edges can exhibit multiple invalid signal patterns. This will increase uncertainty in both E1 and E4 detection positions and thus resulting in a large variation in the calculated BTMCD values.

After understanding how edge contamination feature damage impact edge detection, we reviewed all these GRR samples by both Die locations and the SEM tools to identify where these failure modes occur most frequently. The variance analysis highlighted the Die 02, the Die 21, and the Metro3 as the largest contributor to the measurement variation.

When we inspect the corresponding SEM image matrix here, we find the strongest correlation with the previously identified failure modes. The Metro3 here show the multiple cases of feature damage and edge contamination across different dies, while the Die 21 consistently exhibit edge contamination across different dies. This observation provides additional evidence that the image quality-related defects, including the contamination edges and damaged features, are the primary drivers of the repeatability issue.

After completing the repeatability analysis, we next investigate the reproducibility, which will focus on the variation between the different four SEM tools. Since the Die 17, as we highlight here, has a low within-part variation, any significant difference is likely to come from the measurement tools themselves rather than the wafer features. The results show that the Metro2 is statistically different from the Metro1, 3, and 4. If we see the overlays, they're traces.

Then, if we try to be more statistically accurate, we do the matched-pair comparisons here, and matched comparisons show that the Metro2 is the only one that has the differences with rest of the Metros 1, 3, and 4. Observed both from the graphical builders, the difference is from the edge signal patterns and also from the match pairs data.

After identifying the Metro2 as the source of the tool-to-tool bias, we try to investigate the pixel calibration accuracy across all four SEM tools. First, we compare the edge profiles and the pixel-based measurement on the same features, after the edge detection location appear similar and underlying data are discrete, and do not follow the normal distributions. Therefore, the conventional statistical assumptions are not fully applicable.

To address this, we can use the Monte Carlo-based estimation to evaluate the true calibration differences between the tools. This result shows a strong correlation between the estimated BTMCD differences versus your actual measurement differences. More importantly, the comparison matrix reveals a clear deviation, if we see from here, from the ideal 45-degree identity line, that indicates the tools are not perfectly matched in the pixel calibration.

Among all the tool combinations, the largest calibration mismatch was observed here between the Metro2 and Metro4. This aligns with the reproducibility analysis and confirms that the pixel calibration variation is a dominant contributor to the observed metro-to-metro measurement bias. Therefore, the pixel calibration matching becomes the highest priority to improve the action for the reproducibility reductions.

After completing the repeatability and reproducibility root cause analysis, we summarize the results using ANOVA to understand the relative contribution of each source of variations. The ANOVA shows that both the metro effect and Die location effect are highly significant, with both p-values are below the significance level. This will tell us that selected dies successfully captured the real process variation, while the measurement differences can also exist between the SEM tools.

However, as we also witnessed, the metro-to-die interaction is not significant, with the p-value around 0.33. This means the measurement differences are consistent across the dies and are not dependent on the specific wafer location interactions.

From the business perspective, this is actually a positive sign because it indicates that observed measurement bias is systematic and primarily driven by the metro effect rather than a complicated tool-by-part interaction. Therefore, the improvement effort should focus on the metro reproducibility, especially the pixel calibration matching, rather than spending resources on the metro-to-die interaction studies.

Before running the final GRR analysis, the study parameters were configured using standard MSA settings, the 6 sigma multiplier and the 0.05 significance level selected, and best analysis options were used for the variance estimations. The tolerance for the BTMCD were entered based on the production specifications, and these settings provide a foundation for calculating the final GRR performance metric.

These, as we previously discussed, we use the P/T because we have the lower spec limit and upper spec limit. The other P/B and P/TV or P/M because of the limitations are not preferred to be selected here.

Now, let's review the overall GRR performance. Unfortunately, the current measurement system does not meet the acceptance criteria. The P/T ratio is approximately 146%, which is well above 30% threshold. NDC value is only just 1, indicating that the measurement system cannot reliably distinguish different part categories. If we're looking at the variance breakdown, the repeatability and reproducibility contribute almost equally, around 103% of the tolerance. However, within the reproducibility, the dominant factor is metro-to-metro variance, while the metro-to-die interaction is relatively small. It's only about 20%.

The result is consistent with our root cause analysis. Repeatability is mainly driven by the edge contamination and feature damage, while the reproducibility is primarily driven by the pixel calibration mismatches between samples.

After reviewing the GRR results, we want to determine whether the issue was caused by the insufficient measurement resolution or by excessive measurement variation. The analysis shows that the measurement resolution is actually adequate. Even after reducing one decimal place, the system will still provide strong, distinct measurement levels and satisfy effective resolution requirements. However, the NDC value remains at 1 because measurement variation is much larger than the part-to-part variation. In other words, the problem is not the resolution capability itself, but the poor GRR performance. Therefore, improving repeatability and reproducibility is a priority rather than increasing the measurement resolution.

This plot here provide a visual summary of the GRR results. The mean plot shows that die-to-die variation is similar to the magnitude of part metro-to-metro variation, which explain the low NDC value observed earlier. The standard deviation plot further confirmed that the Metro3 and the NDC and the Die 21 exhibit the highest measurement variation. Overall, these plots are consistent with our root cause analysis executed before.

Beyond the GRR metrics itself, the most important question is how measurement variation may impact our true manufacturing decisions. Based on this classification analysis, about 20% of good wafers could be incorrectly rejected due to the measurement error. This directly will reduce the yield and create unnecessary manufacturing cost. More importantly, approximately 39% of bad wafers may be incorrectly accepted, and this will create a risk that an out-of-spec product could escape detection and reach downstream operations or even to the customers, impacting the product reliability and customer confidence.

Basically, this is the evaluation of the Module Bs, and all these scorecards are still good enough for us to proceed with the final Module E, the CIP plans and outcomes here.

Based on the root cause of the investigations before, we identified the two major contributors to the repeatability variations: feature damage and edge background contaminations. Therefore, the improvement plan is to divide it into short-term contamination actions and long-term corrective actions.

For the short term, the observation is to try to make the edge detection algorithm more robust. By trying to introduce the signal smoothing and moving average filtering, the local damage features can be flattened, reducing the likelihood that algorithm mistakenly identify the damage region as the measurement edges. In addition, increasing the edge intensity threshold prevents the false edge detection caused by the background contamination, especially around E4 edge locations. Together, these actions can improve the measurement consistency without requiring any process changes.

For the long term, the focus shift from the measurement algorithm, shifting to the process itself. Since the feature damage originates from the process induced damage, we try to quantify the image quality into quantifiable responses using the Fast Fourier Transform analysis. The higher frequency energy corresponds to a much more severe feature damage, which allows us to translate the image quality into a statistical number in the models. By using this response, we can perform a DSD DOE CIP to connect the feature damage back to the key process tuning parameters.

The initial analysis identified several high impact process factors, with the X2 and X4 currently showing the strongest correlation to the sidewall roughnesses. The process optimization is ongoing, and by doing such a DOE, we did see a very significant improvement before and after the CIP in terms of the sidewall damage improvements.

In a nutshell, the short-term action improves the measurement robustness, while the long-term action eliminates the physical sources of repeatability variations. This is all about the repeatability improvement plans that has been discussed.

Now we switch on to the reproducibility solutions. After identifying the metro-to-metro bias as a dominant reproducibility issue, we developed the improvement plan targeting the two most likely mechanism, the pixel calibration drift and signal non-overlap.

First for the pixel calibration, the objective is to ensure that every SEM tool converts the pixel into the physical dimensions consistently. Even a small pixel size drift can create a systematic measurement offset that becomes amplified through the measurement feature-wise. Therefore, we established a standard calibration workflow on the right side and recommend a tighter metro matching target than the current SOP requirement. The long-term goal is using this common golden wafer and routinely cross-metro verification to maintain calibration consistency across all tools.

The second improvement area is the signal non-overlapping issue. Our investigation shows that Metro2 exhibits a shifted signal profile, which can occur when autofocus and the beam current are not fully optimized. A de-focused beam may broaden the edge transition region and cause the edge detection drift, and ultimately a tool-to-tool measurement bias. To address this, we recommend strengthening the autofocus calibration controls and verifying focus performance across all magnifications.

Finally, we also identify the PMT gain matching as another important long-term control. Although the SEM images may appear visually similar, the differences in detector gain can shift a histogram response and create signal offsets between the tools. By standardizing the PMT gain calibration and introducing cross-metro signal matchings, we can further reduce metro-to-metro variation and improve reproducibility performance. Overall, the reproducibility roadmap focus on the above three key areas: the pixel calibration, autofocus, and PMT matching. Together, these three actions directly target the root cause of metro-to-metro bias identified in this study.

After defined the repeatability and reproducibility improvement plans, we observe the expected impact on the overall GRR performance. On the left is the before CIP, and on the right is the after CIP. By reducing the metro bias through the pixel calibration and signal matchings, and by reducing within-part variation through the repeatability improvement, the measurement distribution becomes much more tighter grouped. The chart on the right represent the projected state after implementing the proposed improvement, and by reducing metro bias through the pixel calibration, we can see that the metro-to-metro variation noise is significantly drifted. Also, what we saw the repeatability within the same part significantly reduced.

To summarize the improvement results, the proposed action significantly improved overall measurement system capabilities. The P/T was reduced approximately 75%, from 146 down to around 36%, while the NDC tries to increase from 1 to 7. This means the metrology system is now much better at distinguishing real part-to-part differences rather than the measurement noise.

Now, looking at the individual components here, the repeatability improved by more than 80% and reproducibility improved by nearly 70%. Most importantly, the major metro-to-metro bias was successfully reduced, confirming the proposed calibration and matching strategy is effective.

From a manufacturing perspective, the false rejection rate drops from 11% to roughly only 2%, which translate directly into the yield recovery and the cost reduction. At the same time, the false acceptance rate decreased from 40% to nearly 13% here, subsequently reducing the customer escape and reliability risk. While the further work is still needed to reduce the remaining part-to-part interactions and achieve the long-term target, the overall results demonstrate that a proposed improvement plan can significantly increase the measurement confidence and supported better production decisions.

The last few slides, it's the JMP. It can provide the Workflow Builder package for all the previous actions to be recorded and to be present in the meetings here.

Here are the dashboard for some of the key elements that has been illustrated previously in the slide here. This slide is trying to tell how the workflow implementations and also some challenges internally and externally. As we saw, the whole story of this project requires a lot of attention and the technical skills. It actually asked for a relatively higher standard for the stakeholders to perform such a workflow in the local field here.

Here is the final Module E self-assessment, including the GRR improvement documentation lines, SOPs, the assessment from the Module A and Module B, and also the implementation difficulties to all the stakeholders here.

Basically, that's all about this project. We covered these eight core elements from the very start to the end. Hopefully, this workflow can give you some ideas if you are working in the manufacturing industries in the qualities or still struggling with metrology noise. Hopefully, this can help you build up a foundational mindset for how to find the problem and how to solve the problem. Thank you, for all.

Bill, I think I'm done.