%3CLINGO-SUB%20id%3D%22lingo-sub-809935%22%20slang%3D%22en-US%22%20mode%3D%22CREATE%22%3E%E8%AD%98%E5%88%A5%E5%8F%AF%E8%83%BD%E8%A1%A8%E6%98%8E%E8%B3%87%E6%96%99%E5%AE%8C%E6%95%B4%E6%80%A7%E5%95%8F%E9%A1%8C%E7%9A%84%E7%95%B0%E5%B8%B8%E6%A8%A1%E5%BC%8F%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-809935%22%20slang%3D%22en-US%22%20mode%3D%22CREATE%22%3E%0A%3CP%3E%3C%2FP%3E%3CDIV%20class%3D%22lia-vid-container%20video-embed-center%22%3E%3CDIV%20id%3D%22lia-vid-6252069804001w960h540r792%22%20class%3D%22lia-video-brightcove-player-container%22%3E%3CVIDEO-JS%20data-video-id%3D%226252069804001%22%20data-account%3D%226058004218001%22%20data-player%3D%22default%22%20data-embed%3D%22default%22%20class%3D%22vjs-fluid%22%20controls%3D%22%22%20data-application-id%3D%22%22%20style%3D%22width%3A%20100%25%3B%20height%3A%20100%25%3B%22%3E%3C%2FVIDEO-JS%3E%3C%2FDIV%3E%3CSCRIPT%20src%3D%22https%3A%2F%2Fplayers.brightcove.net%2F6058004218001%2Fdefault_default%2Findex.min.js%22%3E%3C%2FSCRIPT%3E%3CA%20class%3D%22video-embed-link%22%20href%3D%22https%3A%2F%2Fcommunity.jmp.com%2Ft5%2Fvideo%2Fgallerypage%2Fvideo-id%2F6252069804001%22%20target%3D%22_blank%22%3E%EF%BC%88%E5%9C%A8%E6%88%91%E7%9A%84%E5%BD%B1%E7%89%87%E4%B8%AD%E6%9F%A5%E7%9C%8B%EF%BC%89%3C%2FA%3E%3C%2FDIV%3E%3CP%3E%3C%2FP%3E%0A%3CP%3E%20%3C%2FP%3E%0A%3CP%3E%E8%AB%8B%E5%8F%83%E9%96%B1%E5%A6%82%E4%BD%95%EF%BC%9A%3C%2FP%3E%0A%3CUL%20class%3D%22list-bullet%22%3E%0A%3CLI%3E%E6%8E%A2%E7%B4%A2%E6%A8%A1%E5%BC%8F%3C%2FLI%3E%0A%3CLI%3E%E8%AD%98%E5%88%A5%E9%87%8D%E8%A4%87%E5%80%BC%3CUL%20class%3D%22list-bullet%22%3E%3CLI%3E%E6%89%BE%E5%87%BA%E6%9C%80%E9%87%8D%E8%A4%87%E7%9A%84%E5%80%BC%20-%20%E5%88%97%E4%B8%AD%E5%87%BA%E7%8F%BE%E6%9C%80%E9%A0%BB%E7%B9%81%E7%9A%84%E5%80%BC%3C%2FLI%3E%3CLI%3E%E5%B0%8B%E6%89%BE%E6%9C%80%E9%95%B7%E9%81%8B%E8%A1%8C%20-%20%E5%9C%A8%E5%88%97%E5%85%A7%E7%9A%84%E9%80%A3%E7%BA%8C%E8%A1%8C%E4%B8%AD%E9%87%8D%E8%A4%87%E7%9A%84%E5%80%BC%3C%2FLI%3E%3CLI%3E%E5%B0%8B%E6%89%BE%E6%9C%80%E9%95%B7%E7%9A%84%E9%87%8D%E8%A4%87%E5%BA%8F%E5%88%97%20-%20%E5%9C%A8%E5%88%97%E4%B8%AD%E9%87%8D%E8%A4%87%E7%9A%84%E5%80%BC%E5%BA%8F%E5%88%97%3C%2FLI%3E%3CLI%3E%E8%B7%A8%E5%88%97%E5%B0%8B%E6%89%BE%E9%87%8D%E8%A4%87%E9%A0%85%20-%20%E8%B7%A8%E5%A4%9A%E5%88%97%E5%87%BA%E7%8F%BE%E5%9C%A8%E5%90%8C%E4%B8%80%E8%A1%8C%E4%B8%AD%E7%9A%84%E5%80%BC%E5%BA%8F%E5%88%97%3C%2FLI%3E%3CLI%3E%E4%BD%BF%E7%94%A8%E7%A8%80%E6%9C%89%E5%BA%A6%E5%88%86%E6%95%B8%E4%BE%86%E8%A7%A3%E9%87%8B%E9%87%8D%E8%A4%87%E9%A0%85%3CUL%3E%3CLI%3E%E5%BE%9E%E6%A6%82%E5%BF%B5%E4%B8%8A%E8%AC%9B%EF%BC%8C%E6%A8%A1%E5%BC%8F%E8%88%87%E6%93%B2%E4%B8%80%E6%9E%9A%E5%85%AC%E5%B9%B3%E7%9A%84%E7%A1%AC%E5%B9%A3%E6%99%82%E9%80%A3%E7%BA%8C%E5%87%BA%E7%8F%BE%5B%E7%A8%80%E6%9C%89%E5%80%BC%5D%E6%AD%A3%E9%9D%A2%E7%9A%84%E5%8F%AF%E8%83%BD%E6%80%A7%E5%B7%AE%E4%B8%8D%E5%A4%9A%3C%2FLI%3E%3CLI%3E%E7%B5%B1%E8%A8%88%E4%B8%8A%EF%BC%8C-Log2(p)%EF%BC%9B%E5%85%B6%E4%B8%AD%20p%20%E6%98%AF%E5%81%87%E8%A8%AD%E5%80%BC%E9%9A%A8%E6%A9%9F%E6%8E%92%E5%BA%8F%E7%9A%84%E6%A8%A1%E5%BC%8F%E6%A9%9F%E7%8E%87%3C%2FLI%3E%3C%2FUL%3E%3C%2FLI%3E%3C%2FUL%3E%3C%2FLI%3E%0A%3CLI%3E%E8%AD%98%E5%88%A5%E7%95%B0%E5%B8%B8%E5%80%BC%3CUL%20class%3D%22list-bullet%22%3E%3CLI%3E%E6%89%BE%E5%88%B0%E5%96%AE%E5%85%83%E6%A0%BC%E5%85%A7%E7%9A%84%E6%A0%BC%E5%BC%8F%E5%8C%96%E5%AF%AC%E5%BA%A6%20-%20%E6%95%B4%E9%AB%94%E5%92%8C%E5%B0%8F%E6%95%B8%3C%2FLI%3E%3CLI%3E%E6%89%BE%E5%88%B0%E5%8F%AF%E7%96%91%E7%9A%84%E5%88%86%E6%95%B8%E9%95%B7%E5%BA%A6%3C%2FLI%3E%3CLI%3E%E6%89%BE%E5%88%B0%E9%81%8E%E6%96%BC%E7%B5%B1%E4%B8%80%E7%9A%84%E5%8F%AF%E7%96%91%E5%89%8D%E5%B0%8E%E6%95%B8%E5%AD%97%3CUL%3E%3CLI%3E%E6%A0%B9%E6%93%9A%E6%9C%AC%E7%A6%8F%E5%BE%B7%E5%AE%9A%E5%BE%8B%E6%AA%A2%E6%9F%A5%E5%89%8D%E5%B0%8E%E6%95%B8%E5%AD%97%E7%9A%84%E5%88%86%E4%BD%88%EF%BC%8C%E8%A9%B2%E5%AE%9A%E5%BE%8B%E6%8C%87%E5%87%BA%EF%BC%8C%E5%9C%A8%E8%A8%B1%E5%A4%9A%E8%87%AA%E7%84%B6%E5%87%BA%E7%8F%BE%E7%9A%84%E6%95%B8%E5%AD%97%E7%B5%84%E4%B8%AD%EF%BC%8C%E5%89%8D%E5%B0%8E%E6%95%B8%E5%AD%97%E7%9A%84%E5%88%86%E4%BD%88%E4%B8%A6%E4%B8%8D%E5%9D%87%E5%8B%BB%3C%2FLI%3E%3CLI%3E%3CP%3ELog10(%20(d%2B1)%20%2F%20d)%EF%BC%8C%E5%85%B6%E4%B8%AD%20d%20%E6%98%AF%E5%89%8D%E5%B0%8E%E6%95%B8%E5%AD%97%3CSPAN%3E%20%3C%2FSPAN%3E%3C%2FP%3E%3C%2FLI%3E%3C%2FUL%3E%3C%2FLI%3E%3C%2FUL%3E%3C%2FLI%3E%0A%3CLI%3E%E8%AD%98%E5%88%A5%E6%84%8F%E5%A4%96%E7%9A%84%E7%B7%9A%E6%80%A7%E9%97%9C%E4%BF%82%EF%BC%8C%E5%85%B6%E4%B8%AD%E5%9C%A8%E6%9F%90%E4%BA%9B%E9%80%A3%E7%BA%8C%E8%A1%8C%E7%B5%84%EF%BC%88%E9%A0%90%E8%A8%AD%E7%82%BA%2010%EF%BC%89%E4%B8%AD%EF%BC%8C%E4%B8%80%E5%88%97%E8%88%87%E5%8F%A6%E4%B8%80%E5%88%97%E5%85%B7%E6%9C%89%E7%B2%BE%E7%A2%BA%E7%9A%84%E7%B7%9A%E6%80%A7%E9%97%9C%E4%BF%82%3C%2FLI%3E%0A%3CLI%3E%E8%AD%98%E5%88%A5%E5%85%B7%E6%9C%89%E8%A6%8F%E6%A0%BC%E9%99%90%E5%B1%AC%E6%80%A7%E7%9A%84%E5%88%97%E7%9A%84%E8%A6%8F%E6%A0%BC%E9%99%90%E7%95%B0%E5%B8%B8%3CUL%20class%3D%22list-bullet%22%3E%3CLI%3E%E6%89%BE%E5%88%B0%E8%A6%8F%E6%A0%BC%E9%99%90%E5%88%B6%E5%8C%B9%E9%85%8D%EF%BC%8C%E5%85%B6%E4%B8%AD%E5%96%AE%E5%85%83%E6%A0%BC%E4%B8%AD%E7%9A%84%E9%99%90%E5%88%B6%E8%88%87%20LSL%20%E6%88%96%20USL%20%E5%AE%8C%E5%85%A8%E5%8C%B9%E9%85%8D%3C%2FLI%3E%3CLI%3E%E6%AF%94%E8%BC%83%E8%A6%8F%E6%A0%BC%E9%99%90%E5%88%B6%E5%88%86%E4%BD%88%E4%BB%A5%E5%B0%87%E4%B8%8D%E5%90%88%E8%A6%8F%E6%A0%BC%E5%80%BC%E8%88%87%E9%A0%90%E6%9C%9F%E4%B8%8D%E5%90%88%E8%A6%8F%E6%A0%BC%E5%80%BC%E9%80%B2%E8%A1%8C%E6%AF%94%E8%BC%83%3C%2FLI%3E%3C%2FUL%3E%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Explore%20Patterns.JPG%22%20style%3D%22width%3A%20607px%3B%22%3E%3Cspan%20class%3D%22lia-inline-image-display-wrapper%22%20image-alt%3D%22Explore%20Patterns.JPG%22%20style%3D%22width%3A%20607px%3B%22%3E%3Cimg%20src%3D%22https%3A%2F%2Fcommunity.jmp.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F32543i3621A9C90A4A6499%2Fimage-size%2Flarge%3Fv%3Dv2%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22Explore%20Patterns.JPG%22%20alt%3D%22Explore%20Patterns.JPG%22%20%2F%3E%3C%2Fspan%3E%3C%2FSPAN%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%20%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Benford's%20Law.JPG%22%20style%3D%22width%3A%20400px%3B%22%3E%3Cspan%20class%3D%22lia-inline-image-display-wrapper%22%20image-alt%3D%22Benford's%20Law.JPG%22%20style%3D%22width%3A%20400px%3B%22%3E%3Cimg%20src%3D%22https%3A%2F%2Fcommunity.jmp.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F32544i8430B62173302748%2Fimage-size%2Fmedium%3Fv%3Dv2%26amp%3Bpx%3D400%22%20role%3D%22button%22%20title%3D%22Benford's%20Law.JPG%22%20alt%3D%22Benford's%20Law.JPG%22%20%2F%3E%3C%2Fspan%3E%3C%2FSPAN%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%E8%B3%87%E6%BA%90%EF%BC%9A%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Fcreativecommons.org%2Flicenses%2Fby-sa%2F3.0%2Fdeed.en%22%20target%3D%22_self%22%20rel%3D%22nofollow%20noopener%20noreferrer%22%3EKurt%20Schwitter%3C%2FA%3E%E8%AB%96%E9%80%A3%E5%88%86%E6%95%B8%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-809935%22%20slang%3D%22en-US%22%20mode%3D%22CREATE%22%3E%3CLINGO-LABEL%3E%E5%9F%BA%E7%A4%8E%E8%B3%87%E6%96%99%E5%88%86%E6%9E%90%E8%88%87%E5%BB%BA%E6%A8%A1%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Identifying Unusual Patterns that Might Indicate Data Integrity Issues
Published on
11-07-2024
03:28 PM
by
| Updated on
11-08-2024
08:07 AM
See how to:
- Explore Patterns
- Identify duplicate values
- Find Most Duplicated Values - values that appear most frequently within column
- Find Longest Runs - values that repeats in consecutive rows within column
- Find Longest Duplicated Sequences- sequence of values that repeats within column
- Find Duplicates Across Columns- sequence of values that appears in the same rows across multiple columns
- Use Rarity Score to interpret duplications
- Conceptually a pattern is about as likely as getting [rarity value] heads in a row when flipping a fair coin
- Statistically, -Log2(p); where p is probability of pattern assuming random ordering of values
- Identify unusual values
- Locate Formatted Width within cells - both overall and decimals
- Locate suspicious Fraction Lengths
- Locate suspicious Leading Digits that are too uniform
- Identify unexpected linear relationships where, within some group of consecutive rows (default is 10), one column has an exact linear relationship with another column
- Identify specification limit anomalies for columns with spec limit properties
- Locate Spec Limit Matches where limits in cells exactly match LSL or USL
- Compare Spec Limits Distribution to compare out-of-spec values to expected out-of-spec values


Resources:
Start:
Wed, Jun 17, 2020 02:00 PM EDT
End:
Wed, Jun 17, 2020 03:00 PM EDT