Creating Structured Tables From Unstructured Text Logs With Text Explorer
Sep 7, 2017 1:15 PM
Shannon Conners, PhD, JMP Director of Research and Development, SAS
JMP 13 introduced the Text Explorer platform to find important words and phrases within large document collections and topics for further analysis. If you want to create structured data tables from unstructured text, you should not overlook Text Explorer’s Customize Regex and Save to Column options. I’ll show how I use these features to extract key details from text logs output by JMP build and test projects into new table variables. My approach can be generalized to retrieve and process text logs from other web-based or file-based systems. After creating structured data tables with Text Explorer, I use various JMP tools to finalize their format. Text to Columns parses the delimited text produced by the Save to Column feature, splitting a single column into multiple columns on white space or custom character delimiters. I use JMP Query Builder (introduced in JMP 13) to interactively subset, reformat and rename columns, and save a script that replicates the process. I’ll give a sneak peek at how my Jenkins data processing workflow has recently improved thanks to several new features in JMP 14, including JSL functions handling HTTP requests to the Jenkins web API and embedded clickable web links in JMP data tables.