topic Re: Text analysis : need help to deal with phrases in Discussions

Text analysis : need help to deal with phrases

LogitTurtle576 — Sun, 11 Jun 2023 00:01:47 GMT

Hi,

I am working on a computer log file and when using the text explorer tool, in the phrase list, I have lots of row linked to the same phrase.

For example, in my log file, at different time, I have the text "Write error:status 1, facility 667,driverstatus 1304".

The problem is that in the Phrase list, It will show counts for "Write error", "error:status", "status 1,facility 667", "667, driverstatus", and so on...

Is there a way to show one row for the phrase "Write error:status 1, facility 667,driverstatus 1304" and not decomposing it ?

Otherwise, in the Term list, is there a way to exclude all numbers and keep only text ?

Thanks in advance for your advice

Re: Text analysis : need help to deal with phrases

Mark_Bailey — Tue, 30 May 2023 16:21:59 GMT

You can create a new regular expression using the Regex Editor in Text Explorer to recognize this pattern and return a new token in its place. Make sure that it appears ahead of the regex for Numbers.

You can change the Result token for the Numbers regular expression to empty with the Regex Editor.

Re: Text analysis : need help to deal with phrases

Byron_JMP — Tue, 30 May 2023 19:05:42 GMT

in the launch dialog, you can increase the number of words per phrase. The Default is 4.

Re: Text analysis : need help to deal with phrases

LogitTurtle576 — Wed, 31 May 2023 09:37:33 GMT

Hi Mark

Thanks for your answer.

I am not familiar at all with Regex Editor. How do I access it and how to use it ? Is there any related documentation ?

I don't get your point about changing the result token for the Numbers. What should I do exactly ?

Thanks

Re: Text analysis : need help to deal with phrases

Mark_Bailey — Wed, 31 May 2023 14:44:42 GMT

The Regex Editor in Text Explorer is documented.

Here is an example of a text analysis using the default set of regular expressions:

See the Result field? It uses the "\1" back reference to output the result to the token list. Clear this field so that numbers are not entered into the token stream.

Re: Text analysis : need help to deal with phrases

LogitTurtle576 — Thu, 01 Jun 2023 07:08:02 GMT

Many thanks Mark