Subscribe Bookmark RSS Feed

Word density

mward

Community Trekker

Joined:

Nov 1, 2012

Hey there,

Does anyone have a way to create a word density distribution for a column of textual data in JMP?  Basically I'd like to create a summary table that has every word in the data file and a column showing how many times it appears.

Thanks!

Matt

3 REPLIES
antony_allianz

Community Member

Joined:

Jul 23, 2012

use

proc means data =dsname n;

class varnm;

run;

give the varnm of the word variable name

otherwise use first.varnm and last.varnm


pmroz

Super User

Joined:

Jun 23, 2011

Here's a JMP solution.  This is the JMP discussion forum after all.

dt = New Table( "Word Table",

    Add Rows( 9 ),

    New Column( "Column 1",

        Character,

        Nominal,

        Set Values(

            {"The quick red fox jumped over the brown log",

            "The quick red fox jumped over the brown log",

            "The quick red fox jumped over the brown log",

            "The quick red fox jumped over the brown log",

            "Apples peaches pumpkin pie.  If you're not ready holler I.",

            "Apples peaches pumpkin pie.  If you're not ready holler I.",

            "Apples peaches pumpkin pie.  If you're not ready holler I.",

            "Apples peaches pumpkin pie.  If you're not ready holler I.",

            "Apples peaches pumpkin pie.  If you're not ready holler I."}

        )

    )

);

dtcol = column(dt, "Column 1");

first_pass = dtcol << get values;

// Convert the column to one big string

first_string = concat items(first_pass, " ");

// Remove periods, change two spaces to one

first_string = substitute(substitute(first_string, ".", " "), "  ", " ");

// Parse all values out of this big string, delimited by a space

all_list = words(first_string, " ");

// Add these values to a new table

wordsdt = New Table("Parsed Words",

        New Column("listvals", Character, Nominal, Set Values(all_list)));

// Run tabulate to get a frequency count

words_tab = wordsdt << Tabulate(

    Show Control Panel( 0 ),

    Add Table( Row Table( Grouping Columns( :listvals ) ) )

);

michael_jmp

Staff

Joined:

Jun 23, 2011

This script on the File Exchange might help you: Word Counts to Columns.

-Michael

Michael Crotty
Sr Statistical Writer
JMP Development