BookmarkSubscribe
Choose Language Hide Translation Bar
mward
Community Trekker

Word density

Hey there,

Does anyone have a way to create a word density distribution for a column of textual data in JMP?  Basically I'd like to create a summary table that has every word in the data file and a column showing how many times it appears.

Thanks!

Matt

0 Kudos
3 REPLIES 3
antony_allianz
Community Member

Re: Word density

use

proc means data =dsname n;

class varnm;

run;

give the varnm of the word variable name

otherwise use first.varnm and last.varnm


0 Kudos
pmroz
Super User

Re: Word density

Here's a JMP solution.  This is the JMP discussion forum after all.

dt = New Table( "Word Table",

    Add Rows( 9 ),

    New Column( "Column 1",

        Character,

        Nominal,

        Set Values(

            {"The quick red fox jumped over the brown log",

            "The quick red fox jumped over the brown log",

            "The quick red fox jumped over the brown log",

            "The quick red fox jumped over the brown log",

            "Apples peaches pumpkin pie.  If you're not ready holler I.",

            "Apples peaches pumpkin pie.  If you're not ready holler I.",

            "Apples peaches pumpkin pie.  If you're not ready holler I.",

            "Apples peaches pumpkin pie.  If you're not ready holler I.",

            "Apples peaches pumpkin pie.  If you're not ready holler I."}

        )

    )

);

dtcol = column(dt, "Column 1");

first_pass = dtcol << get values;

// Convert the column to one big string

first_string = concat items(first_pass, " ");

// Remove periods, change two spaces to one

first_string = substitute(substitute(first_string, ".", " "), "  ", " ");

// Parse all values out of this big string, delimited by a space

all_list = words(first_string, " ");

// Add these values to a new table

wordsdt = New Table("Parsed Words",

        New Column("listvals", Character, Nominal, Set Values(all_list)));

// Run tabulate to get a frequency count

words_tab = wordsdt << Tabulate(

    Show Control Panel( 0 ),

    Add Table( Row Table( Grouping Columns( :listvals ) ) )

);

Re: Word density

This script on the File Exchange might help you: Word Counts to Columns.

-Michael

Michael Crotty
Sr Statistical Writer
JMP Development