Entropy

[#1]

Hello,

I want to perform some basic descriptive statististics on time series data such as mean, median, kurtosis, etc.. but also the degree of variability over time which is described in the literature as "Entropy". I can not find this in JMPpro. Anybody a suggestion how to calculate this?

Lu

[#2]

Can you provide a literature citation or two for 'entropy'? Truth be told, I understand the definition of entropy from a physics point of view...but it's a term unfamiliar to me wrt to statistics, data visualization, or analysis. Maybe with a citation we could suggest a path forward in JMP?

[#3]

Hi,

I attached a recent publication in which entropy statistics was used on time series data; See feature engineering (page 4).

As I understand, entropy describe the change in variability over time.

Regards

Ludo

[#4]

Hi Lu,

Here is the simples description I could find on how to calculate Shannon Entropy (see file attached and source information):

1. Calculate the frequency of occurrence of each symbol / case = px (Distribution > Frequencies > Make Into Table)
2. In the resulting table ==> New column "B"=  px * Log2 (px)
3. In the resulting table ==> New column "H" = - Col Sum (:B)  or if you prefer to express in terms of bits :  -ROUND (Col Sum (:B),0)

Best,

TS

[#5]

Hi Lu,
I forgot to mention that this is only applicable of discrete symbols / cases because the entropy of a sequence of continuous data (i.e. real numbers) is essentially infinite.
Best,
TS

[#6]

Cool! .zip files do this. If you've ever compressed a .zip a second time, you might have noticed it does not help; the zipped data already has nearly maximum entropy (looks random) and can't be compressed much more and may get a little bigger to hold the header information.

[#7]

Hi,

Thanks for the support, but this is not what I needed. My data are indeed continuous.

Regards

Lu

[#8]

Hi Lu,
Sorry to hear that you were thinking of something different. For my education, could you point me to a reference or resource that defines or uses Entropy for continuous variables?
Best
TS

[#9]

Hi Thierry

See my reply to P Bartell and the attached file above.

regards

Lu

[#10]

Thanks.
I'll continue searching for possible reference on how Entropy is calculated for continuous variables.
Best,
TS