How to perform FFT on time series data

[#1]

Hello All,

I have a few questions for the JMP community. 

1a. Is it possible to batch process multiple files(.csv's) in JMP? If so how would I go about this?

1b. Is it possible to perform FFT on time series data? If so are there other frequency domain analyses than can be performed in JMP?

If any of this information is covered some place else I'd appreciate it if you could point me to its location. Thank you for your time.

[#2]

Hi @Damian ,

  I would imagine you can batch process/load CSV files with JMP. At the very least, I know you can script JMP so that it loops through all the CSV files in your directory and builds multiple or single data tables to analyze within JMP. I do this regularly when I access our quality data and need to merge data from different sources. Depending on connection speed and file size, it can be slow. I'm not the best programmer, so I'm sure there are more efficient and elegant ways than a simple loop.

  In the Time Series platform, there is a pretty basic Spectral Density feature that might be able to handle what you're after, depending on the complexity. I did find this post from @JerryFish , which was presented at the JMP Discover Summit in 2014, that is an add-in to enhance the Time Series platform. The add-in can be found here. I haven't tested it, so I can't comment on its functionality.

  Lastly, the Time Series platform (Analyze > Specialized Modeling > Time Series) has a lot of options that might work for you -- really depends on what you're after -- but it can do seasonal ARIMA models and forecasting. There are lots of example data and scripts in the Help > Sample Data Index.

  This should get you started, at least.

Hope this helps!,

DS

[#3]

One way to avoid part of the scripting: File->MultipleFileImport. It expects csv files in a directory.

Here are some FFT ideas if you want to use the fft() function (requires writing JSL, unlike the time series platform):

FFT Video  - showing frequencies in an audio file, every 1/30 second

Butterworth 2D FFT Filter  - filtering the high frequencies out of an image

FFT and DTMF- extracting pairs of telephone touch pad tones from a recording of dialing a number

Tracing Pictures- two 1D FFTs and code to reconstruct a reduced resolution outline

Looking back at it, I think the 4th is using the crude brick-wall filter described in the 2nd -- truncating the FFT bins rather than tapering.

[#4]

Hi @Damian ,

  Came back across this posting recently while working on some FFT work.

  As has been mentioned, you can do a FFT on a time series data, but you have to script it. You'll have to generate your own frequency column based on the number of rows and sampling rate. Also, a very important thing to keep in mind is that you need to scale the FFT results based on the number of rows you have in your data table. Why JMP doesn't do this automatically, I don't know.

  When you call the FFT function you need to include a command <<Scale(1/NRows(dt)) inside the FFT call function so it looks like this:

FFT({data}, <<Scale(1/Nrows(dt)))

  where dt is the reference to your data table in the JSL script. If you don't and you perform the inverse FFT, you'll get results that are NRows(dt) times bigger than the original data.

  Unfortunately, there's not a lot of information within the Scripting Index or online JMP Help that describes the FFT and how it's done or why the scaling is not automatically there.

  Anyway, with the FFT command you can then plot the absolute value of the FFT output vs your calculated Frequency domain and see where the peaks in frequency are.

Hope this helps!,

DS