Playing videos in JMP and evaluating video files

[#1]

Dear Community,

as videos are getting more and more important as "measurement data" to evaluate,

I'm thinking of an application that combines a video of a process with other scalar (timeseries) data.

For example, a JMP window that plays a video in the right half, and shows other time series data on the left half,

indicating with a vertical line, where on the time scale the video shows the picture corresponding to some process or measurement data.

For displaying time series data, JMP is the right choice, but whats about playing videos?

Is it possible to play a video in JMP?

Is it possible to read/analyze any video data (e.g. mp4, mpeg, ... files)?

Any experience with tasks like this, any suggestion?

Thanks in advance!

PS: using JMP16.1(Pro) on win10

[#2 · Accepted solution]

JMP handles JPG and PNG files without any external software. You can turn them into RGB matrices and do your own processing.

To load a video, frame-by-frame, you could use OpenCV and Python, something like this

JMP's Python interface using OpenCV to find a face.

I used python -m pip install opencv-python to install.

pySetup =
"\[
# https://www.c-sharpcorner.com/article/detecting-faces-from-image-and-video-using-python/
# https://stackoverflow.com/questions/66942582/error-215assertion-failed-empty-in-function-cvcascadeclassifierdetec
# https://www.nps.gov/media/video/view.htm?id=C8B260AC-1DD8-B71B-0B793ECB496E64E5

import cv2  
      
# Load the cascade  
face_cascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml')  
      
# To capture video from webcam.   
#cap = cv2.VideoCapture(0)  
cap = cv2.VideoCapture(file) 
strcap = str(cap)
#print(cap,flush=True)	
]\";

pyFinish = "\[
# Release the VideoCapture object  
cap.release()  	
]\";

pyGet =
"\[
# Read the frame  
_, img = cap.read()  
#print(img,flush=True)
# Convert to grayscale  
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)  
# Detect the faces  
faces = face_cascade.detectMultiScale(gray, 1.1, 4)  
# Draw the rectangle around each face  
for (x, y, w, h) in faces:  
    cv2.rectangle(img, (x, y), (x+w, y+h), (255, 0, 0), 2)  
  
# Display  
#cv2.imshow('Video', img)  
#print(type(img),img.shape)              
	
red = img[:,:,2]
green = img[:,:,1]
blue = img[:,:,0]

]\";

Python Init();
file = "https://www.nps.gov/nps-audiovideo/legacy/nature/C8B260AC-1DD8-B71B-0B793ECB496E64E5/nri-EOIntro3_240x134.mp4";
Python Execute( {file}, {strcap}, pySetup );
Show( strcap );

New Window( "x", bb = Border Box( Text Box( "" ) ), f=textbox() );
Wait( 1 );
frame=0;
While( 1,
    x = Log Capture( rc = Python Execute( {}, {red, green, blue, faces}, pyGet ) );
    If( Trim( x ) != "", Show( x ) );
    If( rc == -1, Break() ); // -1 is python fail on eof
    img = New Image( "rgb", {red / 255, green / 255, blue / 255} );
    (bb << child) << delete;
    bb << append( img );
    frame += 1;
    f<<settext(evalinsert("frame ^frame^ faces(^char(faces)^)"));
    bb << updatewindow;
);

Python Execute( {}, {}, pyFinish );

I borrowed the example from the link and modified it so JMP could drive it a picture at a time. It is very fast on this low res video, but most of the faces go unrecognized, probably because they are not facing straight at the camera. OpenCV has a lot of algorithms you'll find useful. I built this video a long time ago for motion detection.

I did not try the webcam, but believe it should work.

[#3]

Thanks a lot, I think this will work, I need some time to test in detail.

[#4]

Great! If you go the Python+OpenCV route, I'd like to see what you did.

(I'm a little embarrassed to not have handled the end of file exception better. The rc==-1 is detecting anything that goes wrong, which is good enough when everything goes well. You might need a try...except in the pyGet code to distinguish between the EOF exception you expect and other exceptions (like out of memory, ...) that might happen.)

Anyone else, OpenCV is a computer vision library. Face detection is just one possibility.