topic Playing videos in JMP and evaluating video files in Discussions

Playing videos in JMP and evaluating video files

Georg — Sun, 11 Jun 2023 11:23:43 GMT

Dear Community,

as videos are getting more and more important as "measurement data" to evaluate,

I'm thinking of an application that combines a video of a process with other scalar (timeseries) data.

For example, a JMP window that plays a video in the right half, and shows other time series data on the left half,

indicating with a vertical line, where on the time scale the video shows the picture corresponding to some process or measurement data.

For displaying time series data, JMP is the right choice, but whats about playing videos?

Is it possible to play a video in JMP?

Is it possible to read/analyze any video data (e.g. mp4, mpeg, ... files)?

Any experience with tasks like this, any suggestion?

Thanks in advance!

PS: using JMP16.1(Pro) on win10

Re: Playing videos in JMP and evaluating video files

Craige_Hales — Thu, 14 Apr 2022 03:49:33 GMT

JMP handles JPG and PNG files without any external software. You can turn them into RGB matrices and do your own processing.

To load a video, frame-by-frame, you could use OpenCV and Python, something like this

JMP's Python interface using OpenCV to find a face.

I used python -m pip install opencv-python to install.

pySetup =
"\[
# https://www.c-sharpcorner.com/article/detecting-faces-from-image-and-video-using-python/
# https://stackoverflow.com/questions/66942582/error-215assertion-failed-empty-in-function-cvcascadeclassifierdetec
# https://www.nps.gov/media/video/view.htm?id=C8B260AC-1DD8-B71B-0B793ECB496E64E5

import cv2  
      
# Load the cascade  
face_cascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml')  
      
# To capture video from webcam.   
#cap = cv2.VideoCapture(0)  
cap = cv2.VideoCapture(file) 
strcap = str(cap)
#print(cap,flush=True)	
]\";

pyFinish = "\[
# Release the VideoCapture object  
cap.release()  	
]\";

pyGet =
"\[
# Read the frame  
_, img = cap.read()  
#print(img,flush=True)
# Convert to grayscale  
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)  
# Detect the faces  
faces = face_cascade.detectMultiScale(gray, 1.1, 4)  
# Draw the rectangle around each face  
for (x, y, w, h) in faces:  
    cv2.rectangle(img, (x, y), (x+w, y+h), (255, 0, 0), 2)  
  
# Display  
#cv2.imshow('Video', img)  
#print(type(img),img.shape)              
	
red = img[:,:,2]
green = img[:,:,1]
blue = img[:,:,0]

]\";

Python Init();
file = "https://www.nps.gov/nps-audiovideo/legacy/nature/C8B260AC-1DD8-B71B-0B793ECB496E64E5/nri-EOIntro3_240x134.mp4";
Python Execute( {file}, {strcap}, pySetup );
Show( strcap );

New Window( "x", bb = Border Box( Text Box( "" ) ), f=textbox() );
Wait( 1 );
frame=0;
While( 1,
    x = Log Capture( rc = Python Execute( {}, {red, green, blue, faces}, pyGet ) );
    If( Trim( x ) != "", Show( x ) );
    If( rc == -1, Break() ); // -1 is python fail on eof
    img = New Image( "rgb", {red / 255, green / 255, blue / 255} );
    (bb << child) << delete;
    bb << append( img );
    frame += 1;
    f<<settext(evalinsert("frame ^frame^ faces(^char(faces)^)"));
    bb << updatewindow;
);

Python Execute( {}, {}, pyFinish );

I borrowed the example from the link and modified it so JMP could drive it a picture at a time. It is very fast on this low res video, but most of the faces go unrecognized, probably because they are not facing straight at the camera. OpenCV has a lot of algorithms you'll find useful. I built this video a long time ago for motion detection.

I did not try the webcam, but believe it should work.

Re: Playing videos in JMP and evaluating video files

Georg — Tue, 19 Apr 2022 09:47:19 GMT

Thanks a lot, I think this will work, I need some time to test in detail.

Re: Playing videos in JMP and evaluating video files

Craige_Hales — Tue, 19 Apr 2022 13:19:08 GMT

Great! If you go the Python+OpenCV route, I'd like to see what you did.

(I'm a little embarrassed to not have handled the end of file exception better. The rc==-1 is detecting anything that goes wrong, which is good enough when everything goes well. You might need a try...except in the pyGet code to distinguish between the EOF exception you expect and other exceptions (like out of memory, ...) that might happen.)

Anyone else, OpenCV is a computer vision library. Face detection is just one possibility.