Integrating JMP with Anaconda Python and Tensorflow

[#1]

I'm working to integrate an Anaconda environment of Python 3.8.11 with JMP to leverage Tensorflow with JMP.

I've been successful integrating JMP and the Anaconda environment and was able to verify operation of some typical machine learning packages installed in that environment.  I have been unsuccessful in importing Tensorflow.  Please see the attached code.

Names Default To Here( 1 );

Python Init(
	Path( "C:\ProgramData\Anaconda3\envs\JMP ML\python38.dll" ),
	Python Sys Path(
		{"", 
		"C:\\ProgramData\\Anaconda3\\envs\\JMP ML\\DLLs",
		"C:\\ProgramData\\Anaconda3\\envs\\JMP ML\\lib",
		"C:\\ProgramData\\Anaconda3\\envs\\JMP ML\\libs",		
		"C:\\ProgramData\\Anaconda3\\envs\\JMP ML",
		"C:\\ProgramData\\Anaconda3\\envs\\JMP ML\\lib\\site-packages"
           
		}
	)
);

Show( Python Get Version() );

Python Submit( "\[

import pandas as pd
print("Pandas Version: ", pd.__version__)

import numpy as np
print("NumPy Version: ", np.__version__)

import scipy
print("SciPy Version: ", scipy.__version__)

]\");

Python Submit( "\[

import tensorflow as tf
print(tf.__version__)

]\");

Python Term();

The above produces the following output in my embedded log:

Python Get Version() = {3, 8, 11, "final", 0};
Pandas Version:  1.3.2
NumPy Version:  1.20.3
SciPy Version:  1.6.2


/**********/


import tensorflow as tf
print(tf.__version__)

/**********/

0

When I add the statement: import tensorflow as tf

...to the first instance of Python Submit, it produces results similar to the second instance of Python Submit in that it appears that the python code is just printed into the embedded log.  I've verified that indeed I do have tensorflow installed in the environment I'm referencing.  I have also tested other packages installed in that environment and some produce the correct results while others produce what I'm calling the printed (non-functional) output.

Any ideas as to why tensorflow is not being imported correctly?  I have a ticket in with support but was curious if the community might be able to help.

Thank you in advance for any suggestions/insights.

[#2]

I'm not using Anaconda or virtual environments. I just installed with

pip3 install --user --upgrade tensorflow

and got this output with your example

Python Init();
0
Show( Python Get Version() );
Python Get Version() = {3, 9, 5, "final", 0};
Python Submit( "\[
import pandas as pd
print("Pandas Version: ", pd.__version__)
import numpy as np
print("NumPy Version: ", np.__version__)
import scipy
print("SciPy Version: ", scipy.__version__)
]\");
Pandas Version:  1.2.4
NumPy Version:  1.20.3
SciPy Version:  1.6.3
0
Python Submit( "\[
import tensorflow as tf
print(tf.__version__)
]\");
2.6.0
0
Python Term();
0

There were a lot of messages in the install about paths. Let us know if you get a more helpful answer.

View more...

Microsoft Windows [Version 10.0.19043.1165]
(c) Microsoft Corporation. All rights reserved.

C:\Users\v1>pip3 install --user --upgrade tensorflow
WARNING: Ignoring invalid distribution -pencv-python (c:\users\v1\appdata\local\programs\python\python39\lib\site-packages)
WARNING: Ignoring invalid distribution -pencv-python (c:\users\v1\appdata\local\programs\python\python39\lib\site-packages)
Collecting tensorflow
Downloading tensorflow-2.6.0-cp39-cp39-win_amd64.whl (423.3 MB)
|████████████████████████████████| 423.3 MB 26 kB/s
Collecting protobuf>=3.9.2
Downloading protobuf-3.17.3-cp39-cp39-win_amd64.whl (909 kB)
|████████████████████████████████| 909 kB 6.4 MB/s
Collecting grpcio<2.0,>=1.37.0
Downloading grpcio-1.39.0-cp39-cp39-win_amd64.whl (3.2 MB)
|████████████████████████████████| 3.2 MB 6.8 MB/s
Collecting keras~=2.6
Downloading keras-2.6.0-py2.py3-none-any.whl (1.3 MB)
|████████████████████████████████| 1.3 MB 6.4 MB/s
Collecting termcolor~=1.1.0
Downloading termcolor-1.1.0.tar.gz (3.9 kB)
Collecting clang~=5.0
Downloading clang-5.0.tar.gz (30 kB)
Collecting flatbuffers~=1.12.0
Downloading flatbuffers-1.12-py2.py3-none-any.whl (15 kB)
Collecting keras-preprocessing~=1.1.2
Downloading Keras_Preprocessing-1.1.2-py2.py3-none-any.whl (42 kB)
|████████████████████████████████| 42 kB 103 kB/s
Collecting wrapt~=1.12.1
Downloading wrapt-1.12.1.tar.gz (27 kB)
Collecting h5py~=3.1.0
Downloading h5py-3.1.0-cp39-cp39-win_amd64.whl (2.7 MB)
|████████████████████████████████| 2.7 MB 6.4 MB/s
Collecting tensorflow-estimator~=2.6
Downloading tensorflow_estimator-2.6.0-py2.py3-none-any.whl (462 kB)
|████████████████████████████████| 462 kB 6.8 MB/s
Collecting wheel~=0.35
Downloading wheel-0.37.0-py2.py3-none-any.whl (35 kB)
Collecting opt-einsum~=3.3.0
Downloading opt_einsum-3.3.0-py3-none-any.whl (65 kB)
|████████████████████████████████| 65 kB 363 kB/s
Collecting numpy~=1.19.2
Downloading numpy-1.19.5-cp39-cp39-win_amd64.whl (13.3 MB)
|████████████████████████████████| 13.3 MB 6.4 MB/s
Collecting google-pasta~=0.2
Downloading google_pasta-0.2.0-py3-none-any.whl (57 kB)
|████████████████████████████████| 57 kB 800 kB/s
Collecting absl-py~=0.10
Downloading absl_py-0.13.0-py3-none-any.whl (132 kB)
|████████████████████████████████| 132 kB ...
Collecting astunparse~=1.6.3
Downloading astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting six~=1.15.0
Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting gast==0.4.0
Downloading gast-0.4.0-py3-none-any.whl (9.8 kB)
Collecting typing-extensions~=3.7.4
Downloading typing_extensions-3.7.4.3-py3-none-any.whl (22 kB)
Collecting tensorboard~=2.6
Downloading tensorboard-2.6.0-py3-none-any.whl (5.6 MB)
|████████████████████████████████| 5.6 MB 6.4 MB/s
Requirement already satisfied: setuptools>=41.0.0 in c:\users\v1\appdata\local\programs\python\python39\lib\site-packages (from tensorboard~=2.6->tensorflow) (56.0.0)
Collecting tensorboard-data-server<0.7.0,>=0.6.0
Downloading tensorboard_data_server-0.6.1-py3-none-any.whl (2.4 kB)
Collecting tensorboard-plugin-wit>=1.6.0
Downloading tensorboard_plugin_wit-1.8.0-py3-none-any.whl (781 kB)
|████████████████████████████████| 781 kB 6.8 MB/s
Collecting google-auth-oauthlib<0.5,>=0.4.1
Downloading google_auth_oauthlib-0.4.6-py2.py3-none-any.whl (18 kB)
Collecting markdown>=2.6.8
Downloading Markdown-3.3.4-py3-none-any.whl (97 kB)
|████████████████████████████████| 97 kB 460 kB/s
Collecting requests<3,>=2.21.0
Downloading requests-2.26.0-py2.py3-none-any.whl (62 kB)
|████████████████████████████████| 62 kB 72 kB/s
Collecting werkzeug>=0.11.15
Downloading Werkzeug-2.0.1-py3-none-any.whl (288 kB)
|████████████████████████████████| 288 kB 6.8 MB/s
Collecting google-auth<2,>=1.6.3
Downloading google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
|████████████████████████████████| 152 kB 6.8 MB/s
Collecting rsa<5,>=3.1.4
Downloading rsa-4.7.2-py3-none-any.whl (34 kB)
Collecting pyasn1-modules>=0.2.1
Downloading pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
|████████████████████████████████| 155 kB ...
Collecting cachetools<5.0,>=2.0.0
Downloading cachetools-4.2.2-py3-none-any.whl (11 kB)
Collecting requests-oauthlib>=0.7.0
Downloading requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
Collecting pyasn1<0.5.0,>=0.4.6
Downloading pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
|████████████████████████████████| 77 kB 615 kB/s
Collecting urllib3<1.27,>=1.21.1
Downloading urllib3-1.26.6-py2.py3-none-any.whl (138 kB)
|████████████████████████████████| 138 kB 6.4 MB/s
Collecting idna<4,>=2.5
Downloading idna-3.2-py3-none-any.whl (59 kB)
|████████████████████████████████| 59 kB 554 kB/s
Collecting charset-normalizer~=2.0.0
Downloading charset_normalizer-2.0.4-py3-none-any.whl (36 kB)
Collecting certifi>=2017.4.17
Downloading certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
|████████████████████████████████| 145 kB 6.8 MB/s
Collecting oauthlib>=3.0.0
Downloading oauthlib-3.1.1-py2.py3-none-any.whl (146 kB)
|████████████████████████████████| 146 kB 6.4 MB/s
Using legacy 'setup.py install' for clang, since package 'wheel' is not installed.
Using legacy 'setup.py install' for termcolor, since package 'wheel' is not installed.
Using legacy 'setup.py install' for wrapt, since package 'wheel' is not installed.
WARNING: Ignoring invalid distribution -pencv-python (c:\users\v1\appdata\local\programs\python\python39\lib\site-packages)
Installing collected packages: urllib3, pyasn1, idna, charset-normalizer, certifi, six, rsa, requests, pyasn1-modules, oauthlib, cachetools, requests-oauthlib, google-auth, wheel, werkzeug, tensorboard-plugin-wit, tensorboard-data-server, protobuf, numpy, markdown, grpcio, google-auth-oauthlib, absl-py, wrapt, typing-extensions, termcolor, tensorflow-estimator, tensorboard, opt-einsum, keras-preprocessing, keras, h5py, google-pasta, gast, flatbuffers, clang, astunparse, tensorflow
WARNING: The script normalizer.exe is installed in 'C:\Users\v1\AppData\Roaming\Python\Python39\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
WARNING: The scripts pyrsa-decrypt.exe, pyrsa-encrypt.exe, pyrsa-keygen.exe, pyrsa-priv2pub.exe, pyrsa-sign.exe and pyrsa-verify.exe are installed in 'C:\Users\v1\AppData\Roaming\Python\Python39\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
WARNING: The script wheel.exe is installed in 'C:\Users\v1\AppData\Roaming\Python\Python39\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
WARNING: The script f2py.exe is installed in 'C:\Users\v1\AppData\Roaming\Python\Python39\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
WARNING: The script markdown_py.exe is installed in 'C:\Users\v1\AppData\Roaming\Python\Python39\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
WARNING: The script google-oauthlib-tool.exe is installed in 'C:\Users\v1\AppData\Roaming\Python\Python39\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Running setup.py install for wrapt ... done
Running setup.py install for termcolor ... done
WARNING: The script tensorboard.exe is installed in 'C:\Users\v1\AppData\Roaming\Python\Python39\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Running setup.py install for clang ... done
WARNING: The scripts estimator_ckpt_converter.exe, import_pb_to_tensorboard.exe, saved_model_cli.exe, tensorboard.exe, tf_upgrade_v2.exe, tflite_convert.exe, toco.exe and toco_from_protos.exe are installed in 'C:\Users\v1\AppData\Roaming\Python\Python39\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed absl-py-0.13.0 astunparse-1.6.3 cachetools-4.2.2 certifi-2021.5.30 charset-normalizer-2.0.4 clang-5.0 flatbuffers-1.12 gast-0.4.0 google-auth-1.35.0 google-auth-oauthlib-0.4.6 google-pasta-0.2.0 grpcio-1.39.0 h5py-3.1.0 idna-3.2 keras-2.6.0 keras-preprocessing-1.1.2 markdown-3.3.4 numpy-1.19.5 oauthlib-3.1.1 opt-einsum-3.3.0 protobuf-3.17.3 pyasn1-0.4.8 pyasn1-modules-0.2.8 requests-2.26.0 requests-oauthlib-1.3.0 rsa-4.7.2 six-1.15.0 tensorboard-2.6.0 tensorboard-data-server-0.6.1 tensorboard-plugin-wit-1.8.0 tensorflow-2.6.0 tensorflow-estimator-2.6.0 termcolor-1.1.0 typing-extensions-3.7.4.3 urllib3-1.26.6 werkzeug-2.0.1 wheel-0.37.0 wrapt-1.12.1
WARNING: Ignoring invalid distribution -pencv-python (c:\users\v1\appdata\local\programs\python\python39\lib\site-packages)
WARNING: Ignoring invalid distribution -pencv-python (c:\users\v1\appdata\local\programs\python\python39\lib\site-packages)
WARNING: Ignoring invalid distribution -pencv-python (c:\users\v1\appdata\local\programs\python\python39\lib\site-packages)
WARNING: You are using pip version 21.1.1; however, version 21.2.4 is available.
You should consider upgrading via the 'c:\users\v1\appdata\local\programs\python\python39\python.exe -m pip install --upgrade pip' command.

C:\Users\v1>

[#3]

Sorry if this is a silly question, but did you install tensorflow?  I ask because the other three libraries you showed come with the anaconda installer.  Where tensorflow does not.  

https://docs.anaconda.com/anaconda/packages/py3.8_win-64/

[#4]

In fact no question is silly and yes I did install Tensorflow.  

[#5]

As an additional clue, I've found that if I add in the following:

help("modules")
help("tensorflow")

.. into my Python Submit block, tensorflow is listed as an installed module and for the second help I get back:

problem in tensorflow - AttributeError: type object 'h5py.h5.H5PYConfig' has no attribute '__reduce_cython__'

This to me suggests that h5py might be an issue.  

If this sparks any interest or ideas, I'm still happy to hear them.

[#6 · Accepted solution]

So indeed my theory was correct and the issue stemmed from a bad h5py package. 

Steps to integrate an Anaconda hosted python environment with Tensorflow:

1. Create a new python Environment in Anaconda.  I named mine "JMP ML" for the script example below.  (I've reproduced this with python 3.8.11 on 9/2/2021 multiple times)
2. Install matplotlib, pandas, numpy, and scipy.
3. Check the installed version of h5py.  Anaconda will probably tell you it's 2.10.0.  If this is the case, open a terminal window and use pip install h5py==3.1.0
4. Install Tensorflow (for my repro I used v2.3).
5. Open JMP and execute a script similar to the one linked in this post.  The key parts are adding in the correct path and python system path for your environment.  What I've learned is that JMP will default to your base (root) version of python from anaconda assuming you have no other python versions installed outside anaconda.  

Names Default To Here( 1 );

Python Init(
	Path( "D:\Anaconda\envs\JMP ML\python38.dll" ),
	Python Sys Path(
		{"", 
		"D:\\Anaconda\\envs\\JMP ML\\DLLs",
		"D:\\Anaconda\\envs\\JMP ML\\lib",
		"D:\\Anaconda\\envs\\JMP ML\\libs",		
		"D:\\Anaconda\\envs\\JMP ML",
		"D:\\Anaconda\\envs\\JMP ML\\lib\\site-packages"
           
		}
	)
);

Show( Python Get Version() );

Python Submit( "\[

import matplotlib as mpl
print("MatPlotLib: ", mpl.__version__)

import pandas as pd
print("Pandas Version: ", pd.__version__)

import numpy as np
print("NumPy Version: ", np.__version__)

import scipy
print("SciPy Version: ", scipy.__version__)

import tensorflow as tf
print("Tensorflow Version: ", tf.__version__)

]\");

Python Term();

This code provides me with:

Python Get Version() = {3, 8, 11, "final", 0};
MatPlotLib: 3.4.2
Pandas Version:  1.3.2
NumPy Version:  1.20.3
SciPy Version:  1.6.2
Tensorflow Version:  2.3.0


0

And these match the versions I have installed in Anaconda in the python 3.8.11 environment.