Try the Materials Informatics Toolkit, which is designed to easily handle SMILES data. This and other helpful add-ins are available in the JMP® Marketplace
I believe you are misusing the outlier analysis. It applies to linear correlations. You have a non-linear relationship between two variables. I would not expect it to correctly identify outliers and non-outliers in such a case.