Force import format of Excel date column

[#1]

Hi JMP Community,

  I'm looking for help with forcing a date format import using the Excel Import Wizard via JSL. I'm trying to import data that is in only two columns in an Excel file, one date, one data, it looks like the following:

  The problem I have is at the transition from the dates 12.02.2019 to 13.02.2019 (marked by red at the left side). For some reason, JMP is reading the date values from the start to the last 12.02.2019 value as dd.mm.yyyy format, but then switches at 13.02.2019 to reading dates as mm.dd.yyyy format. When I go into the Excel file and review the formatting for the cells, they're identical. I can't figure out why JMP is suddenly switching. When it makes this switch, it leaves the date cells empty in the JMP table.

  If I import the data in two different blocks, those from the start to the last dd.mm.yyyy format, and a second one for the others, it reads each date correctly, however the Column Info for the two different dates are formatted differently and are incompatible with each other. If I try concatenating the two sub-sets, it just switches the day/month of whichever table I concatenate to.

  I've also tried changing several of the toggle options in JMP preferences to try and force system or JMP settings, but all were unsuccessful.

  If I can import the date as a character, I can modify things accordingly and then switch it back to a continuous data type, and it should be all OK. The only problem is I am not sure how to force this, or if this is the best way.

  As with most automation attempts, I STRONGLY prefer to not go in and edit every Excel sheet or file. I want to have JMP do this via JSL script.

  The JSL code I use to import is:

Open(
	"file_location\file.xlsx",
	Worksheets( "Sheet 1" ),
	Use for all sheets( 1 ),
	Concatenate Worksheets( 0 ),
	Create Concatenation Column( 0 ),
	Worksheet Settings(
		1,
		Has Column Headers( 1 ),
		Number of Rows in Headers( 1 ),
		Headers Start on Row( 1 ),
		Data Starts on Row( 2 ),
		Data Starts on Column( 15 ),
		Data Ends on Row( 0 ),
		Data Ends on Column( 16 ),
		Replicated Spanned Rows( 1 ),
		Replicated Spanned Headers( 0 ),
		Suppress Hidden Rows( 1 ),
		Suppress Hidden Columns( 1 ),
		Suppress Empty Columns( 1 ),
		Treat as Hierarchy( 0 ),
		Multiple Series Stack( 0 ),
		Import Cell Colors( 0 ),
		Limit Column Detect( 0 ),
		Column Separator String( "-" )
	)
)

  Is it related to the "Limit Column Detect(0)" option? I can't find any documentation on this and what it does.

  This issue is similar to one posted by @ghartel back in May of 2017 (https://community.jmp.com/t5/Discussions/Excel-Import-Date-Format/td-p/16619), which didn't get a direct solution to their specific issue, at least as far as I can tell.

  Any help is much appreciated!

Thanks,

DS

[#2]

Hi @briancorcoran,

  Sorry, I keep forgetting to put my system details at the start of a new post.

W7 Enterprise, 64-bit

JMP Pro 14.1.0 (and JMP 14.0.0)

  I am running an English (US) OS and trying to write some automation code for my German colleagues that are also running W7 Enterprise, but set to a German language for the OS. With other projects like this, I haven't had any issues with the dates being misread. I do have to write two different versions of code though, one for my US colleagues and one for my German colleagues because the normal US OS's read in the dates as text. And as @gzmorgan0 has mentioned, when it's read in as text, handling and converting it is no problem. But, if my German colleagues try to run the JSL scripts that were written to run on US computers, it doesn't work well, so I have to change the code to run correctly on a German language OS.

  In order to write code for running on the German computers, I change my regional/local settings in my OS to "German (Germany)". Unfortunately, JMP does not seem to import the date settings correctly from the Excel file I have.

[#3]

@SDF1, I have two suggestions:

1. JSL via Run Program() can run other programs such as VBSript.  Your JSL could be written to Open teh file via Excel, format the column as text (if needed); save the file as tab delimited text; close Excel then read in the text file where you have the option to read the column as text and then convert.
2. Excel Files can be used as a database. I did a little reading and the probelm your German colleagues are seeing might be due an Excel converter  for the data type.  Below is JSL to use Open Database(). This takes a little prep work. Your German colleagues would need to define a DSN for Excel and use the same name.  Now if everyone has 64 bit JMP and 64 bit MW Office that is pretty easy. If they have 32 bit MS Office, I can send the steps to take to set that up.  The code IMEX=1 tells Excel if it sees a column  that looks like a mixed data type to Excel the convert it to text. 

dt2 = Open Database( "DSN=Excel Files;
    DBQ=c:/temp/ExcelDateBlog.xlsx;IMEX=1",
	"SELECT * FROM [Sheet1$]",
	"Result from XLS"
);

I think working with a text file would be easier until JMP 15 when the Excel wizard allows specfiying the column formats. Let me know if you need help/example for option #1.

[#4]

Hi JMPers,

Australia also uses the "incorrect" date format :( and so days are interpreted as months and >12 become missing value due to format violation. This can happen on the second line.  it is very frustrating because if you're unlucky the first day > 12 can happen down the file where you don't notice and you end up with a missing value without warning as JMP decides it has read an incorrect date format. 

There are two things that might make it work.  First in JMP Preferences under 'Windows Specific' unckeck the box 'Use JMP language rather than System Locale settings...'.  I cant find any place in JMP where you can set a default date format - so not sure what this flag refers to.  The second step is to change the windows locale setting as the previous poster showed, BUT JMP doesnt like the dd/mm/YYYY setting - it has to be dd/MM/yy - don't know why, but then it works at least for me.  Don't know if this is jsut a Windows problem, ie if Macs don't have this problem.  I run JMP on a windows 7 machine and a windows 10 machine - JMP PRO 14.3 64 bit, but I've had this problem since Excel imports were a thing in JMP.  

Some suggested reading in the column as text - I can't see any options for forcing JMP to read specific columns in as text - is that possible?

Another workaround is to get your data in CSV format as for some reason JMP allows very minute control over inputs unlike for Excel.  The third option is to read in Excel files using the database interface.  But that requires appropriate ODBC settings on your PC and it is a bit of a pain too.

Note to JMP - it would be very good if JMP sent some warning or error messages if it fails to import a field due to wrong format.  I have clients who add * or < to numeric fields which Excel happily turns into character fields and JMP happily and silenty turns into missing values.  JSL is fantastic for automating tasks but this sort of thing makes it very hard to error proof your programs.

Cheers

Gunter

[#5]

@ghartel, Note that @briancorcoran mentioned there might be more options in JMP 15.

Attached is a script that saves an Excel sheet as a tab delimited text file then reads in the text file into JMP. 

You might want to try reading the text file with your system defaults (in Australia and Germany), to see if the JMP import text defaults will read the Date as a text. If not, then you have to name all your columns and specify the data type.

Since this script uses VBScript, it likley doe not work for a Mac.

The script is provided as a "just in case this might help." 

[#6]

Thanks @gzmorgan0, especially for the example import file. I'll be able to test it out next week and see if it works, not able to get to it soon.

Look forward to the new "force" option in JMP 15.

Thanks!,
DS

[#7]

Hi @gzmorgan0, 

  I've finally gotten back and been able to test out your JSL script that uses VB to save as a txt. It works, and I can import the information as needed, my only problem is that it only does it for the last tab in the Excel file.

  My Excel file has multiple tabs that I need to import, all with the date problem. Once in JMP, I concatenate all the data into a single data table for analysis. I also just found out that I will be needing to modify the JSL to open multiple of these Excel files and then put all the data together. An added level of complexity.

  Unfortunately, I'm not familiar enough with VB to write something that handles multiple tabs. I have a colleague who is though, and they're working on it.

  I've tried changing my preferences in JMP to see if that fixes the issue, but it doesn't. There definitely seems to be a bug that even though I set my OS date format to Germany (d.m.y), it still uses the install OS (English) and reads the date as m.d.y, so it just ignores any day greater than 12.

  The JSL script is very helpful, thanks for posting it. We'll just need to modify the VB portion to deal with multiple tabs, that's all.