cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Browse apps to extend the software in the new JMP Marketplace
Choose Language Hide Translation Bar
NRW
NRW
Level IV

Import character value issue

Folks,

 

I'm importing values from cvs files. One of them is a string. However, JMP is unable to read the value. The first image is the cvs file where all values are imported except "FlexSEM1000." So no issues there. However, second image shows the JMP table with the F????? value. How do I tackle this issue? Relevant code attached.

 

Neil

Join_files = Expr(
		
	For( i = 1, i <= N Items( path ), i+=2, 
			/*area, %area, and pore pixel count*/
			jointable1 = Open( path[i] );
			//jointable1 << Get Name;
			samplename = Substr( jointable1 << Get Name, 1, 18 );			//sample replicate name //
			samplename2 = Word(1,samplename, "_");							// Only sample name, no replicate suffix (i.e. a1, a2...)			
			
			
			
			jointable1 << New Column( "Sample",
				Character,
				Nominal,	
				Set Each Value( samplename )								// setting each row in column 1 to samp rep name //
				);
			
			/*diameter and pore pixel count*/
			jointable2 = Open( path[i + 1] );

			dt = Data Table( jointable1 ) << Join(
				With( Data Table( jointable2 ) ),
				Select( :Sample, :Area, :Name( "%Area" ),:scale, :thick, :pixelrad, :mag, :system),
				SelectWith( :pixD1, :count1), 
				Cartesian Join	
				);
			
			Close( jointable1, no save );
			Close( jointable2, no save );
			
			Columns();
			Graphs();
			Graphs2();
			Graphs3();
			dt << Set Name(samplename);
			po = dt << get name;
			dt << Save(SD1 || po);
			
		);
	
	
);


Columns = Eval Expr(
	//dt << New Column( "Condition", Character, Nominal, Set Each Value( "1:1" ) );
	dt:Name( "%Area" ) << Set Name( "Image A pore area (%)", Numeric, Continuous );
	dt:Area << Set Name( "Total Image pixel count (pore_part)", Numeric, Continuous );
	dt:scale << Set Name("pix/mm scale", Numeric, Continuous);
	dt:pixD1 << Set Name( "Pore Diameter (pixel)", Numeric, Continuous );
	dt:count1 << Set Name( "Pore Pixel Count", Numeric, Continuous );
	dt:thick << Set Name("d - thickness (m)", Numeric, Continuous);
	dt:pixelrad << Set Name("LT Radius", Numeric, Continuous);
	dt:mag << Set Name("Mag", Numeric, Continuous);
	dt:system << Set Name("Image Mode", Character, Nominal);
	dt:Sample << Set Display Width( 150 );
	
	

charac import_a.JPGcharac import_b.JPG 

Neil
1 ACCEPTED SOLUTION

Accepted Solutions
Craige_Hales
Super User

Re: Import character value issue

Depending on the editor, retyping it might change it back to the preferred unicode charset. Ideally the program that produces the data would use ascii or unicode. Or, if that can't work, tell jmp which charset to use via the charset option.
Craige

View solution in original post

8 REPLIES 8
Craige_Hales
Super User

Re: Import character value issue

Away from computer, can't give exact words...
Try using an alternate charset, either through preferences.. text import, or through the text import wizard. The diamond question mark means the character set jmp guessed is not correct.
Craige
NRW
NRW
Level IV

Re: Import character value issue

Thanks Craige,

Per the cvs value "FlexSEM1000", when I change this to "FlexSEM", JMP is able to read without issues

 

I'll poke around using your suggestions.

 

Neil

 

Neil
Craige_Hales
Super User

Re: Import character value issue

Depending on the editor, retyping it might change it back to the preferred unicode charset. Ideally the program that produces the data would use ascii or unicode. Or, if that can't work, tell jmp which charset to use via the charset option.
Craige
NRW
NRW
Level IV

Re: Import character value issue

Craige,

 

I'll keep plugging away. I tried attaching the csv doc in the case someone could get it to work, but the site was not allowing the upload.

 

In the meantime I'll go to program that generated the file to see how it is formatting. Originally the system ID data was being read from a system text file (accompanied with images from a SEM) which was UTF16. However a script was written to change to UTF8 and save to csv file. 

I've had JMP look at both types, but no success.

 

Neil

 

Neil
Craige_Hales
Super User

Re: Import character value issue

You can see the hex representation of the file using jmp. Approximately like this...
Write(hex(loadtextfile(filename,blob)))
Craige
NRW
NRW
Level IV

Re: Import character value issue

Craige,

 

Quick update.

 

I found out if I manually open the csv file and save as a "csv UTF-8", JMP will recognize  "FlexSEM1000". However that's a manual step I would like to eliminate. 

 

Neil

Neil
NRW
NRW
Level IV

Re: Import character value issue

Craige,

 

I was able to go back to the source code that generated the csv, and found the issue. All the items in the the table were UTF-8, EXCEPT the last column item of interest. I fixed the code and now everything works. I guess JMP was seeing both UTF-8 and UTF-16 and and since it was already seeing UTF-8, didn't recognize the last column.

 

Many thanks for all your valuable time.

 

Neil

Neil
Craige_Hales
Super User

Re: Import character value issue

Yes, if there is a unicode Byte Order Mark at the beginning, Jmp believes it. Without a BOM, I'm not sure exactly how the guess proceeds. Glad you found it!
Craige