Socket communication to get and log messages from JMP, R, and python scripts

[#1]

It is sometimes useful to collect messages or data in a central location from long running scripts, or scripts running on remote computers.  Attached are four files that demonstrate this:

1. A 'server' which collects messages from other machines or processes and puts them in a data table. Run this in a separate instance of JMP from any other scripts or tables.
2. A 'client' JSL script which sends messages to the server, embed this in scripts.
3. A 'client' R script which sends messages to the server
4. A 'client' python script which sends messages to the server

Here is an example of how this could look, the clients on the right can be on different computers, or separate copies of JMP on the same computer.

The important code looks like this:

Server

//Create a socket
con = Socket();

//Use a specific address and port on this computer
con << Bind( "10.0.0.2", "12345" );

//Tell this socket to be ready for other sockets to connect to it
con << Listen();

//Verify that the connection is active
con << getsockname;

//When something connects to this socket, call a function to handle
//the connection.  Accept connections for the next 5 minutes.
con << Accept(callbackfn, 300);

//Now messages can be sent to this connection.

//When all done, close the connection
con << close();

Where the callback function would look something like this (the actual function is more complex, see below):

//Simplified function to handle communication with the client, do not use as is
callbackfn = function( {x}, 
	
	If( x[2] == "ok", 
		//Get data
		r = x[4] << recv( 1000 );
		
		//While data was received
		While( r[2] == "ok",
		
			//Write it to the log
			write( Blob to Char( r[3] ) );
			
			//try to get more data
			r = x[4] << recv( 1000 );
		);
	);
	
	//close the connection to the client
	x[4] << Close;
);

Here is a slightly more complex callback function that handles blocking (common):

View more...

callbackfn = function( {x}, local( {r,msg},
	if(try(if(n items(x)==4, 1,0),0),
		If( x[2] == "ok", 
			//Get data
			r = x[4] << recv( 1000 );
			
			//While data was received
			While( (r[2] == "ok") | (left(r[2],10) == "WOULDBLOCK"),
			
				//Write it to the log
				try(write(Blob to Char( r[3] )));
				
				//try to get more data
				r = x[4] << recv( 1000 );
				if(left(r[2],10) == "WOULDBLOCK", wait(1));
			);
		);
		
		write("\!N");
		
		//close the connection to the client
		x[4] << Close;
	)
) );

Here is an example of a complete server code:

View more...

Names default to here( 1 );

//Table to hold received data
dtLog = New Table( "Log",
	New Column( "Time", Numeric, "Continuous", Format( "m/d/y h:m:s", 23, 0 ),
		Input Format( "m/d/y h:m:s", 0 ) ),
	New Column( "Client", Character, "Nominal" ),
	New Column( "Message", Character, "Nominal" )
);

//Wait for and get data from remote machine
receivedata = Function( {c, waittime = 1, waitcount = 10},
	
	//Varaible to store data received from remote machine
	blobdata = Char To Blob( "" );
	
	//Make socket is connected
	r = c << GetPeerName();
	
	//While receiving data or waiting to receive data
	i = 0;
	While( 
		( 
			(r[2] == "ok") | 
			((blobdata == Char To Blob( "" )) & (Left(r[2], 11) == "WOULDBLOCK:" ))
		) & 
		(i < waitcount),
		
		i++;
		
		//Get data
		r = c << recv( 100000 );
		
		//If no data is received yet, wait 
		If(Left(r[2], 11) == "WOULDBLOCK:", Wait( waittime ) );
		
		//Write whatever is received to the log
		Write( Blob to Char( r[3] ) );
		
		//join data to whatever is already received
		blobdata = blobdata || r[3];
	);
	
	return( blobdata );
);

//Get log info from remote machine, store it in a data table
processlogconnection = function( {c},

	//New row in log table
	dtLog << Add Rows( 1 );
	rowout = N Rows( dtLog );
	Column( dtLog, "Time" )[rowout] = Today();
	
	//Get computer name of remote machine
	r = c << GetPeerName();
	Column( dtLog, "Client" )[rowout] = r[3];
	
	//Get message from the remote machine
	Column( dtLog, "Message" )[rowout] = Blob to Char( receivedata( c ) );
	
	//Close the connection
	c << Close;
);

//Function to accept conections from remote machine, calls another
//function to handle communication with that machine
callbackfn = function( {x}, 
	//Call another function with the connection returned from the Accept message
	If( x[2] == "ok", processlogconnection( x[4] ) );
);

//Create a socket
con = Socket();

//Use a specific port on this computer
con << Bind( "10.0.0.2", "12345" );

//Tell this socket to be ready for other sockets to connect to it
con << Listen();

//Verify that the connection is active
con << getsockname;

//When something connects to this socket, call a function to handle
//the connection.  Accept connections for the next 5 minutes.
con << Accept(callbackfn, 300);

//Now messages can be sent to this connection.

//When all done, close the connection
con << close();

JSL Client

Names default to here( 1 );

//function to send a message to a waiting connection
SendMessage = Function( {message},

	//Create a socket
	con = Socket();
	
	//Connect to a socket that is already bound to, listening on, and accepting
	//connections on a port.  This is the server IP address, not the client.
	con << Connect( "10.0.0.2", "12345" );
	
	//Send data
	r = con << Send( Char To Blob( message ) );
	
	///check that the data was sent
	if(r[2] == "ok", result = "sent", result = "fail" );
	
	//close the connection
	con << Close;
	return( result );
);

//send messages
Write( SendMessage( "Hi" ) );

R Client

sendmessage = function( message ) {
  con <- socketConnection(host="10.0.0.2", port = 12345, blocking=FALSE,
                          server=FALSE, open="r+")
  
  write_resp <- write(message, con)
  close(con)
}

sendmessage("Hi from R")

Python Client

import socket
client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client_socket.connect(("10.0.0.2", 12345))
client_socket.send("Hello from Python")
client_socket.close()
break;

Powershell Client

$socket = new-object System.Net.Sockets.TcpClient( "10.0.0.2", 12345)
$stream = $socket.GetStream()
$writer = new-object System.IO.StreamWriter $stream
$writer.WriteLine( "Hello from Powershell`n" )
$writer.Flush()
$writer.Close()

This is intended to be more of an introduction to socket communication for anyone looking for example code than a functional logging mechanism but I welcome your comments and suggestions!