Solved: Re: Use JSL to get data from the web - Page 2

uday_guntupalli · Oct 28, 2016 6:25 AM

Hello All,

After looking at this discussion (Use JSL to Scrape Data From the Web and Predict Football Wins!) - I got very excited about the # of data sets available online to enhance my JSL skills. However - I haven't had much success after trying the code provided in this post. I tried to write to the author and am still waiting to hear back. If some one has done this before - can someone - please show how to get data from the web .

Lets say For E.g. I want to get the stock price for Twitter , its 52 - week low , 52-week high and current price - how can I do that in JSL .

Any inputs are appreciated.

Best

uday

Best
Uday

uday_guntupalli · Apr 15, 2016 06:14 PM

@msharp

I am connected to the internet.

I am able to open the pages in my browser ,

I am using JMP 12

I have Windows.

uday

Best
Uday

msharp · Apr 15, 2016 07:08 PM

Yeah, I figured you were connected to the internet since your posting on the forums

Like I said, I'm not sure what the issue is. I'd just try the normal troubleshooting techniques:

Opening closing JMP.

Restarting your computer.

Clearing cache.

Reinstalling JMP? There is a JMP 12.2 you could update to.

Alternatively you can contact support@jmp.com. They might be able to provide additional support.

Craige_Hales · Apr 16, 2016 10:19 AM

the 302 error is a redirection. I don't know why you are seeing that; it might be a firewall behavior.

If you want to try to puzzle it out, run this script, a line at a time, and see what each line produces in the JMP log.

site="finance.yahoo.com";

url="http://finance.yahoo.com/echarts?s=TWTR#{\!"allowChartStacking\!":true}";

tCall = Socket();

tCall << connect( site, "80" );

tCall << Send( Char To Blob( "GET " || url || " HTTP/1.0~0d~0a~0d~0a", "ASCII~HEX" ) );

tMessage = tCall << Recv( 1000 );

text = Blob To Char( tMessage[3], "ASCII~HEX" );

Show( text );

tCall << Close();

connect should say: {"connect", "ok"}

send should say: {"Send", "ok", Char To Blob( "", "ascii~hex" )}

recv should say: {"Recv", "ok", Char To Blob(

"HTTP/1.0 200 OK~0D~0AX-Frame-Options: SAMEORIGIN~0D~0AX-Content-Type-Options: nosniff~0D~0AX-XSS-Protection: 1; mode=block~0D~0AContent-Type: text/html~0D~0ASet-Cookie: B=c3r14p5bh4hr3&b=3&s=p2; expires=Sat, 16-Apr-2018 14:08:35 GMT; path=/; domain=.yahoo.com~0D~0ADate: Sat, 16 Apr 2016 14:08:35 GMT~0D~0AServer: ATS~0D~0AVary: X-Ssl~0D~0AVia: http/1.1 media-border115.global.media.bf1.yahoo.com (ApacheTrafficServer [cMsSf ]), http/1.1 media-router4.prod.media.bf1.yahoo.com (ApacheTrafficServer [cMsSf ]), http/1.1 r25.ycpi.dcb.yahoo.net (ApacheTrafficServer [cMsSf ])~0D~0AAge: 0~0D~0AY-Trace: BAEAQAAAAACXg0qXxLFf4AAAAAAAAAAAw514XKmQNkkAAAAAAAAAAAAFMJqpyGJPAAUwmqnLTak2C_LwAAAAAA--~0D~0A~0D~0A<~21DOCTYPE html>~0A<html>~0A<head>~0A <~21-- customizable : anything you expected. -->~0A <title>TWTR Interactive Stock Chart | Yahoo~21 Inc. Stock - Yahoo~21 Finance</title>~0A~0A <meta http-equiv=~22X-UA-Compatible~22 content=~22IE=edge,chrome=1~22 />~0A <meta http-equiv=~22Content-Type~22 content=~22text/html; charset=UTF-8~22 />~0A~0A~0A~0A ~0A <link rel=~22stylesheet~22 type","ascii~hex")}

If you don't get the 200 OK response, maybe you'll get some information about the 302 response.

Craige

uday_guntupalli · Apr 18, 2016 09:48 AM

Craige@JMP :

text = "HTTP/1.1 302 Found~0D~0ALocation: http://tmp1mdf01-fpx01/?cfru=aHR0cDovL2ZpbmFuY2UueWFob28uY29tL2VjaGFydHM/cz1UV1RSI3siYWxsb3dDaGFydFN... no-cache~0D~0APragma: no-cache~0D~0AContent-Type: text/html; charset=utf-8~0D~0AConnection: close~0D~0AContent-Length: 624~0D~0A~0D~0A<HTML><HEAD>~0A<TITLE>Redirect</TITLE>~0A</HEAD>~0A<BODY>~0A~0A<big></big> ~0A~0A<blockquote>~0A<TABLE border=0 cellPadding=1 width=~2280%~22>~0A<TR><TD>~0A~0A<big>Redirect (authentication_redirect_to_virtual_host)</big>~0A ~0A ~0A~0A</TD></TR>~0A<TR><TD>~0A~0AYou are being redirected to the authentication virtual host.~0A~0A</TD></TR>~0A<TR><TD>~0A~0A~0A~0A</TD></TR>~0A<TR><TD>~0A~0A ~0AFor assistance, contact your network support team.~0A~0A</TD></TR>~0A</TABLE>~0A</blockquote>~0A~0A</BODY></HTML>~0A";

{"Close", "ok"}

This is what I got

Best
Uday

msharp · Apr 18, 2016 10:39 AM

Key information from above:

Redirect (authentication_redirect_to_virtual_host)

You are being redirected to the authentication virtual host.

For assistance, contact your network support team.

Doing a quick search through google, everything seems to be pointing to a proxy issue. This is something your company network support team should be able to easily solve. Have you contacted them?

I might personally try manually setting up the proxy server information in any web browsers you might have, since you are on Windows I'd start with IE. Change proxy server settings in Internet Explorer - Windows Help
However, this will require you to contact the network support team anyways, since I assume you don't know your company's proxy settings.

Craige_Hales · Apr 18, 2016 10:43 AM

I'm out of my depth. from the messages (which include one saying contact your support team) I think it is a firewall/proxy issue. See:

http://stackoverflow.com/questions/2408964/why-am-i-getting-authentication-redirect-to-virtual-host-...

(which I think might be asking the correct question)

Craige

theresagoodyear · Apr 21, 2016 09:56 AM

Hi Uday.

I was interested in getting football data from the web as well after attending that seminar in San Diego. I created the attached script and got quarterback data from espn.com. I figured out I could get historical data simply be replacing the link with the corresponding year. Once I got historical data and concatenated it, I ran a regression analysis and found that 'passing yards' is the biggest contributor to the quarterback's rank on the ESPN top QB list.

I've modeled other analysis after this one. Hope this helps. Thank you.

uday_guntupalli · Apr 21, 2016 10:17 AM

theresagoodyear : Thanks for sharing the script and analysis Theresa. I think the primary issue that has plagued me in this context has been that I have network related issues as Craige and MSharp have previously assessed. My network has been redirecting me out of the web when ever I try to access it through JSL. Once I am able to get through to the data - it will be a great experience to start looking at your work and learning what ever is helpful and incorporating those concepts

Best
Uday