Solved: Re: How to use JSL form of "POST" WEB data?

lwx228 · Nov 20, 2019 06:11 AM

After I learned to use JMP to download web page data, ga mother likes to use JMP to download web page data now, but it is more difficult to process the web page of POST.

Get expert help: how to download data using JSL in the following way:

POST http://listxbrl.sse.com.cn/companyInfo/showBalance.do HTTP/1.1
Host: listxbrl.sse.com.cn
Connection: keep-alive
Content-Length: 54
Accept: */*
Origin: http://listxbrl.sse.com.cn
X-Requested-With: XMLHttpRequest
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Referer: http://listxbrl.sse.com.cn/companyInfo/toCompanyInfo.do?stock_id=600007&report_period_id=5000

report_year=2018&stock_id=600007&report_period_id=5000

I can do it with VBA.

Thanks!

Ryan_Gilmore · Nov 25, 2019 09:08 AM

These lines of code,

query = Associative Array();
query["report_year"] = 2018;
query["stock_id"] = 600007;
query["report_period_id"] = 5000;

are used to construct the query parameter list in the URL, e.g.

report_year=2018&stock_id=600007&report_period_id=5000

In the original code I supplied, the code had

query["period_id"] = 5000;

which would have given an incorrect query parameter list of

report_year=2018&stock_id=600007&period_id=5000

View solution in original post

bryan_boone · Nov 25, 2019 09:36 AM

Using example in the thread, you'd want something like:

query = Associative Array();
query["report_year"] = 2018;
query["stock_id"] = 600007;
query["period_id"] = 5000;

request = HTTP Request(
	Url( "http://listxbrl.sse.com.cn/companyInfo/showBalance.do" ),
	Method( "POST" ),
	Form( Fields(query) )
);

request << Send;

The Form message has been there since the beginning.

It has Fields and Files sub parts to it. Both are associative arrays.

In the "Fields" case it's name/value pairs where the value can be either character or a List of characters.

In the "Files" case it's name/value pairs where the value can be either filename or a List of filenames.

If only "Fields" is used, then the request header is automatically set to application/x-www-form-urlencoded

If "Fields" and "Files" (or just "Files") is used, then the request header is automatically set to multipart/form-data.

I'll update the thread with this information, too.

View solution in original post

lwx228 · Nov 20, 2019 06:15 AM

I searched for-
Method("POST")
in the help file 、but found no similar code.

lwx228 · Nov 21, 2019 02:54 AM

Below is the VBA code, can be downloaded to get the page data.
But I still haven't been able to convert it into JSL code.

Please give me help.Thank you very much!

VBA:

Sub Main()
    Dim strText As String
    With CreateObject("MSXML2.XMLHTTP")
        .Open "POST", "http://listxbrl.sse.com.cn/companyInfo/showBalance.do", False
        .setRequestHeader "Content-Type", "application/x-www-form-urlencoded"
        .Send "report_year=2018&stock_id=600007&report_period_id=5000"
        strText = .responsetext
        Debug.Print strText
    End With
    Cells(1, 1) = strText
End Sub

Ryan_Gilmore · Nov 21, 2019 09:51 AM

I think the following might be close to what you need,

request_headers = Associative Array();
request_headers["Content-type"] = "application/x-www-form-urlencoded";
	
query = Associative Array();
query["report_year"] = 2018;
query["stock_id"] = 600007;
query["period_id"] = 5000;

request = HTTP Request(
	Url( "http://listxbrl.sse.com.cn/companyInfo/showBalance.do" ),
	Method( "POST" ),
	Headers( request_headers ),
	QueryString( query )
);

lwx228 · Nov 21, 2019 07:10 PM

Thank Ryan_Gilmore !

This approach still hasn't worked.

It seems that this form of POST method, JMP application is not yet.

Ryan_Gilmore · Nov 22, 2019 07:35 AM

Once the request has been created. you then need to issue the Send command, e.g.

data = request << Send;
If( request << IsSuccess,
    json_data = Parse JSON( data );
);

lwx228 · Nov 22, 2019 07:09 PM

My knowledge of the JMP approach is limited.
I can only imitate.Still no results.

Thank Ryan_Gilmore!

lwx228 · Nov 22, 2019 07:12 PM

get "DATA" of theta:

" <!DOCTYPE html PUBLIC \!"-//W3C//DTD XHTML 1.0 Transitional//EN\!" \!"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\!"> <html xmlns=\!"http://www.w3.org/1999/xhtml\!"> <head> <meta http-equiv=\!"Content-Type\!" content=\!"text/html; charset=utf-8\!" />
……
</p> </body> </html> "

Ryan_Gilmore · Nov 25, 2019 08:16 AM

I notice that one of the parameters is incorrect. The code I suggested had "period_id" rather than "report_period_id" as shown in your original post. It's possible that may be contributing to the "500 Internal Server Error" message.

lwx228 · Nov 25, 2019 09:01 AM

I SEE.