cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
  • JMP 19 is here! See the new features at jmp.com/new.
  • Register to attend Discovery Summit 2025 Online: Early Users Edition, Sept. 24-25.
Choose Language Hide Translation Bar
lala
Level IX

How can the download of this be completed?

The original data captured in the F12 browser

curl ^"https://cn.govopendata.com/xinwenlianbo/20250422/^" ^
  -H ^"accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7^" ^
  -H ^"accept-language: zh-CN,zh;q=0.9^" ^
  -H ^"cache-control: max-age=0^" ^
  -b ^"csrftoken=pNwjIMIrOOrnBe4GHTApOViIeNRBWwtI; _ga=GA1.1.239242113.1753140858; FCNEC=^%^5B^%^5B^%^22AKsRol_7sr_jz8BXh0YeUEUR5mSy1qL4wfsfs9XfmUOOzlIdLMJvsfgol4pcXisr7VbaTJZwRTrVSw8GGMEaAamWDh9LSXIxXj9O5k3V0EkilBGqe15w6DSZ4BceMdE4r1jUJZWTKbEm3STkdLujOfq755fcOqJP5A^%^3D^%^3D^%^22^%^5D^%^5D; _clck=1wdxmr4^%^7C2^%^7Cfxt^%^7C0^%^7C2029; _ga_7NESVRSV0K=GS2.1.s1753185097^$o3^$g1^$t1753186048^$j60^$l0^$h0; _clsk=nb3rgu^%^7C1753186048890^%^7C5^%^7C1^%^7Cv.clarity.ms^%^2Fcollect; __gads=ID=335a8fb772d726f1:T=1753140847:RT=1753186047:S=ALNI_MYKyblOPT3WE9PIgjHdPtYlQdN2wQ; __gpi=UID=0000116b9b56a5c8:T=1753140847:RT=1753186047:S=ALNI_Mb_sdW8WlGVmWp2SQ0U3v9G7fppFg; __eoi=ID=f8abc10bf0356803:T=1753140847:RT=1753186047:S=AA-Afjazt5ToEHQpkDn9PtsRpGnn^" ^
  -H ^"priority: u=0, i^" ^
  -H ^"referer: https://cn.govopendata.com/xinwenlianbo/?page=13^" ^
  -H ^"sec-ch-ua: ^\^"Not)A;Brand^\^";v=^\^"8^\^", ^\^"Chromium^\^";v=^\^"138^\^", ^\^"Google Chrome^\^";v=^\^"138^\^"^" ^
  -H ^"sec-ch-ua-mobile: ?0^" ^
  -H ^"sec-ch-ua-platform: ^\^"Windows^\^"^" ^
  -H ^"sec-fetch-dest: document^" ^
  -H ^"sec-fetch-mode: navigate^" ^
  -H ^"sec-fetch-site: same-origin^" ^
  -H ^"sec-fetch-user: ?1^" ^
  -H ^"upgrade-insecure-requests: 1^" ^
  -H ^"user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36^"

python code modified by AI:

import requests
from pathlib import Path

url = "https://cn.govopendata.com/xinwenlianbo/20250721/"

headers = {
    "sec-ch-ua": '"Not A;Brand";v="8", "Chromium";v="138", "Google Chrome";v="138"',
    "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
    "accept-language": "zh-CN,zh;q=0.9",
    "cache-control": "max-age=0",
    "referer": "https://cn.govopendata.com/xinwenlianbo/20250721/",
    "sec-ch-ua-mobile": "?0",
    "sec-ch-ua-platform": '"Windows"',
    "sec-fetch-dest": "document",
    "sec-fetch-mode": "navigate",
    "sec-fetch-site": "same-origin",
    "sec-fetch-user": "?1",
    "upgrade-insecure-requests": "1",
    "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36"
}
cookies = {
    "csrftoken": "pNwjIMIrOOrnBe4GHTApOViIeNRBWwtI",
    "__gads": "ID=335a8fb772d726f1:T=1753140847:RT=1753140847:S=ALNI_MYKyblOPT3WE9PIgjHdPtYlQdN2wQ",
    "__gpi": "UID=0000116b9b56a5c8:T=1753140847:RT=1753140847:S=ALNI_Mb_sdW8WlGVmWp2SQ0U3v9G7fppFg",
    "__eoi": "ID=f8abc10bf0356803:T=1753140847:RT=1753140847:S=AA-Afjazt5ToEHQpkDn9PtsRpGnn",
    "_ga": "GA1.1.239242113.1753140858",
    "FCNEC": '[[\"AKsRol_lyJVKvuPII10tMMWp3m_gh6UZiJ9Iej1Qg7VnR8bSTCnrLKTYNiQpPvEtsQkWDDlJCXxFJLurwTgZhoOiDfPe8XWxbQiQxlkfi3r7qGSPUlSTQyiBCTOpkIxy1RVLbQtVFHCYXz9FGzaWXlOp1l-wGib7Gg==\"]]',
    "_ga_7NESVRSV0K": "GS2.1.s1753140857$o1$g1$t1753140966$j39$l0$h0"
}

with requests.Session() as s:
    resp = s.get(url, headers=headers, cookies=cookies, timeout=10)
    resp.raise_for_status()         
    resp.encoding = resp.apparent_encoding  
    Path("xinwenlianbo_20250721.html").write_text(resp.text, encoding="utf-8")
1 ACCEPTED SOLUTION

Accepted Solutions

回复: How can the download of this be completed?

Looks like you have cert issues, Certs are supposed to be handled by a server, and not client, tho some companies _do_ install local certificates.

For that error, you can do:

request = New HTTP Request(
	URL( "https://cn.govopendata.com/xinwenlianbo/20250422/" ),
	Method( "GET" ),
       Verify SSL( "false" )
);
html = request << download("$TEMP/govopendata.html");
Web(html);

This is the equivalent of curl's --ssl-no-revoke option

 

View solution in original post

11 REPLIES 11
lala
Level IX

回复: How can the download of this be completed?

jsl not work
Thanks Experts!

u = "https://cn.govopendata.com/xinwenlianbo/20250721/";
cookie_str = "csrftoken=pNwjIMIrOOrnBe4GHTApOViIeNRBWwtI; __gads=ID=335a8fb772d726f1:T=1753140847:RT=1753140847:S=ALNI_MYKyblOPT3WE9PIgjHdPtYlQdN2wQ; __gpi=UID=0000116b9b56a5c8:T=1753140847:RT=1753140847:S=ALNI_Mb_sdW8WlGVmWp2SQ0U3v9G7fppFg; __eoi=ID=f8abc10bf0356803:T=1753140847:RT=1753140847:S=AA-Afjazt5ToEHQpkDn9PtsRpGnn; _ga=GA1.1.239242113.1753140858; FCNEC=[[\!"AKsRol_lyJVKvuPII10tMMWp3m_gh6UZiJ9Iej1Qg7VnR8bSTCnrLKTYNiQpPvEtsQkWDDlJCXxFJLurwTgZhoOiDfPe8XWxbQiQxlkfi3r7qGSPUlSTQyiBCTOpkIxy1RVLbQtVFHCYXz9FGzaWXlOp1l-wGib7Gg==\!" ]]; _ga_7NESVRSV0K=GS2.1.s1753140857$o1$g1$t1753140966$j39$l0$h0";
headers = {
    "Cookie: " || cookie_str,
    "sec-ch-ua: \!"Not A;Brand\!" ;v=\!"8\!" , \!"Chromium\!" ;v=\!"138\!" , \!"Google Chrome\!" ;v=\!"138\!"",
    "accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
    "accept-language: zh-CN,zh;q=0.9",
    "cache-control: max-age=0",
    "referer: https://cn.govopendata.com/xinwenlianbo/20250721/",
    "sec-ch-ua-mobile: ?0",
    "sec-ch-ua-platform: \!"Windows\!"",
    "sec-fetch-dest: document",
    "sec-fetch-mode: navigate",
    "sec-fetch-site: same-origin",
    "sec-fetch-user: ?1",
    "upgrade-insecure-requests: 1",
    "user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36"
};
try(
    rs = New HTTP Request(URL(u), Method("GET"), Headers(headers));
    rr = rs << Send();
    status = rr << GetStatusCode();
    If(status == 200,
        body = rr << GetBody();
        txt = Blob To Char(body, "UTF-8");
        Write File("xinwenlianbo_20250721.html", txt);
    ,
        Show("Error: Status Code = " || Char(status));
    );
);
Craige_Hales
Super User

回复: How can the download of this be completed?

What does JMP expect for the argument to the Headers(...)? I think it should be an associative array, not a list.

JSL and Python have nearly opposite use of [] and {} ; I'm sure the ai will always be confused.

 

Craige
lala
Level IX

回复: How can the download of this be completed?

Thanks Experts!
I didn't succeed when I used the JSL I was accustomed to writing.
I think it's a problem with the processing of cookies.

But I don't know how to deal with it.Could an expert please offer guidance?

 

h=[=>]; u="https://cn.govopendata.com/xinwenlianbo/20250422/";
  h["accept"]="text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7";
  h["accept-language"]="zh-CN,zh;q=0.9";
  h["cache-control"]="max-age=0";
  h["cookies"]="csrftoken=pNwjIMIrOOrnBe4GHTApOViIeNRBWwtI; _ga=GA1.1.239242113.1753140858; FCNEC=%5B%5B%22AKsRol_7sr_jz8BXh0YeUEUR5mSy1qL4wfsfs9XfmUOOzlIdLMJvsfgol4pcXisr7VbaTJZwRTrVSw8GGMEaAamWDh9LSXIxXj9O5k3V0EkilBGqe15w6DSZ4BceMdE4r1jUJZWTKbEm3STkdLujOfq755fcOqJP5A%3D%3D%22%5D%5D; _clck=1wdxmr4%7C2%7Cfxt%7C0%7C2029; _ga_7NESVRSV0K=GS2.1.s1753185097$o3$g1$t1753186048$j60$l0$h0; _clsk=nb3rgu%7C1753186048890%7C5%7C1%7Cv.clarity.ms%2Fcollect; __gads=ID=335a8fb772d726f1:T=1753140847:RT=1753186047:S=ALNI_MYKyblOPT3WE9PIgjHdPtYlQdN2wQ; __gpi=UID=0000116b9b56a5c8:T=1753140847:RT=1753186047:S=ALNI_Mb_sdW8WlGVmWp2SQ0U3v9G7fppFg; __eoi=ID=f8abc10bf0356803:T=1753140847:RT=1753186047:S=AA-Afjazt5ToEHQpkDn9PtsRpGnn";
  h["priority"]="u=0, i";
  h["referer"]="https://cn.govopendata.com/xinwenlianbo/?page=13";
  h["sec-ch-ua"]="\!"Not)A;Brand\!";v=\!"8\!", \!"Chromium\!";v=\!"138\!", \!"Google Chrome\!";v=\!"138\!"";
  h["sec-ch-ua-mobile"]="?0";
  h["sec-ch-ua-platform"]="\!"Windows\!"";
  h["sec-fetch-dest"]="document";
  h["sec-fetch-mode"]="navigate";
  h["sec-fetch-site"]="same-origin";
  h["sec-fetch-user"]="?1";
  h["upgrade-insecure-requests"]="1";
  h["user-agent"]="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36";
try(rs=New HTTP Request(URL(u),Method("GET"),Headers(h)));try(rr=rs<<Send);  a=length(rr);txt=(Blob To Char(rr));

 

Craige_Hales
Super User

回复: How can the download of this be completed?

hard to say for certain, but you don't want to hard code the token.
Generally, when you're logged into a website that's using that csrf token, you get a fresh one with each web page that contains a form that you might fill out. Depending on the website technology, the token might be part of the headers on the request, or it might be embedded in the form data that sent as a post. You appear to be doing a get rather than a post, so putting it in the headers makes sense. If you haven't updated it, you're probably using one. That's a day or so old and well expired.
Craige
lala
Level IX

回复: How can the download of this be completed?

Thank Craige!

Then I'll still handle it by calling the python code with JSL

 

The ChromiumPage library in python can automatically obtain complete packet capture data and smoothly download web page content.

回复: How can the download of this be completed?

request = New HTTP Request(
	URL( "https://cn.govopendata.com/xinwenlianbo/20250422/" ),
	Method( "GET" )
);
html = request << download("$TEMP/govopendata.html");
Web(html);

This will download and open the the webpage

lala
Level IX

回复: How can the download of this be completed?

Thanks Experts!

GET https://cn.govopendata.com/xinwenlianbo/20250422/
schannel: next InitializeSecurityContext failed: CRYPT_E_REVOCATION_OFFLINE (0x80092013)

回复: How can the download of this be completed?

Looks like you have cert issues, Certs are supposed to be handled by a server, and not client, tho some companies _do_ install local certificates.

For that error, you can do:

request = New HTTP Request(
	URL( "https://cn.govopendata.com/xinwenlianbo/20250422/" ),
	Method( "GET" ),
       Verify SSL( "false" )
);
html = request << download("$TEMP/govopendata.html");
Web(html);

This is the equivalent of curl's --ssl-no-revoke option

 

回复: How can the download of this be completed?

You generally don't use client side cookies either. Cookies are supplied by the server.

Recommended Articles