topic 回复： How can the download of this be completed? in Discussions

How can the download of this be completed?

lala — Tue, 22 Jul 2025 13:22:00 GMT

The original data captured in the F12 browser

curl ^"https://cn.govopendata.com/xinwenlianbo/20250422/^" ^
  -H ^"accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7^" ^
  -H ^"accept-language: zh-CN,zh;q=0.9^" ^
  -H ^"cache-control: max-age=0^" ^
  -b ^"csrftoken=pNwjIMIrOOrnBe4GHTApOViIeNRBWwtI; _ga=GA1.1.239242113.1753140858; FCNEC=^%^5B^%^5B^%^22AKsRol_7sr_jz8BXh0YeUEUR5mSy1qL4wfsfs9XfmUOOzlIdLMJvsfgol4pcXisr7VbaTJZwRTrVSw8GGMEaAamWDh9LSXIxXj9O5k3V0EkilBGqe15w6DSZ4BceMdE4r1jUJZWTKbEm3STkdLujOfq755fcOqJP5A^%^3D^%^3D^%^22^%^5D^%^5D; _clck=1wdxmr4^%^7C2^%^7Cfxt^%^7C0^%^7C2029; _ga_7NESVRSV0K=GS2.1.s1753185097^$o3^$g1^$t1753186048^$j60^$l0^$h0; _clsk=nb3rgu^%^7C1753186048890^%^7C5^%^7C1^%^7Cv.clarity.ms^%^2Fcollect; __gads=ID=335a8fb772d726f1:T=1753140847:RT=1753186047:S=ALNI_MYKyblOPT3WE9PIgjHdPtYlQdN2wQ; __gpi=UID=0000116b9b56a5c8:T=1753140847:RT=1753186047:S=ALNI_Mb_sdW8WlGVmWp2SQ0U3v9G7fppFg; __eoi=ID=f8abc10bf0356803:T=1753140847:RT=1753186047:S=AA-Afjazt5ToEHQpkDn9PtsRpGnn^" ^
  -H ^"priority: u=0, i^" ^
  -H ^"referer: https://cn.govopendata.com/xinwenlianbo/?page=13^" ^
  -H ^"sec-ch-ua: ^\^"Not)A;Brand^\^";v=^\^"8^\^", ^\^"Chromium^\^";v=^\^"138^\^", ^\^"Google Chrome^\^";v=^\^"138^\^"^" ^
  -H ^"sec-ch-ua-mobile: ?0^" ^
  -H ^"sec-ch-ua-platform: ^\^"Windows^\^"^" ^
  -H ^"sec-fetch-dest: document^" ^
  -H ^"sec-fetch-mode: navigate^" ^
  -H ^"sec-fetch-site: same-origin^" ^
  -H ^"sec-fetch-user: ?1^" ^
  -H ^"upgrade-insecure-requests: 1^" ^
  -H ^"user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36^"

python code modified by AI:

import requests
from pathlib import Path

url = "https://cn.govopendata.com/xinwenlianbo/20250721/"

headers = {
    "sec-ch-ua": '"Not A;Brand";v="8", "Chromium";v="138", "Google Chrome";v="138"',
    "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
    "accept-language": "zh-CN,zh;q=0.9",
    "cache-control": "max-age=0",
    "referer": "https://cn.govopendata.com/xinwenlianbo/20250721/",
    "sec-ch-ua-mobile": "?0",
    "sec-ch-ua-platform": '"Windows"',
    "sec-fetch-dest": "document",
    "sec-fetch-mode": "navigate",
    "sec-fetch-site": "same-origin",
    "sec-fetch-user": "?1",
    "upgrade-insecure-requests": "1",
    "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36"
}
cookies = {
    "csrftoken": "pNwjIMIrOOrnBe4GHTApOViIeNRBWwtI",
    "__gads": "ID=335a8fb772d726f1:T=1753140847:RT=1753140847:S=ALNI_MYKyblOPT3WE9PIgjHdPtYlQdN2wQ",
    "__gpi": "UID=0000116b9b56a5c8:T=1753140847:RT=1753140847:S=ALNI_Mb_sdW8WlGVmWp2SQ0U3v9G7fppFg",
    "__eoi": "ID=f8abc10bf0356803:T=1753140847:RT=1753140847:S=AA-Afjazt5ToEHQpkDn9PtsRpGnn",
    "_ga": "GA1.1.239242113.1753140858",
    "FCNEC": '[[\"AKsRol_lyJVKvuPII10tMMWp3m_gh6UZiJ9Iej1Qg7VnR8bSTCnrLKTYNiQpPvEtsQkWDDlJCXxFJLurwTgZhoOiDfPe8XWxbQiQxlkfi3r7qGSPUlSTQyiBCTOpkIxy1RVLbQtVFHCYXz9FGzaWXlOp1l-wGib7Gg==\"]]',
    "_ga_7NESVRSV0K": "GS2.1.s1753140857$o1$g1$t1753140966$j39$l0$h0"
}

with requests.Session() as s:
    resp = s.get(url, headers=headers, cookies=cookies, timeout=10)
    resp.raise_for_status()         
    resp.encoding = resp.apparent_encoding  
    Path("xinwenlianbo_20250721.html").write_text(resp.text, encoding="utf-8")

回复： How can the download of this be completed?

lala — Tue, 22 Jul 2025 13:22:52 GMT

jsl not work
Thanks Experts!

u = "https://cn.govopendata.com/xinwenlianbo/20250721/";
cookie_str = "csrftoken=pNwjIMIrOOrnBe4GHTApOViIeNRBWwtI; __gads=ID=335a8fb772d726f1:T=1753140847:RT=1753140847:S=ALNI_MYKyblOPT3WE9PIgjHdPtYlQdN2wQ; __gpi=UID=0000116b9b56a5c8:T=1753140847:RT=1753140847:S=ALNI_Mb_sdW8WlGVmWp2SQ0U3v9G7fppFg; __eoi=ID=f8abc10bf0356803:T=1753140847:RT=1753140847:S=AA-Afjazt5ToEHQpkDn9PtsRpGnn; _ga=GA1.1.239242113.1753140858; FCNEC=[[\!"AKsRol_lyJVKvuPII10tMMWp3m_gh6UZiJ9Iej1Qg7VnR8bSTCnrLKTYNiQpPvEtsQkWDDlJCXxFJLurwTgZhoOiDfPe8XWxbQiQxlkfi3r7qGSPUlSTQyiBCTOpkIxy1RVLbQtVFHCYXz9FGzaWXlOp1l-wGib7Gg==\!" ]]; _ga_7NESVRSV0K=GS2.1.s1753140857$o1$g1$t1753140966$j39$l0$h0";
headers = {
    "Cookie: " || cookie_str,
    "sec-ch-ua: \!"Not A;Brand\!" ;v=\!"8\!" , \!"Chromium\!" ;v=\!"138\!" , \!"Google Chrome\!" ;v=\!"138\!"",
    "accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
    "accept-language: zh-CN,zh;q=0.9",
    "cache-control: max-age=0",
    "referer: https://cn.govopendata.com/xinwenlianbo/20250721/",
    "sec-ch-ua-mobile: ?0",
    "sec-ch-ua-platform: \!"Windows\!"",
    "sec-fetch-dest: document",
    "sec-fetch-mode: navigate",
    "sec-fetch-site: same-origin",
    "sec-fetch-user: ?1",
    "upgrade-insecure-requests: 1",
    "user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36"
};
try(
    rs = New HTTP Request(URL(u), Method("GET"), Headers(headers));
    rr = rs << Send();
    status = rr << GetStatusCode();
    If(status == 200,
        body = rr << GetBody();
        txt = Blob To Char(body, "UTF-8");
        Write File("xinwenlianbo_20250721.html", txt);
    ,
        Show("Error: Status Code = " || Char(status));
    );
);

回复： How can the download of this be completed?

Craige_Hales — Tue, 22 Jul 2025 16:50:12 GMT

What does JMP expect for the argument to the Headers(...)? I think it should be an associative array, not a list.

JSL and Python have nearly opposite use of [] and {} ; I'm sure the ai will always be confused.

回复： How can the download of this be completed?

lala — Tue, 22 Jul 2025 23:56:51 GMT

Thanks Experts!
I didn't succeed when I used the JSL I was accustomed to writing.I think it's a problem with the processing of cookies.

But I don't know how to deal with it.Could an expert please offer guidance?

h=[=>]; u="https://cn.govopendata.com/xinwenlianbo/20250422/";
  h["accept"]="text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7";
  h["accept-language"]="zh-CN,zh;q=0.9";
  h["cache-control"]="max-age=0";
  h["cookies"]="csrftoken=pNwjIMIrOOrnBe4GHTApOViIeNRBWwtI; _ga=GA1.1.239242113.1753140858; FCNEC=%5B%5B%22AKsRol_7sr_jz8BXh0YeUEUR5mSy1qL4wfsfs9XfmUOOzlIdLMJvsfgol4pcXisr7VbaTJZwRTrVSw8GGMEaAamWDh9LSXIxXj9O5k3V0EkilBGqe15w6DSZ4BceMdE4r1jUJZWTKbEm3STkdLujOfq755fcOqJP5A%3D%3D%22%5D%5D; _clck=1wdxmr4%7C2%7Cfxt%7C0%7C2029; _ga_7NESVRSV0K=GS2.1.s1753185097$o3$g1$t1753186048$j60$l0$h0; _clsk=nb3rgu%7C1753186048890%7C5%7C1%7Cv.clarity.ms%2Fcollect; __gads=ID=335a8fb772d726f1:T=1753140847:RT=1753186047:S=ALNI_MYKyblOPT3WE9PIgjHdPtYlQdN2wQ; __gpi=UID=0000116b9b56a5c8:T=1753140847:RT=1753186047:S=ALNI_Mb_sdW8WlGVmWp2SQ0U3v9G7fppFg; __eoi=ID=f8abc10bf0356803:T=1753140847:RT=1753186047:S=AA-Afjazt5ToEHQpkDn9PtsRpGnn";
  h["priority"]="u=0, i";
  h["referer"]="https://cn.govopendata.com/xinwenlianbo/?page=13";
  h["sec-ch-ua"]="\!"Not)A;Brand\!";v=\!"8\!", \!"Chromium\!";v=\!"138\!", \!"Google Chrome\!";v=\!"138\!"";
  h["sec-ch-ua-mobile"]="?0";
  h["sec-ch-ua-platform"]="\!"Windows\!"";
  h["sec-fetch-dest"]="document";
  h["sec-fetch-mode"]="navigate";
  h["sec-fetch-site"]="same-origin";
  h["sec-fetch-user"]="?1";
  h["upgrade-insecure-requests"]="1";
  h["user-agent"]="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/138.0.0.0 Safari/537.36";
try(rs=New HTTP Request(URL(u),Method("GET"),Headers(h)));try(rr=rs<<Send);  a=length(rr);txt=(Blob To Char(rr));

回复： How can the download of this be completed?

Craige_Hales — Wed, 23 Jul 2025 03:04:20 GMT

hard to say for certain, but you don't want to hard code the token.
Generally, when you're logged into a website that's using that csrf token, you get a fresh one with each web page that contains a form that you might fill out. Depending on the website technology, the token might be part of the headers on the request, or it might be embedded in the form data that sent as a post. You appear to be doing a get rather than a post, so putting it in the headers makes sense. If you haven't updated it, you're probably using one. That's a day or so old and well expired.

回复： How can the download of this be completed?

lala — Wed, 23 Jul 2025 03:43:35 GMT

Thank Craige!

Then I'll still handle it by calling the python code with JSL

The ChromiumPage library in python can automatically obtain complete packet capture data and smoothly download web page content.

回复： How can the download of this be completed?

bryan_boone — Thu, 24 Jul 2025 01:05:48 GMT

Headers can be either.

If it's an associative array, it does the

key: value

for you.

But since the keys don't have to be unique in a header argument, you can also use a list with the

key: value

as an item in the list.

回复： How can the download of this be completed?

bryan_boone — Thu, 24 Jul 2025 01:08:04 GMT

You generally don't use client side cookies either. Cookies are supplied by the server.

回复： How can the download of this be completed?

bryan_boone — Thu, 24 Jul 2025 01:15:10 GMT

request = New HTTP Request(
	URL( "https://cn.govopendata.com/xinwenlianbo/20250422/" ),
	Method( "GET" )
);
html = request << download("$TEMP/govopendata.html");
Web(html);

This will download and open the the webpage

回复： How can the download of this be completed?

lala — Thu, 24 Jul 2025 02:14:10 GMT

Thanks Experts!

GET https://cn.govopendata.com/xinwenlianbo/20250422/
schannel: next InitializeSecurityContext failed: CRYPT_E_REVOCATION_OFFLINE (0x80092013)

回复： How can the download of this be completed?

lala — Thu, 24 Jul 2025 02:17:57 GMT

But this website address can be opened in a browser

回复： How can the download of this be completed?

bryan_boone — Thu, 24 Jul 2025 18:28:42 GMT

Looks like you have cert issues, Certs are supposed to be handled by a server, and not client, tho some companies _do_ install local certificates.

For that error, you can do:

request = New HTTP Request(
	URL( "https://cn.govopendata.com/xinwenlianbo/20250422/" ),
	Method( "GET" ),
       Verify SSL( "false" )
);
html = request << download("$TEMP/govopendata.html");
Web(html);

This is the equivalent of curl's --ssl-no-revoke option