topic Re: 请教专家:怎样解决下载这个网站数据的乱码问题？ in Discussions

请教专家:怎样解决下载这个网站数据的乱码问题？

lala — Fri, 09 Jun 2023 18:06:11 GMT

我按以下脚本下载这个网站数据、得到的结果是乱码。

虽然我已添加了语言编码。

谢谢！

u="http://basic.10jqka.com.cn/600000/equity.html";d1=Open(u,HTML Table(3,Column Names(1),Data Starts(2)),Charset("gb2312"));

Re: 请教专家:怎样解决下载这个网站数据的乱码问题？

Craige_Hales — Sun, 28 Nov 2021 04:31:28 GMT

try something like this

u="https://www.w3schools.com/html/html_tables.asp";
blobdata = loadtextfile(u,blob);
//blobdata=blobpeek(blobdata,0,47939)||hextoblob("20CDE2")||blobpeek(blobdata,47939);// insert 外 https://en.wikipedia.org/wiki/GB_2312
chardata = blobtochar(blobdata,"gb2312");
filename = savetextfile("$temp/deleteme.html",chardata);
d1=Open(filename,HTML Table(1));

Download the data as a blob.

Convert the blob to character data using the GB2312 encoding.

Save the character data; it will be UTF8 encoding on disk.

Open the saved file.

the blobpeek offset was chosen very carefully to but the character here.

Re: 请教专家:怎样解决下载这个网站数据的乱码问题？

lala — Sun, 28 Nov 2021 05:08:33 GMT

感谢专家!

我试着依照修改、但不能这样下载

Re: 请教专家:怎样解决下载这个网站数据的乱码问题？

lala — Sun, 28 Nov 2021 05:09:44 GMT

u="http://basic.10jqka.com.cn/600000/equity.html";

txt = loadtextfile(u,blob);

它不工作。谢谢!

Re: 请教专家:怎样解决下载这个网站数据的乱码问题？

Craige_Hales — Sun, 28 Nov 2021 07:43:52 GMT

There might be a translation problem.

I think you are saying it did not work.

I am guessing it needs a user name and password.

If I am wrong, you'll need to give me more information.

Here's how you can supply a basic auth password.

You might need to use https instead of http.

//u = "http://basic.10jqka.com.cn/600000/equity.html";
u = "https://www.w3schools.com/html/html_tables.asp";

blobdata = New HTTP Request(
	URL( u ),
	Method( "GET" ),
	Username( "ross" ),
	Password( "Abc123" )
) << Send( "blob" );
	
chardata = Blob To Char( blobdata, "gb2312" );
filename = Save Text File( "$temp/deleteme.html", chardata );
d1 = Open( filename, HTML Table( 1 ) );

Re: 请教专家:怎样解决下载这个网站数据的乱码问题？

lala — Sun, 28 Nov 2021 10:34:50 GMT

感谢专家的帮助!

这个网站是不需要登录的免费网站。

我将它的"网页源代码"截图

Re: 请教专家:怎样解决下载这个网站数据的乱码问题？

lala — Sun, 28 Nov 2021 10:37:30 GMT

奇怪的是这个网站在Chrome中用F12中查看、已经是乱码

Re: 请教专家:怎样解决下载这个网站数据的乱码问题？

lala — Sun, 28 Nov 2021 10:38:58 GMT

希望我的帖子中含有英文字母也能正确翻译。

感谢社区的各位专家!

Re: 请教专家:怎样解决下载这个网站数据的乱码问题？

蘇71 — Mon, 29 Nov 2021 19:07:59 GMT

实际上，Craige_Hales已经给出了答案，在他的基础上，稍作修改如下即可。

Actually, Craige_Hales has already worked it out. The followed code based on his work.

u="http://basic.10jqka.com.cn/600000/equity.html";
blobdata = loadtextfile(u,blob);
chardata = blobtochar(blobdata,"gb2312");
filename = savetextfile("$temp/deleteme.html",chardata);
d1=Open(filename,HTML Table(3));

此外，你也可以在导入时选择从internet导入，打开方式选网页，然后选择需要导出的数据表，也能正常显示。

Besides,Selecting File > Internet Open > Web Page could do this too, just set Open As to Web Page instead of Data.

Re: 请教专家:怎样解决下载这个网站数据的乱码问题？

Craige_Hales — Sun, 28 Nov 2021 17:38:30 GMT

I'm not sure.

Part of the time I get "forbidden" as a result (missing cookie, and maybe missing user-agent might be how the server decides to forbid access.)

I think the web site may be checking to see if an interactive browser with JavaScript enabled is in use.

JMP does not run JavaScript on web pages.

You might want to (1) check for their acceptable use policy, (2) see if they have a rest api for accessing the data.

Screen-scraping this site will be hard to maintain.

Re: 请教专家:怎样解决下载这个网站数据的乱码问题？

lala — Mon, 29 Nov 2021 00:32:12 GMT

感谢各位专家!

奇怪、今天试同样的代码、它能正常了、但结果还是乱码

Re: 请教专家:怎样解决下载这个网站数据的乱码问题？

lala — Tue, 30 Nov 2021 09:46:37 GMT

只能在下载后进一步转码

Re: 请教专家:怎样解决下载这个网站数据的乱码问题？

lala — Tue, 30 Nov 2021 09:53:34 GMT

请教专家:这个转码代码怎样能修改为可用的脚本？谢谢！

Function BytesToBstr(strBody, CodeBase)
Dim objstream
On Error Resume Next
Set objstream = CreateObject("Adodb. Stream")
With objstream
.Type = 1
.Mode = 3
.Open
.Write strBody
.Position = 0
.Type = 2
.Charset = CodeBase
BytesToBstr = .ReadText
End With
obj stream.Close
Set objstream = Nothing
If Err.Number <> 0 Then BytesToBstr = ""
On Error GoTo 0
End Function

Re: 请教专家:怎样解决下载这个网站数据的乱码问题？

Craige_Hales — Tue, 30 Nov 2021 13:18:10 GMT

I don't know what that (visual basic?) code does. It looks similar to JMP's HttpRequest, and similar to earlier code in this thread that used blobToChar to do the conversion.

Re: 请教专家:怎样解决下载这个网站数据的乱码问题？

lala — Tue, 30 Nov 2021 22:53:05 GMT

这个代码是将乱码转化为中文。幸好这些乱码的文字只有几个、我准备用手工操作来替换了。

好的、非常感谢专家!