Re: HTTP request returns html instead of json

dk · Jul 16, 2024 5:29 AM

hello

I tried to request JSON from an API using the following script:

request_neu = New HTTP Request(
	URL( "https://redacted.com/api/path?fileType=Json" ),
	Method( "GET" ),
	//headers( {"Accept: text/plain", "Accept: text/html", "Accept: */*"} ),
	//headers( {"Accept: */*"} ),
	//I tried everything here
	Headers( {"Accept: application/json"} ),
	Username( "api.user" ),
	Password( "redacted" ),
	Timeout( 900000 )
);

response = request_neu << send;
show(request_neu << get mime type);
show(request_neu << get status);
show(request_neu << get status message);
show(substr(response,1,999));
JSON to data table(response);

I added the debugging statements after the JSON to data table function resulted in an error and the output is

request_neu << get mime type = "text/html; charset=utf-8";
request_neu << get status = 200;
request_neu << get status message = "HTTP/1.1 200 OK";
Substr(response, 1, 999) = "
<!DOCTYPE html>
<html class=\!"no-js smooth-scrolling-html\!" lang=\!"en-GB\!" dir=\!"ltr\!">
<head>
etc

so essentially instead of JSON I receive html.

The interesting thing is that it works with Postman, Python, even Excel.

The really interesting thing is that some months ago when I created this script it also was working.

Do you have some idea what is going on here?

thx

jthi · Jul 16, 2024 6:06 AM

Have you tried to see what the HTML contains? You could for example save it as .html page using Save Text File()

I don't quickly see anything wrong in your syntax, below is slightly modified example from scripting index

Names Default To Here(1);

request = New HTTP Request(
	url("http://httpbin.org/get"),
	Method("GET"),
	Headers({"Accept: application/json"}),
	Username("ross"),
	Password("Abc123"), 

);
data = request << Send;
Show(request << get response headers, request << get last url);

-Jarmo

dk · Jul 16, 2024 6:32 AM

When I looked at the html, it looks similar to the platform's web page except that the relative links are broken.

I am afraid that I will need to discuss with the people responsible for the platform since the mistake seems to originate from there.

bryan_boone · Jul 29, 2024 2:59 PM

"Accept: application/json" is just a hint to the web service as to the type of response you want returned.

Web services are completely free to ignore it.

Is this an external web service or one developed "in house" if "in house", then you'll want to talk to the developer about returning json data when requested by a client.

If external, can you post the api doc?