I am trying to get HTML in URL(.
Document doc = Jsoup.connect("URL").userAgent("Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36").timeout(111110).followRedirects(true).execute().parse();
System.out.println(doc.text());
I did userAgent by Java(Jsoup): How to parse http://host:port But It does not work.
How should I connect that URL?
You get the error because this URL under Cloudflare DDoS protection. If you open this URL in a browser you will see that first you got the 503 response code and after that, you will be redirected to the target URL. If you want you can set ignoreHttpErrors to true and you will see that with your code.
~ ➭ http http://www.kg348.com:8880/play
HTTP/1.1 503 Service Temporarily Unavailable
Cache-Control: no-cache
Connection: close
Content-Type: text/html; charset=UTF-8
Date: Sat, 15 Jul 2017 14:58:52 GMT
Refresh: 8;URL=/cdn-cgi/l/chk_jschl?pass=sdfsdfsdf.127-dsfsdfd
Server: cloudflare-nginx
Set-Cookie: __cfduid=xxxx; expires=Sun, 15-Jul-18 14:58:52 GMT; path=/; domain=.kg348.com; HttpOnly
Transfer-Encoding: chunked
X-Frame-Options: SAMEORIGIN
And looks like this redirect has been done by javascript, but Jsoup is a parser, it doesn't include a javascript execution engine, so I'm not sure that you are able to get html code of target URL only with Jsoup. In order to execute javascript you will have to use something like selenium webdriver or alternative framework.
Collected from the Internet
Please contact [email protected] to delete if infringement.
Comments