How to connect url with port number in java with Jsoup?

최동규

I am trying to get HTML in URL(.

Document doc = Jsoup.connect("URL").userAgent("Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36").timeout(111110).followRedirects(true).execute().parse();
System.out.println(doc.text());

I did userAgent by Java(Jsoup): How to parse http://host:port But It does not work.

How should I connect that URL?

Alexandr Lihonosov

You get the error because this URL under Cloudflare DDoS protection. If you open this URL in a browser you will see that first you got the 503 response code and after that, you will be redirected to the target URL. If you want you can set ignoreHttpErrors to true and you will see that with your code.

~ ➭ http http://www.kg348.com:8880/play
HTTP/1.1 503 Service Temporarily Unavailable
Cache-Control: no-cache
Connection: close
Content-Type: text/html; charset=UTF-8
Date: Sat, 15 Jul 2017 14:58:52 GMT
Refresh: 8;URL=/cdn-cgi/l/chk_jschl?pass=sdfsdfsdf.127-dsfsdfd
Server: cloudflare-nginx
Set-Cookie: __cfduid=xxxx; expires=Sun, 15-Jul-18 14:58:52 GMT; path=/; domain=.kg348.com; HttpOnly
Transfer-Encoding: chunked
X-Frame-Options: SAMEORIGIN

And looks like this redirect has been done by javascript, but Jsoup is a parser, it doesn't include a javascript execution engine, so I'm not sure that you are able to get html code of target URL only with Jsoup. In order to execute javascript you will have to use something like selenium webdriver or alternative framework.

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

Java(Jsoup): How to parse http://host:port

JSoup cant connect to URL

Remove port number from a url in java

JSOUP URL connect errors in coldfusion

Java JSoup: Modifying URL

Using Jsoup to connect to url, but Jsoup call another url. Why?

How to find Metadata URL of IMS Database to connect from Java when I have host name, port and Datastore name?

How can remove the port number from live vue site url?

How to get the port number from the URL using jQuery?

How to remove the :port number from the end of an IIS URL?

How to get Base URL with port number in codeigniter 3.0.3

How to set a page within a folder to the URL localhost: port-number?

How to tell java application to connect to specific Axon server gPRC port?

Java JSoup error fetching URL

How to connect to TLS 1.2 enabled URL with Java

Remove port number in image url?

Redirect to a port number depending on the URL

Mirth Connect to send file to a port number

How to connect via HTTPS using Jsoup?

How can we call url() with port number in Laravel controller. Node server is running on different port that 8001 and Laravel port is 8000?

How to set the port number for the client in udp (client ,server) program in java?

How to find the process id of a process running on a port number in java

How do X clients know that they will need to connect to TCP port 6000+<display number>?

How to change Host (IP address) and port number to connect to wifi internet in Xubuntu 16.04?

How can I connect to vscode's built-in tsserver? What is the port number of vscode tsserver?

How to connect to a certain port on SSH

Redirect traffic to a domain with a port number without showing the port number in the URL

Jsoup Java Html scraping cannot get number

Problem connecting to a specific URL with Jsoup's connect method