But as I see by
System.out.println(parser.parse(null).toHtml());
the opened page shows this message: "Your browser's cookie functionality is turned off. Please turn it on".
What Can I do?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
If I delete "parser.getConnectionManager().parseCookies(connection)" then nothing change.
But If I delete "parser.setConnection(connection)" then the command "parser.parse(null).toHtml()" return a empty string ( I suppose that it doesn't retrieve the page).
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
ConnectionMonitor monitor = new ConnectionMonitor() {
public void preConnect(final HttpURLConnection connection) {
System.out.println(HttpHeader.getRequestHeader(connection));
}
public void postConnect(HttpURLConnection connection) {
System.out.println(HttpHeader.getResponseHeader(connection));
}
};
manager.setMonitor(monitor);
Parser parser = new Parser("http://www.gmail.com");
It's very strange the output I obtain(it follows), as you can read the expiration date of the cookie is set at "01 Jan 1990". I think that cookies enabling doesn't work for this reason:
START OUTPUT
GET http://www.gmail.com HTTP/1.1
Accept-Encoding: gzip, deflate
User-Agent: HTMLParser/1.6
I resolved the problem, I add this lines to my code:
//first, out of class
import java.io.*;
import java.net.*;
import java.util.*;
...........
//First all, but inside the class
CookieManager cookieManager = new CookieManager();
cookieManager.setCookiePolicy(CookiePolicy.ACCEPT_ALL);
CookieHandler.setDefault(cookieManager);
Now it works, perfectly.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I need to open a page that needs authentication and cookies enabled but I fail to do it.
These are steps I follow (I encrypted email and password):
Parser parser = new Parser();
parser.getConnectionManager().setRedirectionProcessingEnabled(true);
parser.getConnectionManager().setCookieProcessingEnabled(true);
URLConnection connection = parser.getConnectionManager().openConnection("https://www.google.com/accounts/LoginAuth?Email=xxxxxxxxxxxxxx&Passwd=xxxxxxxxxxx");
parser.getConnectionManager().parseCookies(connection);
parser.setConnection(connection);
But as I see by
System.out.println(parser.parse(null).toHtml());
the opened page shows this message: "Your browser's cookie functionality is turned off. Please turn it on".
What Can I do?
These two steps should not be needed...
parser.getConnectionManager().parseCookies(connection);
parser.setConnection(connection);
If I delete "parser.getConnectionManager().parseCookies(connection)" then nothing change.
But If I delete "parser.setConnection(connection)" then the command "parser.parse(null).toHtml()" return a empty string ( I suppose that it doesn't retrieve the page).
my bad,
I didn't look closely enough.
The call:
URLConnection connection = parser.getConnectionManager().openConnection("https://www.google.com/accounts/LoginAuth?Email=xxxxxxxxxxxxxx&Passwd=xxxxxxxxxxx");
should be:
parser.setResource ("https://www.googl...")
I've modified my source and now It looks so:
ConnectionManager manager = Parser.getConnectionManager();
manager.setCookieProcessingEnabled(true);
ConnectionMonitor monitor = new ConnectionMonitor() {
public void preConnect(final HttpURLConnection connection) {
System.out.println(HttpHeader.getRequestHeader(connection));
}
public void postConnect(HttpURLConnection connection) {
System.out.println(HttpHeader.getResponseHeader(connection));
}
};
manager.setMonitor(monitor);
Parser parser = new Parser("http://www.gmail.com");
It's very strange the output I obtain(it follows), as you can read the expiration date of the cookie is set at "01 Jan 1990". I think that cookies enabling doesn't work for this reason:
START OUTPUT
GET http://www.gmail.com HTTP/1.1
Accept-Encoding: gzip, deflate
User-Agent: HTMLParser/1.6
HTTP/1.1 302 Moved Temporarily
Set-Cookie: GV=EXPIRED;Domain=mail.google.com;Path=/;Expires=Mon, 01-Jan-1990 00:00:00 GMT
Set-Cookie: GV=EXPIRED;Domain=mail.google.com;Path=/mail;Expires=Mon, 01-Jan-1990 00:00:00 GMT
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: Fri, 01 Jan 1990 00:00:00 GMT
Date: Sat, 20 Dec 2008 22:39:45 GMT
Location: https://www.google.com/accounts/ServiceLogin?service=mail&passive=true&rm=false&continue=http%3A%2F%2Fmail.google.com%2Fmail%2F%3Fui%3Dhtml%26zy%3Dl&bsv=1k96igf4806cy<mpl=default<mplcache=2
Content-Type: text/html; charset=UTF-8
X-Content-Type-Options: nosniff
Transfer-Encoding: chunked
Server: GFE/1.3
GET https://www.google.com/accounts/ServiceLogin?service=mail&passive=true&rm=false&continue=http%3A%2F%2Fmail.google.com%2Fmail%2F%3Fui%3Dhtml%26zy%3Dl&bsv=1k96igf4806cy<mpl=default<mplcache=2 HTTP/1.1
Accept-Encoding: gzip, deflate
User-Agent: HTMLParser/1.6
HTTP/1.1 200 OK
Content-Type: text/html; charset=UTF-8
Cache-control: no-cache, no-store
Pragma: no-cache
Expires: Mon, 01-Jan-1990 00:00:00 GMT
Date: Sat, 20 Dec 2008 22:39:48 GMT
X-Content-Type-Options: nosniff
Transfer-Encoding: chunked
Server: GFE/1.3
END OUTPUT
I resolved the problem, I add this lines to my code:
//first, out of class
import java.io.*;
import java.net.*;
import java.util.*;
...........
//First all, but inside the class
CookieManager cookieManager = new CookieManager();
cookieManager.setCookiePolicy(CookiePolicy.ACCEPT_ALL);
CookieHandler.setDefault(cookieManager);
Now it works, perfectly.