Problems with sites using a seperate site for cookie-setting
Status: Beta
Brought to you by:
huni
Some webistes use a sepearate site for setting a session-cookie (like "http://www.bobbyberkhome.com").
For the first request a redirect to another site is responded. This other site sets a session-cookie and then redirects BACK to
the entry page. But the crawler already requested this site and doesn't request it again and sthe crawling-process stops.
Anonymous
Another example-page:
http://management-ressources.wikispaces.com