Menu

#20 Allow redirect by single iframe / frameset on page

open
nobody
5
2013-05-29
2013-05-29
MadEgg
No

As discussed in: http://sourceforge.net/p/phpcrawl/discussion/307696/thread/6bf703b0/

Several websites I'm trying to crawl use multiple URL's for their website. On some of those, the page is actually a HTML page that contains an iframe or a frameset which loads the content from the actual URL. Since I don't want to crawl unrelated websites, I disabled cross-domain crawling. However, in the cases where the first content retrieved only contains a single frameset or iframe (except other required html, such as html, head, body tags, etc), this URL should be followed.

Discussion

Anonymous
Anonymous

Add attachments
Cancel





Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.