-
Hi radutoev,
of course, you can do with found sites and all corresponding data whatever you want.
Please take a look at the example-script in the package (or here: http://phpcrawl.cuab.de/example.html).
From line 14 (inside the handlePageData-function), you can access all data delivered by the crawler.
So, if you want to put the content of a found site to a db-table, simply do a...
2009-11-04 13:32:39 UTC by huni
-
Hi,
I am new to the concept of web crawling, so sorry if my question is stupid. <br />
I want to know if there is a way to redirect the content received from websites crawled with PHPCrawl to a mysql database for later usage. <br />Any help would be appreciated. Thanks.
2009-11-02 11:41:51 UTC by radutoev
-
Hi strgg,
the testinteface doesn't store anything anywhere.
It is just a little "tool" to watch the behaviour of the crawler with different settings and to help you to find the right crawler-setup for your needs by that.
That's what the docs say btw.:
"Since PHPCrawl version 0.7, a visual HTML-testinterface comes along with the package. It was build to test the crawler,
2009-11-02 00:23:13 UTC by huni
-
hi and thx for your efforts,
great class
by using testinterface i checked all boxes in OUTPUT but the content(i mean content/source of website i crawl) is not stored. How do i refer to the CONTENT?????
I mean the main-use for the class should be to get the content and not some protocoll-data so im wondering why there is no introduction to this.
thanks and a very nice day.
2009-11-01 10:34:49 UTC by strgg
-
Hi aluvian,
i'm sorry, but POSTING data is currently not supported.
You could try to pass the fields via GET-parameters, like $crawler->setURL("site.php?formfield1=foo&formfield2=bar");
Maybe the site you want to crawl will accept GET-params.
2009-10-14 11:05:11 UTC by huni
-
For authentication and for crawling a site I need to send form data. From web developer toolbar for Firefox I could retrieve the names of the form fields. How can I send them using phpcrawl?.
2009-10-12 17:01:58 UTC by aluvian
-
it works for me - perhaps also for you
change the line #150 in phpcrawlerpagerequest.class.php in:
$s = @fsockopen ("PROXYURL", "PROXYPORT", $e, $t, $this->socket_mean_timeout);
with your values in place of the capitalized letters.
2009-07-28 12:39:14 UTC by radatz
-
yes in the link AND in the Site Header!
2009-07-22 17:46:49 UTC by nobody
-
Hi,
sorry, but proxy-support is currently not implemented in phpcrawl.
2009-07-19 13:04:46 UTC by huni
-
how to set proxy in phpcrawl??
i want to connect through a proxy server that requires authentication.
2009-07-11 10:58:53 UTC by kfarvandi
