Re: Playback of HTTP access logs

[Val S. <va...@nv...>]

Ron,
  curl-loader can be used for a large number of URLs.  The right way to do it is 
to use a small config file that references a file with the URLs as a tokens 
file.  Grep the Readme for tokens, you'll see what I mean.  The space for those 
URLs is going to be allocated upfront, so the limit is the amount of memory you 
have on the machine.
/Val




________________________________
From: Ron Parker <rp...@mo...>
To: "cur...@li..." 
<cur...@li...>
Sent: Wed, July 28, 2010 3:57:29 PM
Subject: Playback of HTTP access logs

 
Hi,
 
I need to obtain or build a tool that can issue HTTP requests based on the 
contents of an http access log file.   I’m assuming, of course, that I would 
need to translate my proprietary log file into a properly formatted config 
file.   The log file contains approximately 1 million HTTP requests (GET/POST) 
spread across approximately 30,000 clients.   I don’t know the distribution of 
the URLs, but lets say there are 200,000 unique URLs.
 
While the curl-loader tool looks very interesting, all of the examples are built 
around a small number of URLs that are cycled over a large number of clients.  
Can curl-loader be used to support my requirements?   What is the maximum number 
of URLs that may be contained in a config file (theoretical and tested)?
 
If curl-loader is not the right tool for this application, do you have any other 
suggestions?
 
Thanks, in advance.
 
Ron Parker
Movik Networks