Search Results for "python web crawler"

Apache Log Parser and Data Normalization Application Python handles File Processing & MySQL handles Data Processing ApacheLogs2MySQL consists of two Python Modules & one MySQL Schema to automate importing Access & Error files and normalizing data into database designed for reports & data analysis. Runs on Windows, Linux and MacOS & tested with MySQL versions 8.0.39, 8.4.3, 9.0.0 & 9.1.0. 4 LogFormats & 2 ErrorLogFormats can be loaded and 5 MySQL Stored Procedures can be processed in a single Python `ProcessLogs function` execution. ...

OSSIM, AlienVault’s Open Source Security Information and Event Management (SIEM) product, provides event collection, normalization and correlation. For more advanced functionality, AlienVault Unified Security Management (USM) builds on OSSIM with these additional capabilities: * Log management * Advanced threat detection with a continuously updated library of pre-built correlation rules * Actionable threat intelligence updates from AlienVault Labs Security Research Team * Rich...

This is a web log file analyzer we are making using python. First the IIS parsing engine wil be built and then Apache and possibly other servers. It is going to support multiple log files from any date and output the statistics in html formatted files, incorporating automatically build charts. It will be a pure python solution which is going to be self contained, ie no installation will be required other from the standard python modules.

Use python to parse privoxy log in clf format and refer to wot ratings of the site and then dump the data in a sqlite database for statistics generation later.

Cislog is a syslog server that specifically targets Cisco devices and their implementation of the syslog protocol. It consists of a Django web interface and a syslog listener with a database between them.

This is a Python script to parse your irssi logs and input them into a MySQL database which you can then use to search and display your logs on the web. It incrementally updates the database from the logs and is ideally run as a cronjob often.

LogCrawler is an ANT task for automatic testing of web applications. Using a HTTP crawler it visits all pages of a website and checks the server logfiles for errors. Use it as a "smoketest" with your CI system like CruiseControl.

process runtime detection with xml/image statistic output - like xfire .com

This project is a Python-based HTTP web proxy server that hooks into MySQL to store a full history of your browsing. Allows you to check out statistics about your browsing habits. Creates a personal portal page, has search features, multi-user, filters.

Find out where your website visitors are from: generates statistics on hits per top level domain (e.g. .com, .org, .uk, .se, .fr). Currently works with apache log format, but can easily be extended to understand other formats. Written in vanilla Python.

PyLogAlyzer is a Web Log Analyzer in Pure Python (a clone of Awstats). PyLogAlyzer produces a XML result and uses XSLT to generate the HTML files.