Apache Log Parser and Data Normalization Application

Python handles File Processing & MySQL handles Data Processing

ApacheLogs2MySQL consists of two Python Modules & one MySQL Schema to automate importing Access & Error files and normalizing data into database designed for reports & data analysis.

Runs on Windows, Linux and MacOS & tested with MySQL versions 8.0.39, 8.4.3, 9.0.0 & 9.1.0.

4 LogFormats & 2 ErrorLogFormats can be loaded and 5 MySQL Stored Procedures can be processed in a single Python `ProcessLogs function` execution.

Database system designed to accommodate unlimited domains. Step-by-step guide for easy installation.

Web interface with Drill Down Capability and apache/echarts Log Visualization integration in development.

MySQL apache_logs schema currently has 49 Tables, 853 Columns, 168 Indexes, 66 Views, 7 Stored Procedures and 43 Functions to process Apache Access log in 4 formats & Apache Error log in 2 formats. Database normalization at work!

Features

  • Creates MySQL schema - apache_logs
  • Includes 2 Python modules
  • Imports Apache Access Logs - common, combined and vhost_combined
  • Imports Apache Error Logs
  • Complete Apache Log Data Normalization
  • MySQL does all Data Manipulation & Processing
  • Python handles polling of log file folders
  • Python executes MySQL Database LOAD DATA statements
  • Python executes Stored Procedures & Functions and SQL Statements
  • Requires installation of 4 Python modules - requirements.txt included

Project Samples

Project Activity

See All Activity >

License

Apache License V2.0

Follow apache-logs-to-mysql

apache-logs-to-mysql Web Site

Other Useful Business Software
Crowdtesting That Delivers | Testeum Icon
Crowdtesting That Delivers | Testeum

Unfixed bugs delaying your launch? Test with real users globally – check it out for free, results in days.

Testeum connects your software, app, or website to a worldwide network of testers, delivering detailed feedback in under 48 hours. Ensure functionality and refine UX on real devices, all at a fraction of traditional costs. Trusted by startups and enterprises alike, our platform streamlines quality assurance with actionable insights.
Click to perfect your product now.
Rate This Project
Login To Rate This Project

User Reviews

Be the first to post a review of apache-logs-to-mysql!

Additional Project Details

Operating Systems

Linux, Mac, Windows

Intended Audience

Developers, Information Technology, Management, System Administrators

User Interface

Command-line, Console/Terminal

Programming Language

Python

Database Environment

Python Database API, SQL-based

Related Categories

Python Log Analysis Software, Python Log Management Tool, Python Log Parsers

Registered

2024-11-19