Why SolrLog4j?

If you've ever had to pour over log files looking for a literal needle in a haystack, you already know the answer to the question posed by the title of this blog. The Logging Layer, while a seeminly innocuous or mundane part of an enterprise application, is actually one of it's most important layers. It's rarely, if ever, seen by users, marketers, business analysts, corporate executives, and so on. Nevertheless, it remains a critical feature that if implemented wisely makes debugging issues (and there are ALWAYS issues to debug) much much simpler. Further, it makes the log analysis project far more powerful. In the past, custom scripts are generally crafted to parse massive log files line by line and sift out relevant data. This process is cumbersome, at best.

While logging facades like Log4J and SFL4J have provided powerful frameworks for creating logs, there really is no suitable, pluggable framework with which to utlitize the data gathered in the logs.

It is with this in mind that SolrLog4j was created.

In all candor, this marriage should have happened a long time ago. There is a natural synergy between these logging facades and the big data engine that is Solr. Logging tends to be an afterthought in a development effort. At best, there are guidlines about what logging framework should be used, and possibly when. In most cases, though, logging is left up entirely to the developer or engineering team. In some cases, certain pet peeves will override this, and logging will not be allowed at all! Of course, when the day comes and a bug arises, suddenly logging is the most important thing ever -- at least to the people charged with fixing the issue.

My personal philosophy goes back to a team I worked with many years ago, and it quite simple:

Log Everything.

Some will disagree, of course, but more often than not, those in protest are never the ones responsible for resolving problems. The Truth is that logging is the fastest way to identify and isolate the cause and point of origin of a bug, and without it, the process can be dauting, to say the least.

Having myself spent many an hour pouring over log files, when a collegue and I hatched the idea to create a logging framework based on Solr, well, it just seemed like a natural -- indeed, beautiful -- combingation of techologies.

And thus SolrLog4j was born.

SolrLog4j is a multi-threaded, lightweight tool that provides support for Log4j and Log4j2, with support for SFL4J not far behind. It is written in pure Java, using a minimum of libraries, most of which are already used in a Solr environment.