extending log files generates error

Help
2008-05-27
2013-05-23
  • T Rasmussen
    T Rasmussen
    2008-05-27

    Hi.

    Thanks for a great app.

    Actually so great it inspired me to play around with it. But I have run into some problems...

    I want to add a few filters, such as nationality, ip, searchterms a.s.o. All of this I do by javascript and a modified click.php. This part works fine though it changes my datasets to look like...

    990|138|1003|msie|0|german|\"216.245.0.21\"|\"isearchedforthis\"

    instead of

    990|138|1003|msie|0

    However clickheat (utilizing default heatmapFromClicks.php) is not able to read the log-entries if I extend the individual entries like above. However If I in a file have both normal and extended entries it will read out the normal ebtries fine.

    I suspect the logic for this is to be found in heatmapFromClick.class.php.
    In this file i changed...

    if (count($click) === 3)

    to

    if (count($click) > 0)

    ..but still with no luck.

    It seems to be the base class in heatmap.class which fires the LANG_ERROR_DATA due to $image being 0, but I still suspect it is the parsing of entries in heatmapformClicks.class.php that fails.

    Any ideas of what my problem consists of and a possible solution is much welcome.

    Best Regards

    Trolle

     
    • Blade 1o6
      Blade 1o6
      2008-05-27

      Hello Trolle,

      You're right about the HeatmapFromClicks, that's where the data is prepared, so that's where the problem is.

      The count($click) === 3 should not be changed (line 63). This is a test on temporary files as the process is:
      - read all click logs
      - test if clicks are okay to be drawn (that's where the filters are tested)
      - if the click isn't in the current image (so you're on the first part of the heatmap, and a click should be drawn below), then save it in a temporary file. This is to avoid reading logs for every part of the heatmap, and re-run tests already done.

      So the line 63 is a test on «do we have data for the part 2 or higher, and is there X, Y and button clicked (3 data)?».

      Then, your test would take place after this test, line 134:
      /** Do a regular match (faster and easier for large volume of data) */
      preg_match_all('~^(\d+)\|(\d+)\|(\d+)\|'.($this->browser === 'all' ? '[a-z]+' : $this->browser).'\|(\d)$~m', $buffer, $clicks);

      So you may have something like:
      preg_match_all('~^(\d+)\|(\d+)\|(\d+)\|'.($this->browser === 'all' ? '[a-z]+' : $this->browser).'\|(\d)\|([a-z]+)\|"(\d+\.\d+\.\d+\.\d+)"\|"(.*?)"$~m', $buffer, $clicks);

      I've matched «german» as [a-z]+, don't know if it's okay or not.

      Then you can add your tests near line 140 ($x = (int) $clicks[1][$i]; // X), in the FOR loop:
      if ($clicks[5] !== 'german')
      {
           continue;
      }
      (this example ignores the click if the 5th parameter isn't «german»)

      Does this help you?

      Best regards,
      Yvan.

       
    • T Rasmussen
      T Rasmussen
      2008-05-27

      Hi Yvan

      Great quick response!

      I modified the code with the preg_match and the check.

      However main heatmap class still throws and "No logs for the selected..." error. But if I use the original preg_match and only have the original 4 variables in my log, it works again.

      Maybe a nice solution would be the build a small pregmatch_builder function, which lets you add variables on the fly, maybe something like...

      function preg_match_builder (roginal_$allmatches,variablename, number in $clicks,possiblewrapper,check) {

      //check is int, varchar, email a.s.o.
      //adds the variable to the pregmatchstatement based on variables, wrappers a.s.o
      $allmatches = $allmatches.$newmatch;
      return $allmatches;
      }
      and replace original preg_match([...]) with preg_match($allmatches)
      A function like this would be extremely usefull not only for peadagogical reasons, but also for parsing e.g. external logfiles a.s.o

      Best regards
      Trolle

       
      • Blade 1o6
        Blade 1o6
        2008-05-27

        Hello Trolle,

        I think that you've filtered out every click, hence the main class ends on «No logs», which you should read as «no clicks».

        Adding a function may be complicated, and useful for a very few people (in fact you're the first to ask for a new filter).

        For the fact that you may have removed every click, please add an echo() about line 256 (near «$this->maxY = max($y, $this->maxY);») to see if your clicks are added or not, and another one at the beginning of the FOR loop, near line 140.

        I think we'll remove file support in the future, sticking with MySQL support. Then perhaps we'll add some possible other parameters, but that's hard to maintain from one release to the other, and most of the time this implies many slowdowns.

        Best regards,
        Yvan.

         

  • Anonymous
    2012-09-15

    Ive just finished with adding more filters, i actually tested first with browser and its working like a charm. Just a hint  if anyone else gets stuck. Once you extend log file, you will need to delete all logs before that or they will keep showing errors like the one from above.

    And here is in short how you add new filters / logs.

    1.  clickheat-original.js - here you define new variable that you will get, for example OS version.
    2.  THrought ajax, your var will get to click.php. Now in click.php through GET you get variable and write it into your log file, which is fputs function at the bottom.
    3. Next you add in index.php array of every possible value for your new variable. Check how they did it for Browser version.
    4.  Now add it into html code in view.php, for me it was in options screen.
    5. Go to admin.js and find updateheatmap() function. Add your new var there so it can be passed to the generate.php
    6. In generate.php get that variable through GET. Also find $imagePath and add this new variable here so that log files are with right names. Also in generate.php find $clicksHeatmap = new HeatmapFromClicks(); at the bottom and add this new var, for example $clicksHeatmap->os = $os;
    7. Last part is preg_match_all in heatmapfromclicks.class.php . With this functin you read log file. This is mine:

    preg_match_all('~^(\d+)\|(\d+)\|(\d+)\|'.($this->browser === 'all' ? '+' : $this->browser).'\|(\d)\|'.($this->os === 'all' ? '+' : $this->os).'$~m', $buffer, $clicks);

    Anyway, I am jquery/html/css expert and got basic knowledge with php, and i managed to do it so i suppose that its not really hard. Just google. My next step is to convert to mysql instead of log files. I am not sure if this is hard. I know its possible, just question is how hard it is.

    Oh yes, ive also managed to color click map with different color for every browser.  Something like this http://www.crazyegg.com/demo#confetti