#162 Issues in 2.4.0
Hello!
I guess I've found couple of bugs in sarg version 2.4.0
1) Parameter -e doesn't work.
Command
sarg -d day-0 -l /var/log/squid/access.log -w /tmp -x -e stdout
prints a lot of errors like this:
shell-init: error getting current directory: getcwd: cannot access parent directories: No such file or directory
And in the end:
SARG: sort command return status 2
SARG: sort command: sort -n -T "/tmp/sarg" -t " " -r -k 2,2 -o "" "/tmp/sarg/2015Jul11-2015Jul11/top.tmp"
Without "-e", for example,
sarg -d day-0 -l /var/log/squid/access.log -w /tmp -x
everything is fine, I see a new report on a web-server, even though shell-init errors are displayed.
This behavior is not observed on version 2.3.4. With this version report can be displayed in shell or sent via email.
2) Redirector reports are displayed only if date range of report is provided explicitly.
Example:
# sarg -x -l /var/log/squid/access.log
shows no attempts to get an access to blocked sites, but
# sarg -x -l /var/log/squid/access.log -d day-0
shows extra menu "Redirector" where I can see users and blocked sites they tried to access.
Path to squidguard logs using redirector_log parameter in sarg.conf is provided correctly.
3) If parameter "usertab" in conf file is set to "ldap" and all related information to get LDAP is provided correctly, I see the real usernames instead of their logins. But in redirector log still the logins are displayed.
Thank you for your bug report.
I fixed the problems in points 1 and 2 except for the shell-init error. It is not an error coming from sarg.
A google search reveals it is not all that uncommon. It may originate from a wrong passwd file or user profile.
Sarg does start new shells. Starting a shell may cause that error but I can't reproduce it and I can't figure out how it might even be produced.
Beside, the errors don't appear when generating a html report. It is strange as it is the same code doing both reports.
Do you run sarg with another user's privileges? Something along the line:
Is there something else you might be doing differently when generating an e-mail report or a html report?
Do you have the file ~/.ssh/rc? (see the solution here http://unix.stackexchange.com/questions/196998/ssh-with-y-results-in-shell-init-error-retrieving-current-directory-getcwd for an explanation)
1) Yes, now (-e) works fine. I see reports in console and can get them by e-mail.
The problem with shell-init seems like floating. Sometimes I see it, sometimes not, even in the same ssh-session with the same parameters. (I don't have ~/.ssh directory on my system.)
So, nevermind, the result is the same anyway.
2) Now the "Redirector" menu appears even without -d option.
But when the date range is provided, redirects only for that period are shown. If date range is not provided, all redirects are shown that are in squidGuarg logfiles.
For example, access.log was created today, but redirects in report are month-aged.
3) In this point now I see how it works.
If the login in lsquidGuarg logfile is written in Kerberos style (with @domain.com suffix), then just login is displayed (but without domain suffix). If it is just login (for example, when authenticated by NTLM), then information from ldap-server is displayed.
So these logfiles need to be parsed and all domain suffixes should be removed before making a new report. I'll put it in everyday cron job.
The correction to problem 2 was not right. Thank you for testing it! I improved the patch in the latest commit. Now, the redirector log covers the same date range as the access.log if -d is not used.
As for point 3, I don't use Kerberos nor NTLM nor LDAP. I don't understand what the solution to that problem should be. Can you give me an example of what the input is and how the domain suffixes should be removed?
There was a similar request in this bug report: https://sourceforge.net/p/sarg/bugs/160/#a658 so at least one person is interested in that solution too :-)
That person was me )
So I can't say that it is a problem, but want to suggest you make the following improvement.
As you know, squid can authenticate users by using so called "helpers" - external programs which can operate with services such as LDAP or Kerberos. What do you see in access.log when a user runs a browser an types https://google.com :
Take a look at the username. This name would be put in log in case Kerberos authentication is working on the network and browser supports it (and squid is set up to use it, of course). Domain suffix is written in capital letters.
What would the log record be if Kerberos helper is not used in squid or user's browser does not support it (or any other program that requests for Internet access and can use proxy):
As you see, there is only username used, without domain suffix. This record is logged when NTLM or Basic authentication is used (and maybe some others).
So when sarg makes a report, it shows statistics for two different users: apetrov and apetrov@MYCOMPANY.RU, because, strictly saying, user fields in these records are different.
Squidguard logs redirects and puts a username according to information provided by squid:
In this case my browser authenticated by Kerberos (squidguard writes domain suffixes is in small letters). If program cannot use Kerberos or squid is not set up to use it, the domain suffix is not written. Also, redirecor report made by sarg shows two different users.
What I did. After rotating log 'sed' removes all domain suffixes from yesterday's access log file and from squidguard logfile. Also, these suffixes are removed from all redirector logs. Then report shows only one user apetrov and his total traffic (which is authenticated by Kerberos, NTLM and Basic) and his redirects in redirector report.
It would be a very nice feature, if it could be possible to specify a Kerberos domain name in conf file so sarg could remove it while reading logs.
I tried to use regular expressions for that (in file specified fo useralias), but it only changes apetrov@MYCOMPANY.RU to apetrov and shows two "different" users apetrov with two different for webpage with their accessed sites.
I hope I explained pretty clearly. If not, I can provide logs or screenshot.
Sarg now has the new strip_user_suffix option. It removes the trailing part of the user name.
To make your example work, you would have to add the following line to sarg.conf:
The suffix is case insensitive.
The suffix to remove must be found at the end of the user name.
There can be only one suffix defined. If it is defined multiple time, the last one is used.
Wildcard is not supported. The suffix string is taken as is and is removed from the user name end if it matches. If a user name doesn't end with that suffix, the user name is left unchanged.
That option is very simple and limited. Its purpose is to be easy to configure. There already is the much more powerful useralias option for the more complicated cases.
Now the latest version fail with error
SARG: Sorting file "/tmp/sarg/redirector.int_log"
SARG: Unknown user ID - in file "/tmp/sarg/download.int_log"
That was a stupid mistake that could have been easily avoided…
It is now fixed.
Thank you for testing it!
Now it's all fine!
New parameter works also. I'm very happy )
By the way, is it possible to make such a regular expression that acts like a new strip_user_suffix option? I tried some variants, but got no result.
The regular expression was in a sorry state too (sigh…).
1) It was not possible to use \1 or $1 in the alias.
2) Regex were case sensitive. It was not possible to match both @mycompany.ru and @MYCOMPANY.RU with one regex.
3) The replaced user name was prefixed with * in the report.
1) and 3) are fixed in master branch.
2) is solved by accepting a "i" after the regex to make it case insensitive.
I could emulate the strip_user_suffix option with this useralias file:
Thanks again for not accepting my claims and validating them thoroughly :-)
Great! Everything works fine! Thank you for fixing it :)
Hello! I've found one feature related to regexps in useraliases.
I have couple of configs made for reports for different departments. The set of users is determined by option <include_users>:</include_users>
include_users "user1:user2:user3"
When this report is generated, sarg selects user1 from access log, but ignores user1@MYCOMPANY.RU, even though useralias file is specified in config and it is the same as for report for all users using proxy.
The workaround is to type user twice: user1 and user1@MYCOMPANY.RU. Then sarg will merge traffic for these two accounts into one and statistics will be correct.
Thanks for the report. I committed a patch to check the include_users option after the user names have been aliased.
It sounds logical to decide which user to include in the report after the correct generic user name is found.
The patch is http://sourceforge.net/p/sarg/code/ci/b58ca7214246e9f719ea5ca1f0edfbdf27e99ddd/
Now it is enough to type username only once, the total traffic is counted correctly.
Thank you!
Update from 23.09.2015. Does this patch cover -u option in CLI?
Last edit: Evgeniy Yakushev 2015-09-23
Hello!
Does this option stripusersuffix @mycompany.ru
work correctly in 2.3.10?
Last edit: Bunin Vladimir 2017-03-24
Option
strip_user_suffixis only available in sarg 2.4.It cannot be backported as it requiers features that are not available in sarg 2.3.