Big Sister

Initially I had created the user manually. Once I began seeing issues I
deleted the user and purged all files. I did the install using the Debian
packages. In the past I've always downloaded the files and installed
manually, this time I used the packages.

Yes forcing a password change eliminated the security messages and seems
to have partially fixed things. I read that the zero date (1/1/1970)
causes authentication issues some times. It was causing the security token
to be expired at once, an hence no access for the deamon (or so I assume).

The really odd thing is that in the previous instance (on the old server)
and on every other time I've installed BigSister the behavior was
different. I've altered the screens so that top.html is all that is
displayed. This has always worked well, it gives me a black screen with
green (or whatever) band and I altered the header to display the date and
time of the last scan. This way I can always tell things are being updated
since each 5 minutes the time stamp gets updated on the display. If I see
no change to the time stamp I know something has likely locked up and if an
error occurs the system with the issue pops to the top of the screen.. Now
it only updates that time stamp when there is a status change. I'd prefer
the other behavior and if there is a way to adjust the scripts to have it
refresh top.html at each scan I'd like to know so I can alter the
behavior.. The old server which is a VMware VM running Debian 5 and NOT
Ubuntu works (and is still working) perfectly, this one is a Dell 1850
running Kubuntu (due to disk drivers issues or it would be on debian 5)
this is why I'm confused.

That's normal, bsmon tries to not update files that need no update. Once
you monitor an increasing number of systems you will love this behaviour,
unless you own a server with extremely good I/O performance.

So changing the password did the trick? Quite odd, with or without warning
the daemons should start and work or not start at all. Did you install
BigSister from the .deb package? Did you create the bigsis user manually?

1/1/1970 as the date of last password change is ok - it does just mean it
had never been changed (1/1/1970 is date "0").

Spoke too soon. I shut off a server so that "something" was red. (I use
the big brother skin). NOW things are updating, but ONLY when there is a
status change. The question is why is top.html not updating at each scan
cycle.

Another oddity. Appearently there is a bug in unix/linux that relates to
having 1/1/1970 as the date of last password change. forcing a password
change on the bigsis account stopped the "password change required"
messages.

Now i see files in /var/lib/bigsister/ graphs being updated as the deamon
cycles through the systems it needs to scan. Also files in
/var/lib/bigsister/www/html are being updated. However top.html and the
other files in the var/lib/www folder are NOT being updated.

Thomas,
The deamons appear to stay running now. Runnning "ps -aux | grep mon"
results in this:

bigsis 6126 0.1 0.1 13916 9824 ? S 12:38 0:07 bsmon
bigsis 6174 0.0 0.0 9720 6076 ? S 12:38 0:00 uxmon
root 6180 0.0 0.1 10748 8832 ? S 12:38 0:00 uxmon

agent.log, display.history, and the numbered status logs in
/var/lib/bigsister are being updated. bsmon.state is not nor is the any of
the html. I uninstalled apparmor and saw no difference. if you still
think strace would help i can run it anyway. I'm at a loss.

Kenneth,
no, there is no default password. The start script uses "su" to become the
"bigsis" (or whatever you configured it to be) user, e.g. bbd will be
started with "su - bigsis -c /usr/share/bigsister/bin/bbd". So, the
bigsister account does not need a password (just use a locked account), but
a valid shell (i.e. /bin/sh).

Now as you mention Ubuntu: AFAIK, Ubuntu comes with AppArmor installed by
default. This *might* cause troubles. Unless you intend to maintain
AppArmor profiles an option were to uninstall apparmor (apt-get remove
apparmor). However, you should see messages in /var/log/messages if
AppArmor were causing the troubles.

As I already mentioned, if your daemons die from unknown reasons, try
strace:

- Start BigSister
- find the PID of e.g. bbd and/or bsmon
- execute "strace -f -p <pid> 2> /var/tmp/strace..."
- wait until the daemons die
- send me the files

Note that trace files might become *very* large. Have an eye on disk space
when trying this out.

Best regards,
Tom

Thomas, Is there a default password that the bigsis service acount(s) uses
or should there be no password? I cant locate that info in the docs. I
cant recall having issues with this in the past, of course this is the
first time using ubuntu as well which appears to have different security
requirements than basic Debian.

Came in today to an entire purple screen. IO made copies of everything in
etc/bigsister, var/livb/bigsister and then completely removed and purged
BigSis from the system and reinstalled everything from scratch. I then
copied all files back over to keep my settings. It appeared to be working
but now seems not. Still think it's permissions somehow, just need to
track it down. I see this in the messages. Guess i need to get rid of
that password requirement first of all:

Messages:
Monitor bsmon: You are required to change your password immediately
(root enforced)
Monitor bsmon: su: Authentication token is no longer valid; new one
required
Monitor bsmon: (Ignored)
Server bbd: You are required to change your password immediately (root
enforced)
Server bbd: su: Authentication token is no longer valid; new one
required
Server bbd: (Ignored)
Agent uxmon for uxmon-net: You are required to change your password
immediately (root enforced)
Agent uxmon for uxmon-net: su: Authentication token is no longer valid;
new one required

In going through the system logs I'm convinced the issue is a permissions
issue. What I see now is that all daemons stop shortly after starting.
Directory permissions are wide open but I think the bigsis service account
does not have appropriate permissions somewhere. Odd since it did before.
I'm trying to determine where bigsister gets the information to start the
daemons when bb_start is run. How does it know what password to use? I
cant seem to find that in the docs.

Kenneth,
actually I have no idea. I do not think that there is any interference
with Cacti. You say, the status.log* file is updating? If not: are you sure
the agent(s) actually report to the new server? If yes ... hmh ... try an
"fuser" on the current status log file - the bsmon process should get
listed. Try an strace on the bsmon process. Does it show activity?

This is probably something really silly we cannot even think about :-)

Best regards,
Tom

Another update:
Seems the files are not being updated. It acts as though the service is
hung. Status log do not show anything out of the ordinary. PS -a shows
bsmon and uxmon running, they just arent updating. As a band-aid I'm
adding a cron job to run the service reset command at a different interval
from the cacti job. manually restarting the service makes things update,
but just once.

Heres an update. I opened up all directory permissions to wide open. The
/var/lib/bigsister directory seems to be functioning normally. The log
files in it are being updated but the HTML isn't being regenerated. in the
www directory.