bandwidthd fails to recover logs

Help
BioTube
2011-04-21
2013-04-24
1 2 > >> (Page 1 of 2)
  • BioTube
    BioTube
    2011-04-21

    I can clearly see log files being generated in /var/lib/bandwidthd, but on startup the daemon seems to be ignoring them, despite everything in the conffile looking right.

    ####################################################
    # Bandwidthd.conf
    #
    # Commented out options are here to provide
    # documentation and represent defaults
    # Subnets to collect statistics on.  Traffic that
    # matches none of these subnets will be ignored.
    # Syntax is either IP Subnet Mask or CIDR
    #subnet 192.168.0.0/24
    subnet 192.168.1.0/24
    # Device to listen on
    # Bandwidthd listens on the first device it detects
    # by default.  Run "bandwidthd -l" for a list of
    # devices.
    #dev "eth0"
    dev "wlan0"
    ###################################################
    # Options that don't usually get changed
    # An interval is 2.5 minutes, this is how many
    # intervals to skip before doing a graphing run
    #skip_intervals 0
    # Graph cutoff is how many k must be transfered by an
    # ip before we bother to graph it
    #graph_cutoff 1024
    #Put interface in promiscuous mode to score to traffic
    #that may not be routing through the host machine.
    #promiscuous true
    promiscuous false
    #Log data to cdf file htdocs/log.cdf
    #output_cdf false
    output_cdf true
    #Set the cdf log output directory
    #log_dir "/var/lib/bandwidthd"
    #Read back the cdf file on startup
    #recover_cdf false
    recover_cdf true
    #Libpcap format filter string used to control what bandwidthd see's
    #Please always include "ip" in the string to avoid strange problems
    #filter "ip"
    #Draw Graphs - This default to true to graph the traffic bandwidthd is recording
    #Usually set this to false if you only want cdf output or
    #you are using the database output option.  Bandwidthd will use very little
    #ram and cpu if this is set to false.
    #graph true
    #Set META REFRESH for static pages in seconds(default 150, use 0 to disable).
    #meta_refresh 150
    meta_refresh 150
    #Set the static html output directory
    #htdocs_dir "/var/lib/bandwidthd/htdocs"
    
     
  • tarpit
    tarpit
    2011-05-13

    Same here, on Windows machine. Windows application event viewer says "log.1.0.cdf is corrupted, skipping".
    I did normal Windows restart. Isn't it supposed to work this way? So why is log file corrupted?

     
  • Alestan
    Alestan
    2011-06-02

    My condolences to tarpit for running windows, I can't help you, least probably not.  As for biotube, try su -c 'tail -f /var/log/syslog' and then start it, you might get a message about why it's ignoring the log file.  If you do, post what the message is and I'll see what I can do.

    Regards,

     
  • Bill
    Bill
    2011-08-16

    I am also having this problem. I'm using 2.0 RC3 (this morning's build Aug 16th, 2011)

    I have save log file cdf and re-load on startup. Seems to be ignoring my configuration. :(

     
  • Alestan
    Alestan
    2011-08-16

    bhilton81:  Can you check the contents of /var/log/syslog (if you are running a posix system) or whatever the windows syslog is if you are on NT?  Look for anything posted by bandwidthd.

     
  • Bill
    Bill
    2011-08-17

    Everything related to bandwidthd looked good in the system.log (pfsense FreeBSD) except these two lines:
    Aug 17 09:37:17 pfsense bandwidthd: Cannot open htdocs/index.html for writing
    Aug 17 09:37:17 pfsense bandwidthd: Cannot open htdocs/index.html for writing

    I didn't feel like rebooting 3 separate times, so I did the following:

    cd /usr/local/bandwidthd
    chmod ugo+rw log*
    chmod ugo+rw htdocs
    cd htdocs
    chmod ugo+rw *

    Obviously this is not the best security setup, but there are no bits of information in here that I care about. There isn't WAN access to the box, other than SSH, and I have no internal users to worry about. Just my roommate and I.

    I rebooted with those changes and it works now!

    I lost previous data, but since the initial reboot at 9AM, my latest reboot at 9:50AM, following those changes, has maintained and reloaded the data!

    Hopefully somebody can use that information and work with pfSense to sort the bandwidthd package installation out for other people.

    I will post a follow up to this if I run in to any problems in the next few days or so.

     
  • Bill
    Bill
    2011-08-17

    I just updated to the latest version using the invoke auto update button on pfsense, it rebooted, and voila, I've lost my bandwidthd logs again.

    It may not be security at all!

    The only time I've actually had success with logs sticking after a reboot was when I manually typed "reboot" at shell.

    The other times when I've lost the logs were all after clicking the invoke auto update button within pfsense.

    I am doing another test later today to confirm - but my suspicion is a reboot will retain my logs and reload them just fine. It's something to do with pfsense's auto updater that's killing the logs.

    Please stand by.

     
  • Bill
    Bill
    2011-08-17

    Confirmed. I've rebooted 2 more times and my bandwidthd logs are being saved and reloaded properly.

    The only times I've lost the logs were the 3 times I updated pfsense to the latest version. It auto reboots when the update is done and it must clean the bandwidthd packages folder out too!

    I'll have to open a thread on the pfsense forum now.

    Cheers & Thanks.

     
  • Alestan
    Alestan
    2011-08-25

    Okay, that kinda makes sense, I don't know why your bandwidthd package is configured to point to a directory where it doesn't have write access.  Does bandwidthd run as root?  (On debian it needs to). Just a wild guess, but tarpit is probably running vista or windows 7, and so bandwidthd is failing to create the log files because it's probably not running as administrator and doesn't have permission to create files outside the user's home directory.

     
  • jlct021
    jlct021
    2012-02-17

    Hi I am having same problem on Debian Squeeze  - That being that Bandwidthd is loosing graphs after re-boot.

    My var/log/syslog file is telling me the following:

    Feb 17 17:44:52 debian bandwidthd: Monitoring subnet 192.168.0.0 with netmask 192.168.0.0
    Feb 17 17:44:53 debian bandwidthd: Recovering from /var/lib//bandwidthd/log.1.1.cdf
    Feb 17 17:44:53 debian bandwidthd: Finished recovering 0 records
    Feb 17 17:44:53 debian bandwidthd: Failed to parse part of log file. Giving up on the file
    Feb 17 17:44:53 debian bandwidthd: Recovering from /var/lib//bandwidthd/log.1.0.cdf
    Feb 17 17:44:53 debian bandwidthd: Finished recovering 0 records
    Feb 17 17:44:53 debian bandwidthd: Failed to parse part of log file. Giving up on the file
    Feb 17 17:44:53 debian bandwidthd: Opening br0
    Feb 17 17:44:53 debian bandwidthd: Opening br0
    Feb 17 17:44:53 debian bandwidthd: Recovering from /var/lib//bandwidthd/log.2.0.cdf
    Feb 17 17:44:53 debian bandwidthd: Finished recovering 0 records
    Feb 17 17:44:53 debian bandwidthd: Failed to parse part of log file. Giving up on the file
    Feb 17 17:44:53 debian bandwidthd: Recovering from /var/lib//bandwidthd/log.3.0.cdf
    Feb 17 17:44:53 debian bandwidthd: Opening br0
    Feb 17 17:44:53 debian bandwidthd: Finished recovering 0 records
    Feb 17 17:44:53 debian bandwidthd: Failed to parse part of log file. Giving up on the file
    Feb 17 17:44:53 debian bandwidthd: Opening br0
    Feb 17 17:44:53 debian kernel:  device br0 entered promiscuous mode
    Feb 17 17:44:53 debian bandwidthd: Packet Encoding: Ethernet
    Feb 17 17:44:53 debian bandwidthd: Drawing initial graphs
    Feb 17 17:44:53 debian bandwidthd: Packet Encoding: Ethernet
    Feb 17 17:44:53 debian bandwidthd: Packet Encoding: Ethernet
    Feb 17 17:44:53 debian bandwidthd: Drawing initial graphs
    Feb 17 17:44:53 debian bandwidthd: Packet Encoding: Ethernet
    Feb 17 17:44:53 debian bandwidthd: Drawing initial graphs

    Any help very welcome

    Thanks.

     
  • jlct021
    jlct021
    2012-02-17

    have checked below files and there is data there….

    root@debian:/var/lib/bandwidthd# ls
    htdocs log.1.0.cdf  log.1.1.cdf  log.2.0.cdf  log.3.0.cdf

     
  • Alestan
    Alestan
    2012-02-21

    cdf files are comma deliminated files, so you should be able to open them in a spreadsheet or text editor and verify that they are readable.  If they are, make sure bandwidthd runs as root or at least has read access to the files.

     
  • jlct021
    jlct021
    2012-02-22

    Files are readable; opened with gedit

    How can I check  bandwidthd runs as root or at least has read access to the files?

    Thanks

     
  • warhed
    warhed
    2012-02-22

    Hello,

    I have this same problem, but I am not clear on what the best approach is to do? Give full access rights to myself on the htdocs folder? I set this up as root, so not sure why it isn't gathering the data.

    I have deployed BandwidthD on 3 Untangle/Debian systems, all of them have this problem.

    :(

     
  • jlct021
    jlct021
    2012-02-25

    If the _cdf files are located in /var/lib/bandwidthd

    Why does Bandwidthd need write access to the htdocs folder?

     
  • warhed
    warhed
    2012-02-25

    ^ Sorry i posted the wrong location, I am not that familiar with Linux or BandwidthD.

    Still, what do I need to do to enable BD to recover the logs from a reboot? It seems to be permission based.

     
  • jlct021
    jlct021
    2012-02-25

    " I am not that familiar with Linux or BandwidthD. Still, what do I need to do to enable BD to recover the logs from a reboot? It seems to be permission based."

    I am having the same problem.

    Can anyone help?

    Would really be appreciated.

    Thanks

     
  • jlct021
    jlct021
    2012-02-26

    Though I have applied the patch

    reinstalled using the download from sourceforge

    And am still having the same problem

    Perhaps I missed something; will try again tomorrow.

    Has anyone here had success using this patch?

     
  • warhed
    warhed
    2012-02-27

    I am more than willing to apply the bandwidthd-recover-cdf.patch patch to my BandwidthD but I do not know how? I have WINSCP'd into my Untangle Debian box, BandwidthD is running.
    If someone could provide steps via the console screen (I use putty to log in) that would be great. I placed the file in root/tmp thus far.

    Appreciate any help.

     
  • jlct021
    jlct021
    2012-02-28

    It Works.

    Using this guide as a reference:

    http://raphaelhertzog.com/2010/12/15/howto-to-rebuild-debian-packages/

    I did the following:

    Remove all previous versions of Bandwidthd. (I did a clean install of Debian Squeeze to be safe)

    apt-get update
    apt-get install devscripts
    apt-get install apache2

    mkdir BWD (call the directory what you like)
    apt-get source bandwidhd
    apt-get build-dep bandwidthd
    cd bandwidthd-2.0.1+cvs20090917
    gedit bandwidthd.c
    changed the 7's to 8's near the bottom of the file (useing the patch file as a reference)
    dch -local foo (change foo to something of your choice)
    (make a small note re the changes being made) save and exit
    cd bandwidthd-2.0.1+cvs20090917
    debuild -us -uc
    cd back to the BWD directory
    sudo dpkg -i bandwidthd-2.0.1+cvs20090917-4.1-jb1_amb64.deb (jbl will be what you named it earlier)

    I then copied the bandwithd.conf file from /name/home/BWD/bandwidthd-2.0.1+cvs20090917/etc
    to
    /etc/bandwidthd/bandwidthd.conf
    gedit /etc/bandwithd/bandwithd.conf (and edited accordingly)
    ln -s bandwidthd /var/lib/bandwidthd/htdocs (to create the symlink)
    /etc/init.d/apache2 restart
    /etc/init.d/bandwidthd restart
    172.18.91.x/bandwidth

    Bandwidthd is now recovering the logs form the_cdf files after a restart / reboot

    Big thanks to Logan Gunthorpe for the Bug Fix and help in getting it to work.

     
  • warhed
    warhed
    2012-02-28

    Looks like you are rebuilding the whole package which seems like a good idea, but beyond my linux skills. I got lost at the gedit bandwidthd.c, I cannot find that file.

    Has anyone made a full, newer build, with this patch applied?

     
  • warhed
    warhed
    2012-05-04

    Looks like this has been abandoned which is too bad. I am still stuck.

     
  • Alestan
    Alestan
    2012-05-04

    Not abandoned, just a case of the squeaky wheel, so keep on squeaking.  Rebuilding a debian package is pretty simple, I assume you managed to follow the apt-get commands.  Run them from a working directory, I usually use $HOME/Source.  Once you've run all them, you'll have a folder $HOME/Source/bandwidthd-2.0.1+cvs20090917 (or similar if you are running a different version of debian).
    You'll see a line that says `dpkg-source: warning: unexpected end of diff `bandwidthd-2.0.1+cvs20090917/debian/patches/44_fixrecovercdf.patch'` when running apt-get source bandwidthd, that bug is what you need to fix. 

    My advice is to get it to compile with no changes before trying to change anything.  To that end, go into the bandwidthd directory ($HOME/Source/bandwidthd-2.0.1…) and type `debuild -us -uc`
    Once that runs without errors, then open bandwidthd.c in your favourite text editor.  (something like `sudo gedit $HOME/Source/bandwidthd…/bandwidthd.c`)
    Apply the contents of the corrupted patch file, (replace the 7 with 8 as described above), save the file, and rerun the debuild command.  That should leave you with a working system, you'll find a bandwidthd executable in $HOME/Source/bandwidthd/bandwidthd, which you could copy to /usr/bin/bandwidthd in place of the one from the normal package, or you can run it in place.  If any of that doesn't make sense,

     
  • I had try to rebuild the pack but still not working.

    Some one else had success fixing the bug?

     
1 2 > >> (Page 1 of 2)