#1320 add original sender IP (and forwarder IP) when forwarded by snmptrapd
Hi. This patch adds the information about IP address of each snmptrapd that forwarded the packet. Right now, if you have a chain of forwarders, or the original sending agent address is not preserved (as in SNMP v2 payload) you will not know who had sent the trap originally, especially if the IP is not preserved in the payload. This causes problem with monitoring systems - you will get the trap, but have no idea who sent it, if it went through forwarder.
The patch adds the config option to snmptrapd.conf named "addForwarderInfo" - set it to "1", "yes" or "true".
Or launch with CLI option --add-forwarder-info for the same effect.
This will add the following OID to the payload: .1.3.6.1.6.3.18.1.3.0 (SNMP-COMMUNITY-MIB::snmpTrapAddress.0) when forwarded.
Next forwarder will add 1.3.6.1.6.3.18.1.3.1 (SNMP-COMMUNITY-MIB::snmpTrapAddress.1) when forwarded.
So the end recipient (ie. the monitoring system) has to read the snmpTrapAddress.0 to see who originally sent the trap. If that OID is not populated, it means the trap had been sent directly, without the forwarder.
This was created and tested against net-snmp-5.7.3 on
Centos 6.3 64-bit, kernel 2.6.32-279.22.1.el6.x86_64
Centos 7.0 64-bit, kernel 3.10.0-123.el7.x86_64
2nd version of the patch is attached, nice side effect of adding forwarder information is the ability to detect the forwarding loop: when host A sends to forwarder F1, then F1 -> F2, F2->F3 and F3 sends again to F1.
In that case the second forwarding on F1 will not be done.
Last edit: Pik Master 2015-12-16
References I could find (I don't know how to reply to the old thread):
https://sourceforge.net/p/net-snmp/mailman/message/16657983/ (Re: Changing originator IP in trap/inform)
https://sourceforge.net/p/net-snmp/mailman/message/16013377/ (Re: snmptrapd forwarding)
https://sourceforge.net/p/net-snmp/mailman/message/22932991/ (Is there a way to keep the original sender's IP in forwarded traps)
https://sourceforge.net/p/net-snmp/mailman/message/12804676/ (Re: Suggestions on trap packet forwarding)
* https://support.nagios.com/forum/viewtopic.php?t=9914&p=49790
This patch is very usefull. Please add to upstream
v2 patch - does not properly handles the endianness. No need to convert from network byteorder. It leads to reverted IP on HP-UX.
Also binary IPs should not be compared with strncmp since they can contain \0.
Fixed patch attached.
Hello,
I am using a node with version 5.7.3. When i generated a forwarded trap, it came in the with the forwarded IP address, not the original sender address And I did not see the new OID. Do i need to install the patch separately or is it supposed to come with 5.7.3? I tested with snmptrapd log.
thanks
You need to download 5.7.3 on the forwarder, apply the patch and then compile. Bear in mind that the self-compiled version of snmptrapd will most likely install into /usr/local/bin instead of /usr/bin (unless you override the --prefix option in configure.sh). I usually just delete the net-snmp package provided by the distribution, and set the init scripts (or systemd config files) to point to my custom version, to avoid confusion what's started. Make sure the old daemon is stopped, start the new one, watch for errors in log files and it should work.
Hi ,
Thanks for the clarification. Does version 5.8 contain the patch? I just want to simplify the installation process. With my company software approval process it is easier to have just one piece of software approved rather than two.
Thank you,
Vlad
Last edit: Bart Van Assche 2019-05-10
Hello,
We tried installing the forwarder patch to net-snmp.5.7.3, but got some dependencies issues. Like it was asking for "net-snmp-5.7.3-add-forwader-info" package. Would you have any install instructions, for the patch please?
Thank you,
Vlad
Last edit: Bart Van Assche 2019-05-10
For some reason I could not reply to your message, so I had to post in a separate thread - here https://sourceforge.net/p/net-snmp/patches/1320/#c006
OK, sorry for the late response.
No, the patch is not included in 5.8 release of net-snmp - I tried to get the attention of developers by sending patches and announced to the mailing list. Maybe that will be by next goal - see how these patches are working in that release, but I saw the rewrote or changed some of the network transport code, so it might need some fixes.
Here is the detailed instruction of how to apply it and make it into running installation. I needed some time to test it to make sure it is working fine.
The instruction given will be for Centos 6.10. I had some problem in running this in Centos 7.5, because that OS is using systemd to launch the service, and net-snmp needs some patching for that to work. I will try to test this and post instructions for Centos 7 next.
First, install some prerequisites:
Download the source of net-snmp-5.7.3 and the patch into the folder of your choice, I chose /usr/local/src :
Extract the source:
Apply the patch:
Also, a digression here - this is NOT required, but I apply it anyway. In order to get rid of spurious error in the system log, which look like this:
error on subcontainer 'ia_addr' insert (-1)
I create the patch file first:
# cat > /usr/local/src/snmptrapd-hide-error-subcontainer-ia_addr-insert.patch <<EOF --- ./net-snmp-5.7.3/snmplib/container.c 2014-12-08 20:23:22.000000000 +0000 +++ ./net-snmp-5.7.3-add-forwarder-info/snmplib/container.c 2015-12-16 14:40:40.059768508 +0000 @@ -325,8 +325,8 @@ if(x) { int rc = x->insert(x,k); if(rc) - snmp_log(LOG_DEBUG,"error on subcontainer '%s' insert (%d)\n", - x->container_name ? x->container_name : "", rc); + DEBUGMSGTL(("container:insert_helper", "error on subcontainer '%s' insert (%d)\n", + x->container_name ? x->container_name : "", rc)); else { rc = CONTAINER_INSERT_HELPER(x->next, k); if(rc) EOFThen apply this patch
Now, patching is done, if you ever need to re-configure or re-compile the package, only start from next step, don't repeat the steps above.
Configure the package. I took the options for ./configure.sh from the source RPM of Centos distribution, and removed some options that I don't need:
So this is my full set of configure commands
RPM_OPT_FLAGS="" _sysconfdir="/etc" #so you can use the existing config files in /etc ./configure \ --disable-static --enable-shared \ --with-cflags="$RPM_OPT_FLAGS -D_RPM_4_4_COMPAT" \ --with-ldflags="-Wl,-z,relro -Wl,-z,now" \ --with-sys-location="Unknown" \ --with-logfile="/var/log/snmpd.log" \ --with-persistent-directory="/var/lib/net-snmp" \ --with-mib-modules="$MIBS" \ --with-libwrap=yes \ --sysconfdir=${_sysconfdir} \ --enable-ipv6 \ --enable-ucd-snmp-compatibility \ --with-pic \ --enable-as-needed \ --with-perl-modules="INSTALLDIRS=vendor" \ --enable-mfd-rewrites \ --enable-local-smux \ --with-temp-file-pattern=/var/run/net-snmp/snmp-tmp-XXXXXX \ --with-sys-contact="root@localhost" \ --with-default-snmp-version=1 \ --disable-embedded-perl \ --without-opensslConfigure phase will end with this text, showing the chosen options:
--------------------------------------------------------- Net-SNMP configuration summary: --------------------------------------------------------- UCD-SNMP compatability: enabled SNMP Versions Supported: 1 2c 3 Building for: linux Net-SNMP Version: 5.7.3 Network transport support: Callback Unix Alias TCP UDP TCPIPv6 UDPIPv6 IPv4Base SocketBase TCPBase UDPIPv4Base UDPBase IPv6Base SNMPv3 Security Modules: usm Agent MIB code: default_modules => snmpv3mibs mibII ucd_snmp notification notification-log-mib target agent_mibs agentx disman/event disman/schedule utilities host MYSQL Trap Logging: unavailable Embedded Perl support: disabled SNMP Perl modules: building -- not embeddable SNMP Python modules: disabled Crypto support from: Authentication support: MD5 Encryption support: Local DNSSEC validation: disabledIf you have some errors at this stage, you probably did not install all dependencies. Also mind you, that Ubuntu, or SLES might have different package names for dependencies, the ones above are
Centos specific. Install the dependencies and re-do the configuration step.
Now compile the patched package
# makeIf there were no errors, you can proceed.
I would advise to remove the distribution specific net-snmp package, so the binaries don't mix up (for example the unpatched version of forwarde does not start alongside the patched one, causing race for who binds to the socket first). This is really needed in case you upgrade the system and previously disabled services start up.
But before you do, save the current startup scripts, so you don't have to re-create them from the scratch.
On systems without systemd (like Centos 6 and earlier), you want to copy the init scripts:
Alternatively you can find the init scripts in your dub-directory: ./dist/snmptrapd-init.d and ./dist/snmpd-init.d
Copy the config files (if you modified them - you will need the config file anyway)
Now uninstall the net-snmp provided by the distribution
Install the patched binaries
# make installModify the paths in startup scripts, so the startup scripts will point to /usr/local/sbin where patched binaries will be installed:
# sed -i -e 's,/usr/sbin,/usr/local/sbin,' /usr/local/src/snmpd.initd /usr/local/src/snmptrapd.initdAdd the forwarder option to the config file. This is how my minimal config file looks like. In the last line, you need to replace the your_host_to_forward_traps_to with the IP address or DNS name of the next system, to which you will forward traps to.
Install the modified startup scripts, enable the services and start them
To test:
I have a client 10.10.10.10 sending traps to forwarder 10.11.11.11 (this is the machine we installed the patched binaries on) which forwards to sink 10.33.33.33
In case you don't have the MIBs, here are the numeric OID translations:
On the client I send the trap to forwarder (10.11.11.11):
On the forwarder, I can see in the tcpdump log:
2018-08-29 16:21:09 client.domain.com [10.10.10.10] (via UDP: [10.10.10.10]:58551->[10.11.11.11]:162) TRAP, SNMP v1, community public .1.3.6.1.4.1.8072.2.3.1 Enterprise Specific Trap (17) Uptime: 433 days, 5:09:04.02 .1.3.6.1.4.1.8072.2.1.1 = INTEGER: 123456Also, on the forwarder I can see this using tcpdump (forwarder getting the trap from client, and re-sending it to the sink):
# tcpdump -i ens192 port 161 or port 162 -v tcpdump: listening on ens192, link-type EN10MB (Ethernet), capture size 262144 bytes 16:21:09.897420 IP (tos 0x0, ttl 125, id 26088, offset 0, flags [none], proto UDP (17), length 95) client.domain.com.58551 > forwarder.domain.com.snmptrap: { SNMPv1 { Trap(52) E:8072.2.3.1 10.10.10.10 enterpriseSpecific s=17 3742974402 E:8072.2.1.1=123456 } } 16:21:09.900275 IP (tos 0x0, ttl 64, id 56277, offset 0, flags [DF], proto UDP (17), length 114) forwarder.domain.com.58088 > sink.domain.com.snmptrap: { SNMPv1 { Trap(71) E:8072.2.3.1 10.10.10.10 enterpriseSpecific s=17 3742974402 E:8072.2.1.1=123456 S:18.1.3.0=10.10.10.10 } }On the sink, I can see:
# tcpdump -i eth0 host forwarder.domain.com -v tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 16:24:59.601417 IP (tos 0x0, ttl 61, id 61269, offset 0, flags [DF], proto UDP (17), length 114) forwarder.domain.com.58088 > sink.domain.com.snmptrap: { SNMPv1 { Trap(71) E:8072.2.3.1 10.10.10.10 enterpriseSpecific s=17 3742997383 E:8072.2.1.1=123456 S:18.1.3.0=10.10.10.10 } }So the IP address of the original sender (10.10.10.10) was preserved (well, v1 traps can do it anyway, so see the v2 trap below)
If you add the second forwarder into the mix, so the setup will be:
We can see in the tcpdump logs on the sink:
forwarder2.domain.com.36489 > sink.domain.com.snmptrap: { SNMPv1 { Trap(90) E:8072.2.3.1 10.170.32.21 enterpriseSpecific s=17 3786485186 E:8072.2.1.1=123456 S:18.1.3.0=10.10.10.10 S:18.1.3.1=10.11.11.11 } }If we send the v2c trap:
# snmptrap -v 2c -c public -M c:\tools\snmp\mibs 10.11.11.11 "".1.3.6.1.4.1.8072.2.3.1 .1.3.6.1.4.1.8072.2.1.1 i 123456on the sink (after 2 forwarders), I can see:
forwarder2.domain.com.59235 > sink.domain.com.snmptrap: { SNMPv2c { V2Trap(114) R=16691 system.sysUpTime.0=3786476382 S:1.1.4.1.0=E:8072.2.3.1 E:8072.2.1.1=123456 S:18.1.3.0=10.10.10.10 S:18.1.3.1=10.11.11.11So, normally v2 traps don't preserve the original sender IP, but with our patch, they do it now (you can see it went via client [10.10.10.10], then via forwarder1 [10.11.11.11], then came from forwarder2 [10.22.22.22] to sink [10.33.33.33].
Here is how to make it working with Centos 7.5
Instruction is almost the same as in my previous post, so I will just mention the differences here.
First of all, you need to add some code to net-snmp so it can be started and stopped via systemd.
This is the info - https://github.com/haad/net-snmp/blob/master/README.systemd
Great people at Centos already did that, but Centos 7 has a little bit older release of net-snmp. So I tok the systemd patch for net-snmp 5.7.2 from Centos source RPM:
http://vault.centos.org/centos/7/os/Source/SPackages/net-snmp-5.7.2-32.el7.src.rpm
Extracted it, tried to apply, saw where it cannot apply, corrected this.
This is the resulting file: net-snmp-5.7.2-systemd_corrected_to_5.7.3.patch
So to avoid the patching errors "Hunk #1 FAILED at 40", I re-did the diff command, and here is the result: net-snmp-5.7.3-systemd.patch
To apply it, follow the same instruction as above, which is download, untar and change to the directory
Then apply the systemd patch FIRST:
Now apply the other 2 patches
Then configure - add this to the ./configure command:
--with-systemdcompile as normal:
# makeSave startup files provided by the distribution (you can also use the ones provided in /usr/local/src/net-snmp-5.7.3/dist folder (all files with .service and .socket)
# cp /usr/lib/systemd/system/snmpd.service /usr/lib/systemd/system/snmptrapd.service /usr/local/src/Copy the config files (if you modified them - you will need the config file anyway)
# cp /etc/snmp/snmpd.conf /usr/local/src/Now uninstall the net-snmp provided by the distribution
Install the patched binaries
# make installModify the paths in startup scripts, so the startup scripts will point to /usr/local/sbin where patched binaries will be installed:
# sed -i -e '/ExecStart/s,/usr/sbin,/usr/local/sbin,' /usr/local/src/snmptrapd.service /usr/local/src/snmpd.serviceAdd the forwarder option to the config file. This is how my minimal config file looks like. In the last line, you need to replace the your_host_to_forward_traps_to with the IP address or DNS name
of the next system, to which you will forward traps to.
Install the startup scripts, enable the services and start them
Test (as in previous posts)
Last edit: Pik Master 2019-04-04
Struggling a lot to preserve source IP. Im using redhat linux and my snmp version is 5.9.4. did not used your suggested patch since 5.9.4 version already have this patch but while forwarding my snmptrapd its not preserving the source IP.
Please see my snmptrapd conffilg file
cat snmptrapd.conf
disableAuthorization yes
authCommunity log,execute,net public
addForwarderInfo yes
forward default 10.20.30.40:1163
format1 %V|%y-%02m-%02l %02h:%02j:%02k V1TRAP %b %a %N %w%q %T %v\n
format2 %V|%y-%02m-%02l %02h:%02j:%02k V2TRAP %b %v\n
NET-SNMP Version: 5.9.4.pre2
Web: http://www.net-snmp.org/
Email: net-snmp-coders@lists.sourceforge.net
your help is much appreciated.
Last edit: Rakesh Kumar 2023-09-07
Thanks, Pik. We got it working. Cool stuff.
I just hope NMS companies will pick it up soon. For now we may have to use the traphandle route.
Thank you,
Vlad
Last edit: Bart Van Assche 2019-05-10
I tested this patch against net-snmp 5.7.3 and its working fine. Will there be a patch for 5.8 release.
Instruction is almost the same as in my previous post, so I will just mention the differences here.
First of all, Brother Printer error code 0x803c010b need to add some code to net-snmp so it can be started and stopped via systemd.
Also mind you, that Ubuntu, or SLES might have different package names for dependencies, the ones above are Centos specific Brother Printer error code 0x803c010b. Install the dependencies and re-do the configuration step.
Last edit: Albert John 2018-11-21
When this will part of official release ? in our prod/dev system we dont have permission to compile & build patches ?
we are using 5.7.2 .. in which version it will come as official release ?
The v5.7 branch is no longer maintained. I will have a look and see whether this patch is appropriate for the v5.8 branch.
A modified version of this patch has been applied on the v5.8 and master branches. Please retest.
Bart thanks for that. Did you apply the first version of the patch, or the latest one? sourceforge.net sourceforge.net The first one had some issues with endianess not being handled correctly and other minor things. Pik
Last edit: Bart Van Assche 2019-05-14
Is there a patch for release 5.8. Also
applying net-snmp-5.7.3-snmptrapd-add-forwarder-info-v3.patch is sufficent to recover the orginal ip address for trap forwarding.
Please have a look at the patch that has been checked in on the v5.8 branch. That patch should behave in exactly the same way as v3 of the patch attached to this bug report. See also https://sourceforge.net/p/net-snmp/code/ci/b45782d5632266e86613d9bd5696f0b7fab3ef19/.
I was wondering whether the patch is integrated in the code from the link https://sourceforge.net/projects/net-snmp/files/net-snmp/5.8/. If not where is the above v5.8 branch which contains the above patch.
Hi Jayshankar, the link in your message points at the page with official Net-SNMP releases. An official release that includes this patch is not yet available. Instructions for downloading the latest version of the Net-SNMP source code are available at http://net-snmp.sourceforge.net/wiki/index.php/Git.
Hi PIK Master/Team, I followed steps for centos 7.5 installed 5.7.3 and the 3 patches you provided in the thread. but when i start the snmpd and snmptrapd service , i am getting the following error. Did you face this error, will be very helpful if can advise on this.