Re: [opennms-discuss] OpenNMS monitoring and data collection from the Azure REST API

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 454-5900

I’ve turned on debug for the collector and found the following result returned from Microsoft in the collectd.log:
2018-05-14 13:12:09,050 DEBUG [pool-10-thread-28] o.a.h.wire: http-outgoing-8 << "{"error":{"code":"MissingApiVersionParameter","message":"The api-version query parameter (?api-version=) is required for all requests."}}"

This suggests that the following code in the xml-datacollection-config.xml is not being used correctly:

            <request method="GET">
                <parameter name="api-version" value="2017-12-01"/>

So I tried moving the api-version parameter into the url parameter like this:

        <xml-source url="https://management.azure.com/{assetnumber}/providers/Microsoft.ResourceHealth/availabilityStatuses/current?api-version=2017-07-01">

Which made the missing api-version parameter error go away, but I then noticed something that I had missed before.

The string I’ve stored in the assetnumber field is the Azure resource ID for the Vm which looks like this:

/subscriptions/aaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa/resourceGroups/AB-CDEF99/providers/Microsoft.Compute/virtualMachines/hostname

But the log is showing that all the “/” are being converted to %2F

2018-05-14 14:27:48,873 DEBUG [pool-10-thread-174] o.o.p.x.c.AbstractXmlCollectionHandler: collect: starting source url 'https://management.azure.com/%2Fsubscriptions%2Faaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa%2FresourceGroups%2FAB-CDEF99%2Fproviders%2FMicrosoft.Compute%2FvirtualMachines%2Fhostname/providers/Microsoft.Compute/virtualMachines/hostname?api-version=2018-01-01 ' collection with request: Request [method=GET, parameters=[api-version=2017-12-01], headers=[Authorization=Bearer AUTHTOKEN_REDACTED, Content-Type=application/json], content=null]

And then the GET that is executed, is showing that the substituted ‘%’ is then substituted again:

collectd.log:2018-05-14 14:27:49,034 DEBUG [pool-10-thread-174] o.a.h.i.e.MainClientExec: Executing request GET /%252Fsubscriptions%252Faaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa%252FresourceGroups%252FAB-CDEF99%252Fproviders%252FMicrosoft.Compute%252FvirtualMachines%252Fhostname/Microsoft.Compute/virtualMachines/hostname?api-version=2018-01-01 HTTP/1.1

How do I prevent the xml-collector from expanding special characters when expanding asset info variable names?  Or how do I store a string so that when it hits the actual GET request, the URI is in the correct format of  /subscriptions/aaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa/resourceGroups/AB-CDEF99/providers/Microsoft.Compute/virtualMachines/hostname/Microsoft.Compute/virtualMachines/hostname?api-version=2018-01-01

From: JohnD Blackburn [mailto:joh...@au...]
Sent: Friday, 11 May 2018 4:25 PM
To: General OpenNMS Discussion <ope...@li...>
Subject: Re: [opennms-discuss] OpenNMS monitoring and data collection from the Azure REST API

I now have a script that updates each Azure hosted node with details from azure stored in several asset  info fields, so as that script knows if a node lives in Azure, it should be easy to get that script to assign my Azure services to those nodes using provision.pl

But as a test, I’ve tried using karaf to check that my collection is working, and I get the following error:

admin@opennms> collection:collect -n 223 org.opennms.protocols.xml.collector.XmlCollector 10.2.0.9 collection=xml-ABBCS-Azure-burstVM
[Fatal Error] :1:1: Content is not allowed in prolog.

Collect failed with:java.util.concurrent.ExecutionException: org.opennms.netmgt.collection.api.CollectionException: Can't collect XML data because Content is not allowed in prolog.
       at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
       at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)
       at org.opennms.netmgt.collection.commands.CollectCommand.execute(CollectCommand.java:128)
       at org.opennms.netmgt.collection.commands.CollectCommand.execute(CollectCommand.java:64)
       at org.apache.karaf.shell.impl.action.command.ActionCommand.execute(ActionCommand.java:84)
       at org.apache.karaf.shell.impl.console.osgi.secured.SecuredCommand.execute(SecuredCommand.java:68)
       at org.apache.karaf.shell.impl.console.osgi.secured.SecuredCommand.execute(SecuredCommand.java:86)
       at org.apache.felix.gogo.runtime.Closure.executeCmd(Closure.java:571)
       at org.apache.felix.gogo.runtime.Closure.executeStatement(Closure.java:497)
       at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:386)
       at org.apache.felix.gogo.runtime.Pipe.doCall(Pipe.java:417)
       at org.apache.felix.gogo.runtime.Pipe.call(Pipe.java:229)
       at org.apache.felix.gogo.runtime.Pipe.call(Pipe.java:59)
       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
       at java.lang.Thread.run(Thread.java:745)
Caused by: org.opennms.netmgt.collection.api.CollectionException: Can't collect XML data because Content is not allowed in prolog.
       at org.opennms.protocols.xml.collector.XmlCollector.collect(XmlCollector.java:259)
       at org.opennms.netmgt.collection.client.rpc.CollectorClientRpcModule$1.get(CollectorClientRpcModule.java:83)
       at org.opennms.netmgt.collection.client.rpc.CollectorClientRpcModule$1.get(CollectorClientRpcModule.java:77)
       at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
       ... 3 more
Caused by: org.opennms.netmgt.collection.api.CollectionException: Content is not allowed in prolog.
       at org.opennms.protocols.xml.collector.AbstractXmlCollectionHandler.collect(AbstractXmlCollectionHandler.java:184)
       at org.opennms.protocols.xml.collector.XmlCollector.collect(XmlCollector.java:257)
       ... 6 more
Caused by: org.xml.sax.SAXParseException: Content is not allowed in prolog.
       at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
       at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
       at javax.xml.parsers.DocumentBuilder.parse(Unknown Source)
       at org.opennms.protocols.xml.collector.AbstractXmlCollectionHandler.getXmlDocument(AbstractXmlCollectionHandler.java:482)
       at org.opennms.protocols.xml.collector.AbstractXmlCollectionHandler.getXmlDocument(AbstractXmlCollectionHandler.java:456)
       at org.opennms.protocols.xml.collector.DefaultXmlCollectionHandler.fillCollectionSet(DefaultXmlCollectionHandler.java:55)
       at org.opennms.protocols.xml.collector.AbstractXmlCollectionHandler.collect(AbstractXmlCollectionHandler.java:178)
       ... 7 more

In Collectd-configuration.xml:
   <package name="ABBCS-Azure" remote="false">
      <filter>(IPADDR != '0.0.0.0') &amp; (asset-vendor="Microsoft Azure")</filter>
      <include-range begin="1.1.1.1" end="254.254.254.254"/>
      <include-range begin="::1" end="ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff"/>
      <service name="ABBCS-Azure-BurstVM" interval="300000" user-defined="false" status="on">
         <parameter key="handler-class" value="org.opennms.protocols.json.collector.DefaultJsonCollectionHandler"/>
         <parameter key="collection" value="xml-ABBCS-Azure-burstVM"/>
      </service>
   </package>
…
   <collector service="ABBCS-Azure-BurstVM" class-name="org.opennms.protocols.xml.collector.XmlCollector"/>

In xml-datacollection-config.xml, I have the following collection:

    <xml-collection name="xml-ABBCS-Azure-burstVM">
        <rrd step="300">
            <rra>RRA:AVERAGE:0.5:1:2016</rra>
            <rra>RRA:AVERAGE:0.5:12:1488</rra>
            <rra>RRA:AVERAGE:0.5:288:366</rra>
            <rra>RRA:MAX:0.5:288:366</rra>
            <rra>RRA:MIN:0.5:288:366</rra>
        </rrd>
        <xml-source url="https://management.azure.com/{assetnumber}<https://management.azure.com/%7bassetnumber%7d>">
            <request method="GET">
                <parameter name="api-version" value="2017-12-01"/>
                <header name="Authorization" value="Bearer {password}"/>
                <header name="Content-Type" value="application/json"/>
            </request>
            <import-groups>xml-datacollection/ABBCS-azure-burstVM.xml</import-groups>
        </xml-source>
    </xml-collection>

Where {assetnumber} is the asset info field where I’ve stored the Azure ResourceID and {password} is the asset info field where I have stored the access token

The “GET” defined above when run from CURL returns the attached json data

And xml-datacollection/ABBCS-azure-burstVM.xml contains:

<xml-groups>
    <xml-group name="xml-ABBCS-Azure-burstVM" resource-type="AzureResource"
                resource-xpath="/value"
                key-xpath="name[@value]">
      <xml-object name="Metric" type="string" xpath="name[@value]" />
      <xml-object name="unit" type="string" xpath="@unit" />
      <xml-object name="value" type="GAUGE" xpath="timeseries[@data]/@average" />
    </xml-group>
</xml-groups>

I’m uncertain if I have my xml-group defined correctly, and the above error from karaf suggests I have not.

Either the xml-group is wrong, or the xml-source is wrong (or both)

Regards,
John

From: Dino Yancey [mailto:din...@gm...]
Sent: Tuesday, 8 May 2018 11:45 PM
To: General OpenNMS Discussion <ope...@li...<mailto:ope...@li...>>
Subject: Re: [opennms-discuss] OpenNMS monitoring and data collection from the Azure REST API

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.

With the XML and JSON collection I'm doing, I create a detector to identify the specific device or system type via some other means. For example, in one case I am using the WebDetector to identify the class of device based on regex match in the output, and the service it creates
has data collection defined via the XmlCollector. I'm not sure if that's a tactic that would be helpful to you, as I don't know what Azure VMs "look like" from the outside of what identifying information you could use.
D

On Mon, May 7, 2018 at 10:21 PM, JohnD Blackburn <joh...@au...<mailto:joh...@au...>> wrote:
Thanks for that Dino, I’ve read that document already. However, it doesn’t help answer my question about the fact that the authentication token needed to be passed in every Azure REST API call as an HTTP header is only valid for an hour and a separate API call is required to get a new access token, So I’m trying to figure out the best way to get OpenNMS to have the correct URI. The doco is also not clear how to map the JSON data to fields in the opennms XML config file. The JSON data returned from Azure API is sufficiently different to the documented example that I’m getting confused as to how I should be referencing it. But that could be just that I was looking at it at the end of the week, and/or needed more coffee :D
I think I’ve managed to get something working by using fields in the asset info which I’ve got a shell script that updates the password field in the Authentication section with the current access token.
xml-datacollection-config.xml contains:
RRA:AVERAGE:0.5:1:2016</rra>
RRA:AVERAGE:0.5:12:1488</rra>
RRA:AVERAGE:0.5:288:366</rra>
RRA:MAX:0.5:288:366
RRA:MIN:0.5:288:366
xml-datacollection/ABBCS-azure-test.xml
However, trying to set up a detector in Default Foreign Source, I’m having trouble specifying the http headers required to implement this in the detector:
xml-datacollection/ABBCS-azure-test.xml
Without passing that authorization header in the get request, I cannot get the correct response code to validate “detection”
Any ideas on how to detect this?
(note: the assetnumber field comes from the Asset Info record in OpenNMS and is the field I picked to store the Azure ResourceID, as there isn’t really any Azure(/generic cloud provider) specific fields in the Asset Info record, so I just picked one…)
Or is using a detector in this instance the wrong way to go about this?
Regards
John.
From: Dino Yancey [mailto:din...@gm...<mailto:dino2gnt%2Bo...@gm...>]
Sent: Friday, 4 May 2018 10:16 PM
To: General OpenNMS Discussion <ope...@li...<mailto:ope...@li...>>
Subject: Re: [opennms-discuss] OpenNMS monitoring and data collection from the Azure REST API
https://wiki.opennms.org/wiki/XML_Collector#JSON_Collector<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.opennms.org%2Fwiki%2FXML_Collector%23JSON_Collector&data=02%7C01%7Cjohnd.blackburn%40au.abb.com%7Cc787b84b26844913b97508d5b4ea0cbd%7C372ee9e09ce04033a64ac07073a91ecd%7C0%7C0%7C636613839702196446&sdata=WJkjlCAJbLKdqL4BGwlzjDqwk2hiLO9RigOoeiQiLdw%3D&reserved=0>
On Thu, May 3, 2018 at 10:09 PM, JohnD Blackburn <joh...@au...<mailto:joh...@au...>> wrote:
Hi All,
We are currently using OpenNMS 21.1.0 to monitor all our Azure VM’s, and it has been brought up that there is a whole bunch of metrics and statuses that are only available through Azure. I have been looking at options for getting some of this info into OpenNMS, and have found several potential methods for achieving this.
It is possible to set up webhooks in Azure that get called when events, statuses, and thresholds are triggered, which could possibly work, but would require me to set up an internet facing webserver to receive the webhook calls, and to then process the received JSON data to the correct node in OpenNMS. This would also require configuration added to Azure to “send” these details.
A second option I have come across is to use the Azure REST API which is documented here: https://docs.microsoft.com/en-us/rest/api/azure/<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Frest%2Fapi%2Fazure%2F&data=02%7C01%7Cjohnd.blackburn%40au.abb.com%7Cc787b84b26844913b97508d5b4ea0cbd%7C372ee9e09ce04033a64ac07073a91ecd%7C0%7C0%7C636613839702196446&sdata=6fr%2F6zqxUzJG%2BG4wOTwr0gSEfPKBzSB4gMYeB%2B7MO1c%3D&reserved=0>
Using this should allow me to drive the collection from the OpenNMS server, however I would need to get the mapping between OpenNMS nodes and Azure resources working.
To be able to get details out of Azure for a specific node, I would need to know the “SubscriptionID”, “ResourceGroupName” and “ResourceID” from Azure, and to complicate matters, requests to the Azure API require an access token which I can get with the following curl commandline which returns the access token in a JSON formatted file
curl -X POST -d 'grant_type=client_credentials&client_id='$CLIENTID'&client_secret='$CLIENTSECRET'&resource=https%3A%2F%2Fmanagement.azure.com<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F2Fmanagement.azure.com&data=02%7C01%7Cjohnd.blackburn%40au.abb.com%7Cc787b84b26844913b97508d5b4ea0cbd%7C372ee9e09ce04033a64ac07073a91ecd%7C0%7C0%7C636613839702196446&sdata=WRgLHeAafTVbxSDxuxN6m4vTq1AAbfMm7jvWGMohMDE%3D&reserved=0>%2F' https://login.microsoftonline.com/$TENANTID/oauth2/token<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flogin.microsoftonline.com%2F%24TENANTID%2Foauth2%2Ftoken&data=02%7C01%7Cjohnd.blackburn%40au.abb.com%7Cc787b84b26844913b97508d5b4ea0cbd%7C372ee9e09ce04033a64ac07073a91ecd%7C0%7C0%7C636613839702196446&sdata=MSCBO21J%2B%2Bb8%2BaaDXger8LUMgCIvvA4uKPyrvgAeVT0%3D&reserved=0> > $TOKEN_FILE
This Access token that is returned expires after an hour, so a new access token would need to be requested every hour. The access token is passed into the HTTP request as a header. The following curl command shows the format required for the http request:
curl -X GET -H "Authorization: Bearer $TOKEN_ID" -H "Content-Type: application/json" https://management.azure.com/$URI<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmanagement.azure.com%2F%24URI&data=02%7C01%7Cjohnd.blackburn%40au.abb.com%7Cc787b84b26844913b97508d5b4ea0cbd%7C372ee9e09ce04033a64ac07073a91ecd%7C0%7C0%7C636613839702196446&sdata=vNeZnUfaeHUJH719cVXr%2FSxzU8r1pITrzNdXiNnWtZk%3D&reserved=0>
That access token is 1335 characters long.
What is the best way to capture data from the Azure REST API? Do I need a script that runs hourly to update the access token in the asset comment field and store all those variables in Asset Fields? Would a 1335 character string be storable in an asset field?
Has anyone implemented any OpenNMS monitoring/data collection from the Azure REST API?
Also, Trying to create a collector for the JSON, given the following JSON as returned by Azure:
{
"id": "/subscriptions/SubscriptionID/resourceGroups/ResourceGroupName/providers/Microsoft.Compute/virtualMachines/VMName/providers/Microsoft.ResourceHealth/availabilityStatuses/current",
"name": "current",
"type": "Microsoft.ResourceHealth/AvailabilityStatuses",
"location": "australiaeast",
"properties": {
"availabilityState": "Available",
"summary": "There aren't any known Azure platform problems affecting this virtual machine",
"detailedStatus": "",
"reasonType": "",
"occuredTime": "2018-04-26T09:15:52Z",
"reasonChronicity": "Persistent",
"reportedTime": "2018-05-04T02:02:47.222185Z"
}
}
How would I format the XML Group definition in xml-datacollection? It is not clear to me from the documentation.
I’m guessing something like this:
resource-xpath="/"
key-xpath="@id">
availabilityState]" />
Regards,
John Blackburn

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsdm.link%2Fslashdot&data=02%7C01%7Cjohnd.blackburn%40au.abb.com%7Cc787b84b26844913b97508d5b4ea0cbd%7C372ee9e09ce04033a64ac07073a91ecd%7C0%7C0%7C636613839702196446&sdata=Zors8Epzlh3PzfLMsZFS2YJi7PF0vlSIU67cNM0kkgY%3D&reserved=0>
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.opennms.org%2Findex.php%2FMailing_List_FAQ&data=02%7C01%7Cjohnd.blackburn%40au.abb.com%7Cc787b84b26844913b97508d5b4ea0cbd%7C372ee9e09ce04033a64ac07073a91ecd%7C0%7C0%7C636613839702196446&sdata=wQx7q%2F978h%2BZbm1Wuspcm86SBdfblK8YPwHf9JjjiHg%3D&reserved=0>

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fopennms-discuss&data=02%7C01%7Cjohnd.blackburn%40au.abb.com%7Cc787b84b26844913b97508d5b4ea0cbd%7C372ee9e09ce04033a64ac07073a91ecd%7C0%7C0%7C636613839702196446&sdata=KhgkZvn4jPR2Mzpn11O%2FO4nebKezL0DJodufy5fzbyo%3D&reserved=0>

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsdm.link%2Fslashdot&data=02%7C01%7Cjohnd.blackburn%40au.abb.com%7Cc787b84b26844913b97508d5b4ea0cbd%7C372ee9e09ce04033a64ac07073a91ecd%7C0%7C0%7C636613839702196446&sdata=Zors8Epzlh3PzfLMsZFS2YJi7PF0vlSIU67cNM0kkgY%3D&reserved=0>
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.opennms.org%2Findex.php%2FMailing_List_FAQ&data=02%7C01%7Cjohnd.blackburn%40au.abb.com%7Cc787b84b26844913b97508d5b4ea0cbd%7C372ee9e09ce04033a64ac07073a91ecd%7C0%7C0%7C636613839702196446&sdata=wQx7q%2F978h%2BZbm1Wuspcm86SBdfblK8YPwHf9JjjiHg%3D&reserved=0>

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fopennms-discuss&data=02%7C01%7Cjohnd.blackburn%40au.abb.com%7Cc787b84b26844913b97508d5b4ea0cbd%7C372ee9e09ce04033a64ac07073a91ecd%7C0%7C0%7C636613839702196446&sdata=KhgkZvn4jPR2Mzpn11O%2FO4nebKezL0DJodufy5fzbyo%3D&reserved=0>

Re: [opennms-discuss] OpenNMS monitoring and data collection from the Azure REST API

A Java based fault and performance management system

Re: [opennms-discuss] OpenNMS monitoring and data collection from the Azure REST API