Menu
▾
▴
yokoso-devel — Development list
You can subscribe to this list here.
| 2009 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(2) |
Jun
|
Jul
|
Aug
(17) |
Sep
(5) |
Oct
(4) |
Nov
|
Dec
|
|---|
|
From: Kevin J. <ke...@in...> - 2009-10-03 21:24:22
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 No problem... My brain has been disconnected for years! Kevin On Oct 3, 2009, at 1:29 PM, Jason Wood wrote: > Never mind... I stopped looking for SVN for Samurai and actually > looked for Yokoso instead. Much better results. > > Sorry about that, brain was disconnected. > > Jason -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iEYEARECAAYFAkrHwPUACgkQGDcWptZ2zmROmwCePySm6Q8Y9XqE95gqDofnptKc MpAAoP3/NeEyHogc9UIYeYnm+dSh+k9k =IqIj -----END PGP SIGNATURE----- |
|
From: Kevin J. <ke...@in...> - 2009-10-03 21:23:45
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Oct 3, 2009, at 1:11 PM, Jason Wood wrote: > I was looking around for the SVN server for Yokoso, but I haven't > found it yet. Is it publicly available and is it possible for me to > pull updates to my system? Hi- The SVN server is part of the Sourceforge hosting. You can find it at https://sourceforge.net/projects/yokoso/develop and yes it allows you to check out the code. Kevin Kevin Johnson Senior Security Analyst InGuardians, Inc. office: 202.448.8958 cell: 904.403.8024 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iEYEARECAAYFAkrHwMgACgkQGDcWptZ2zmQFhgCfeXtB6UWIfGw+CN8PEf4/776u n38AoPq+i/FWf/mKgdROUKqMCIZ3Gh0p =iqj7 -----END PGP SIGNATURE----- |
|
From: Jason W. <ja...@jw...> - 2009-10-03 17:29:52
|
Never mind... I stopped looking for SVN for Samurai and actually looked for Yokoso instead. Much better results. Sorry about that, brain was disconnected. Jason On Oct 3, 2009, at 11:11 AM, Jason Wood wrote: > I was looking around for the SVN server for Yokoso, but I haven't > found it yet. Is it publicly available and is it possible for me to > pull updates to my system? > > Thanks, > Jason > > ------------------------------------------------------------------------------ > Come build with us! The BlackBerry® Developer Conference in SF, CA > is the only developer event you need to attend this year. Jumpstart > your > developing skills, take BlackBerry mobile applications to market and > stay > ahead of the curve. Join us from November 9-12, 2009. Register > now! > http://p.sf.net/sfu/devconf > _______________________________________________ > Yokoso-devel mailing list > Yok...@li... > https://lists.sourceforge.net/lists/listinfo/yokoso-devel > |
|
From: Jason W. <ja...@jw...> - 2009-10-03 17:11:57
|
I was looking around for the SVN server for Yokoso, but I haven't found it yet. Is it publicly available and is it possible for me to pull updates to my system? Thanks, Jason |
|
From: Ron <ro...@sk...> - 2009-09-06 15:26:54
|
No problem! My favourite part is when an admin got over 900 emails about failed connections to an APC device. Oops :) Additionally, I'm happy with the current version of the Nmap script. It's made it easy to find/test fingerprints. Sometime soon I'll write a blog about how to use Nmap to find interesting servers and how to make fingerprints for them. I'll post a link here when I do that. Thanks! Ron On 09/06/2009 09:45 AM, Kevin Johnson wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > I have checked these into SVN. I really appreciate your help with the > project! > > Kevin > > On Sep 5, 2009, at 12:23 PM, Ron wrote: > >> Hi all, >> >> I've attached a diff that can be applied against the current .tgz on >> the yokoso site. It adds all the fingerprints I've discovered so far, >> and covers most of the technology I have access to. >> > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.9 (Darwin) > > iEYEARECAAYFAkqjyw8ACgkQGDcWptZ2zmTbiACg9u0EpzoIDk1OjYPfUASya4t7 > cVMAn2LWpiK/zwkR4SXr5TGK1A0CotxF > =v7uY > -----END PGP SIGNATURE----- -- Ron Bowes http://www.skullsecurity.org/ |
|
From: Kevin J. <ke...@in...> - 2009-09-06 14:45:55
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I have checked these into SVN. I really appreciate your help with the project! Kevin On Sep 5, 2009, at 12:23 PM, Ron wrote: > Hi all, > > I've attached a diff that can be applied against the current .tgz on > the yokoso site. It adds all the fingerprints I've discovered so > far, and covers most of the technology I have access to. > -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iEYEARECAAYFAkqjyw8ACgkQGDcWptZ2zmTbiACg9u0EpzoIDk1OjYPfUASya4t7 cVMAn2LWpiK/zwkR4SXr5TGK1A0CotxF =v7uY -----END PGP SIGNATURE----- |
|
From: Kevin J. <ke...@in...> - 2009-09-06 02:58:58
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I just got the initial release put into subversion... I will be working on the 0.2 release based on the fingerprints you guys have been sending in.... Kevin Kevin Johnson Senior Security Analyst InGuardians, Inc. office: 202.448.8958 cell: 904.403.8024 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iEYEARECAAYFAkqjJWIACgkQGDcWptZ2zmRw7QCdGnHGLB2thwSE6PPPATysQcu5 PasAoIVYwnvu6QCMqwgaliKjdvWyV7hR =/cFR -----END PGP SIGNATURE----- |
|
From: Ron <ro...@sk...> - 2009-09-05 16:23:27
|
Hi all, I've attached a diff that can be applied against the current .tgz on the yokoso site. It adds all the fingerprints I've discovered so far, and covers most of the technology I have access to. -- Ron Bowes http://www.skullsecurity.org/ |
|
From: Kevin J. <ke...@in...> - 2009-09-05 16:14:33
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 > I'm currently working on adding some additional functionality to > Laudanum. I'm wondering what the best approach is in terms of file > structure. > > Should I add to the existing shell page, should there be a "utility" > page for selecting prebuilt chained commands/functionality, or > should each function have it's own page? I like the idea of separate utility pages. I am hoping to get things checked into SVN tonight. If you would like, we can provide SVN access and add you to the project? > > Examples of functions I'm working on are automatically searching for > and attempting to display commonly grabbed and sensitive files such > as passwd, shadow, ssl keys, ssh keys, etc. Another is providing an > IP address and injecting BeEF hooks in to the existing web pages. I think both of these are excellent. We need to make sure that we continue the scope limiting features. So for example, the BeEF injection hook could also include the IP limiting feature so that it would only inject if the requester was from an IP in scope? Make sense? Kevin Kevin Johnson Senior Security Analyst InGuardians, Inc. office: 202.448.8958 cell: 904.403.8024 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iEYEARECAAYFAkqijlYACgkQGDcWptZ2zmQPxACfckAafDRxP7M1oJpOxG0cMeSY XJcAnjLms83fvwYtWP92M2ohw1XZyCCY =QZhV -----END PGP SIGNATURE----- |
|
From: Kevin J. <ke...@in...> - 2009-08-24 01:14:23
|
Woot! thanks Ron Kevin Begin forwarded message: > From: Ron <ro...@sk...> > Date: August 22, 2009 7:21:19 PM EDT > To: nma...@in... > Subject: Re: Updates to http-enum.nse > > Committed this in r15223, along with the changes Fyodor requested > and two fingerprints files: the ones that were previously hardcoded, > and yokoso's. > > See the commit log for all the changes, there were a bunch. > > Please test this if you have the time! I plan to continue working on > this till it's perfect, but feedback would be great! > > -- > Ron Bowes > http://www.skullsecurity.org/ > > _______________________________________________ > Sent through the nmap-dev mailing list > http://cgi.insecure.org/mailman/listinfo/nmap-dev > Archived at http://SecLists.Org > Kevin Johnson Senior Security Analyst InGuardians, Inc. office: 202.448.8958 cell: 904.403.8024 |
|
From: Ron <ro...@sk...> - 2009-08-22 16:36:29
|
On 08/22/2009 11:31 AM, Ron wrote: > I was thinking, besides Nikto, there's a project by OWASP whose name > totally escapes me, that's designed for finding hidden directories. Even > their smallest list of common directories is insanely huge. Once I > figure out what the project was called (somebody want to help me out?) > I'll add support for parsing their format, as well. We can end up with a > few different database formats in the end, let people use whatever they > want. Found it: DirBuster. http://www.owasp.org/index.php/Category:OWASP_DirBuster_Project I don't it's worthwhile including their database, though, just supporting it. Their smallest list is over 80,000 entries. :) Ron -- Ron Bowes http://www.skullsecurity.org/ |
|
From: Ron <ro...@sk...> - 2009-08-22 16:31:33
|
On 08/22/2009 01:47 AM, Fyodor wrote: > Sounds reasonable. If there are cases where we want to show other > status codes, perhaps the signature line format could be modified to > permit that. Sounds good. For now, I'm not going to worry about it, I'll just do the 200 and stick with the current database format. I was thinking, besides Nikto, there's a project by OWASP whose name totally escapes me, that's designed for finding hidden directories. Even their smallest list of common directories is insanely huge. Once I figure out what the project was called (somebody want to help me out?) I'll add support for parsing their format, as well. We can end up with a few different database formats in the end, let people use whatever they want. > That sounds fine to me. It sounds like you've already tested it quite > a bit. The Yokoso file is now fine from a copyright perspective > (subject to ther permissions text being added). However, it sounds > like some of the improvements we've discussed may be warranted before > inclusion of that part. All right, I'll do that. I've already made the improvements we talked about (the benefits of Friday-night plans being canceled at the last minute :) ), but I'm trying to chase down a hard-to-find assertion failure in Nsock right now. I'll post that as soon as I can reproduce it with a debugger attached. :) > > Cheers, > -F Ron -- Ron Bowes http://www.skullsecurity.org/ |
|
From: Kevin J. <ke...@in...> - 2009-08-21 16:38:53
|
On Aug 21, 2009, at 10:09 AM, Ron wrote: > Hi all, > Hi All, I am one of the lead developers for Yokoso! so I thought I should respond to some of these points here. > I'm cross-posting this to nmap-dev and yokoso-devel, since it sort of > affects both the products. I don't really know what etiquette is > regarding cross-posting, so hopefully that isn't a no-no. :) I don't mind it at all. And am also cross-posting. ;-) > > On 08/21/2009 02:49 AM, Fyodor wrote: >> On Thu, Aug 20, 2009 at 11:57:43AM -0500, Ron wrote: >>> > I agree that the script should only show 200 by default, but give an > option to show others. That makes sense. You're going to miss hidden > folders, by by default Apache hides certain things so that kinda makes > sense. > > For the average person, just giving 200 makes sense, and for the > non-average who understand the extra errors, they can enable a > script-arg. > This idea makes sense to me. >> The format of the fingerprint file is a bit questionable. Comments >> lines starting with '#' are parsed and then printed in the script >> output when paths given later in the file are discovered. I realize >> you didn't invent this format, but it is so simple that it could >> easily be improved. For example, each path could include the >> description on the same line. Or there could be a keyword >> introducing >> the description on one line, followed by the paths on their own lines >> as they are now. Then we could use comments for notes which we don't >> want parsed and printed by http-enum. In deciding on the format, it >> may be worth thinking about how it could be extended if we want to >> include more information later (just as an idea off the top of my >> head, we might want to later indicate what status code we look for to >> address issues such as http-enum reporting that scanme.nmap.org has >> "TeraStation PRO RAID 0/1/5 Network Attached Storage" just because >> /cgi-bin/image/shikaku2.png shows forbidden). > The format is a bit odd, and I'd talked to Kevin a bit about > changing it > to have the ability to AND things together (so if a host contains > these > 4 files that have common names, but are unique as a combo, it'll check > for all of them). At the same time, maybe we can consider other format > changes. Maybe I'll think about it and propose something, see where > that > goes. > > Regarding including the status code, that is sort of an issue with the > different purposes of Yokoso vs Nmap. Yokoso doesn't care what the > status code is, but rather a binary: the user has been to the page, or > the user hasn't. Nmap, on the other hand, cares of the page exists. > I'm > sure there must be a way to get the best of both worlds, of course -- > maybe optional fields or something? As the guy who created the "format" of the fingerprint file, I think its important to realize that it was just a file. I like the idea of something similar to the nikto DB where we include the status code we expect and a description on the same line. Comments could then be used for notes to end users and ignored by scripts. > >> Another serious issue involves inclusion of the Yokoso DB. You say: >> >>> That last point is the interesting one, to me -- we use the same >>> file >>> format as the Yokoso project (by Kevin Johnson and others, from >>> Intel >>> Guardians). BTW its InGuardians. Not relevant to this discussion but important to me. :-) >>> This lets us leverage their fingerprints as well (and >>> they've given me permission to include a copy of their fingerprints >>> file, too, >> >> That was nice of them, but it is important to get more clarification >> and more explicit permission whenever we include 3rd party code or >> data into Nmap. I hate dealing with copyright stuff as much as the >> next guy, but we really need to be very careful about this sort of >> thing. When they say we can include the DB with Nmap, what does that >> really mean. Remember that Nmap is open source, so people can >> incorporate parts into other projects or fork Nmap under a different >> name. A strict reading of "you may include this file within Nmap" >> would not allow such things, which would mean that part of Nmap is >> not >> open source. Also, a strict reading might mean that we can only >> include the file and not modify it (create derivative works). In >> general, we can put third party code/data in with Nmap if it is given >> to us under one of the following licenses (either via special >> permission or because the code is already under such a license): >> >> o Public domain -- that means people can do whatever they want with >> it. >> >> o BSD-style (includes MIT license, Lua license, etc.) - preferably >> 2-clause variant. If it has the advertising clause, we need to >> mention it explicitly in the man page >> (http://nmap.org/book/man-legal.html#third-party-soft) and >> potentially other places. >> >> o Nmap license - if they're OK with us distributing it under the >> terms >> of the Nmap license (http://nmap.org/data/COPYING), that is OK too. >> >> So if they let us use the data under one of these licenses, inclusion >> with Nmap is OK. In any case, the license permission granted needs >> to >> be included (described) at the top of the file. We only need license >> rights to the list of URL paths and descriptions, and not the rest of >> Yokoso. >> >> Note that even when a data file isn't licensed appropriately for >> distribution with Nmap, we can generally point to it in the >> documentation (e.g. if it is a URL somewhere) so users can download >> and use it. > I'll leave that question for the Yokoso guys to answer. We currently use the GPL for Yokoso! but have no problems with the NMAP license and would love to see it included with that in NMAP. > >>> One thought I had -- http-enum.nse and Yokoso sort of have different >>> points. http-enum.nse is designed for finding common locations, like >>> /icons, /scripts, /test, etc, and Yokoso is designed for >>> fingerprinting >>> common web apps. So, for that reason, it might make sense to put >>> it in a >>> different script that the user can run separately. Or maybe not. I'm >>> happy with going either way. >> >> I'm not sure, but my gut reaction is that with a good file format >> (which doesn't have to be too complex), we could probably combine the >> Yokoso DB with the existing enum DB. There is also a DB by HD Moore >> (a NASL script he wrote) which I hope to request permission for us to >> use. >> >> If we end up with more URLs than we want to scan by default, we could >> look at either splitting it up into multiple scripts or introducing >> NSE arguments to select categories of paths to try. > You're probably right about combining the databases. My concern is > that > it'll make it more difficult to update when Yokoso is updated -- > Yokoso > promises to be an active project, and if they're updating the > fingerprints and we've combined the fingerprints, we might run into > versioning problems. > > I think the best bet is to have multiple fingerprint files of the same > format. Yokoso, defaults, extended, etc. Then the script can load > whatever it sees fit. > > I'd definitely like to use HD's database if it's for the same purpose! I can take responsibility to keep the yokoso fingerprints updated in the nmap file if you would like! Thanks for including us, and if there are more questions feel free to ask! Kevin Kevin Johnson Senior Security Analyst InGuardians, Inc. office: 202.448.8958 cell: 904.403.8024 |
|
From: Ron <ro...@sk...> - 2009-08-21 14:09:13
|
Hi all, I'm cross-posting this to nmap-dev and yokoso-devel, since it sort of affects both the products. I don't really know what etiquette is regarding cross-posting, so hopefully that isn't a no-no. :) On 08/21/2009 02:49 AM, Fyodor wrote: > On Thu, Aug 20, 2009 at 11:57:43AM -0500, Ron wrote: >> >> Me and one of my minions at work (Andrew -- same guy who I did the iis >> unicode script with) have put a lot of work into improving http-enum.nse >> (in case that wasn't obvious from all the http.lua errors I've been >> posting). Rob's script was a great start, but we made a ton of improvements: >> - Cleaned up the code, put a bunch of it into functions >> - Support for many more HTTP status codes >> - Improved detection for 404 pages (especially those that return 200) -- >> we still have some more work to do on this, but it's getting there >> - More intelligent usage of HEAD vs. GET requests >> - Ability to parse external fingerprint file (attached) > > Thanks Ron and Andrew! Those sound like exciting changes! It worked > well in my testing, though the results against scanme.nmap.org are > basically false positives (we might wan to consider only showing 200 > results by default--I'm not sure). I agree that the script should only show 200 by default, but give an option to show others. That makes sense. You're going to miss hidden folders, by by default Apache hides certain things so that kinda makes sense. For the average person, just giving 200 makes sense, and for the non-average who understand the extra errors, they can enable a script-arg. > The format of the fingerprint file is a bit questionable. Comments > lines starting with '#' are parsed and then printed in the script > output when paths given later in the file are discovered. I realize > you didn't invent this format, but it is so simple that it could > easily be improved. For example, each path could include the > description on the same line. Or there could be a keyword introducing > the description on one line, followed by the paths on their own lines > as they are now. Then we could use comments for notes which we don't > want parsed and printed by http-enum. In deciding on the format, it > may be worth thinking about how it could be extended if we want to > include more information later (just as an idea off the top of my > head, we might want to later indicate what status code we look for to > address issues such as http-enum reporting that scanme.nmap.org has > "TeraStation PRO RAID 0/1/5 Network Attached Storage" just because > /cgi-bin/image/shikaku2.png shows forbidden). The format is a bit odd, and I'd talked to Kevin a bit about changing it to have the ability to AND things together (so if a host contains these 4 files that have common names, but are unique as a combo, it'll check for all of them). At the same time, maybe we can consider other format changes. Maybe I'll think about it and propose something, see where that goes. Regarding including the status code, that is sort of an issue with the different purposes of Yokoso vs Nmap. Yokoso doesn't care what the status code is, but rather a binary: the user has been to the page, or the user hasn't. Nmap, on the other hand, cares of the page exists. I'm sure there must be a way to get the best of both worlds, of course -- maybe optional fields or something? > Another serious issue involves inclusion of the Yokoso DB. You say: > >> That last point is the interesting one, to me -- we use the same file >> format as the Yokoso project (by Kevin Johnson and others, from Intel >> Guardians). This lets us leverage their fingerprints as well (and >> they've given me permission to include a copy of their fingerprints >> file, too, > > That was nice of them, but it is important to get more clarification > and more explicit permission whenever we include 3rd party code or > data into Nmap. I hate dealing with copyright stuff as much as the > next guy, but we really need to be very careful about this sort of > thing. When they say we can include the DB with Nmap, what does that > really mean. Remember that Nmap is open source, so people can > incorporate parts into other projects or fork Nmap under a different > name. A strict reading of "you may include this file within Nmap" > would not allow such things, which would mean that part of Nmap is not > open source. Also, a strict reading might mean that we can only > include the file and not modify it (create derivative works). In > general, we can put third party code/data in with Nmap if it is given > to us under one of the following licenses (either via special > permission or because the code is already under such a license): > > o Public domain -- that means people can do whatever they want with it. > > o BSD-style (includes MIT license, Lua license, etc.) - preferably > 2-clause variant. If it has the advertising clause, we need to > mention it explicitly in the man page > (http://nmap.org/book/man-legal.html#third-party-soft) and > potentially other places. > > o Nmap license - if they're OK with us distributing it under the terms > of the Nmap license (http://nmap.org/data/COPYING), that is OK too. > > So if they let us use the data under one of these licenses, inclusion > with Nmap is OK. In any case, the license permission granted needs to > be included (described) at the top of the file. We only need license > rights to the list of URL paths and descriptions, and not the rest of > Yokoso. > > Note that even when a data file isn't licensed appropriately for > distribution with Nmap, we can generally point to it in the > documentation (e.g. if it is a URL somewhere) so users can download > and use it. I'll leave that question for the Yokoso guys to answer. >> One thought I had -- http-enum.nse and Yokoso sort of have different >> points. http-enum.nse is designed for finding common locations, like >> /icons, /scripts, /test, etc, and Yokoso is designed for fingerprinting >> common web apps. So, for that reason, it might make sense to put it in a >> different script that the user can run separately. Or maybe not. I'm >> happy with going either way. > > I'm not sure, but my gut reaction is that with a good file format > (which doesn't have to be too complex), we could probably combine the > Yokoso DB with the existing enum DB. There is also a DB by HD Moore > (a NASL script he wrote) which I hope to request permission for us to > use. > > If we end up with more URLs than we want to scan by default, we could > look at either splitting it up into multiple scripts or introducing > NSE arguments to select categories of paths to try. You're probably right about combining the databases. My concern is that it'll make it more difficult to update when Yokoso is updated -- Yokoso promises to be an active project, and if they're updating the fingerprints and we've combined the fingerprints, we might run into versioning problems. I think the best bet is to have multiple fingerprint files of the same format. Yokoso, defaults, extended, etc. Then the script can load whatever it sees fit. I'd definitely like to use HD's database if it's for the same purpose! > >> I plan to move the hardcoded tests from http-enum.nse into their own >> file, too, once I'm happy with how it's working. > > Or maybe one combined file. We'll see. :) > > Thanks again for all your efforts on this! No problem! Is it ok if I commit what I have in http-enum.nse? I'll leave out the yokoso file for now, and provide instructions to download it. I strongly suspect that the current http-enum.nse works at least as well as the previous version. > > Cheers, > -F > Thanks! Ron |
|
From: Ron <ro...@sk...> - 2009-08-21 12:43:27
|
Hey guys, I'm not sure if any of you read the nmap-dev mailing list, but I posted the updated version of http-enum.nse (including Yokoso stuff). Fyodor's response (and comments) are below. Can you take a look at them (especially the file format and licensing bits) and comment? We've talked about changing the file format a bit to accommodate AND-type stuff. When we do that, and before too many things use Yokoso fingerprints, can we look at other options to improve the format? Thanks! Ron -------- Original Message -------- Subject: Re: Updates to http-enum.nse Date: Fri, 21 Aug 2009 00:49:48 -0700 From: Fyodor <fy...@in...> To: Ron <ro...@sk...> CC: Nmap Dev <nma...@in...>, an...@an... On Thu, Aug 20, 2009 at 11:57:43AM -0500, Ron wrote: > > Me and one of my minions at work (Andrew -- same guy who I did the iis > unicode script with) have put a lot of work into improving http-enum.nse > (in case that wasn't obvious from all the http.lua errors I've been > posting). Rob's script was a great start, but we made a ton of improvements: > - Cleaned up the code, put a bunch of it into functions > - Support for many more HTTP status codes > - Improved detection for 404 pages (especially those that return 200) -- > we still have some more work to do on this, but it's getting there > - More intelligent usage of HEAD vs. GET requests > - Ability to parse external fingerprint file (attached) Thanks Ron and Andrew! Those sound like exciting changes! It worked well in my testing, though the results against scanme.nmap.org are basically false positives (we might wan to consider only showing 200 results by default--I'm not sure). The format of the fingerprint file is a bit questionable. Comments lines starting with '#' are parsed and then printed in the script output when paths given later in the file are discovered. I realize you didn't invent this format, but it is so simple that it could easily be improved. For example, each path could include the description on the same line. Or there could be a keyword introducing the description on one line, followed by the paths on their own lines as they are now. Then we could use comments for notes which we don't want parsed and printed by http-enum. In deciding on the format, it may be worth thinking about how it could be extended if we want to include more information later (just as an idea off the top of my head, we might want to later indicate what status code we look for to address issues such as http-enum reporting that scanme.nmap.org has "TeraStation PRO RAID 0/1/5 Network Attached Storage" just because /cgi-bin/image/shikaku2.png shows forbidden). Another serious issue involves inclusion of the Yokoso DB. You say: > That last point is the interesting one, to me -- we use the same file > format as the Yokoso project (by Kevin Johnson and others, from Intel > Guardians). This lets us leverage their fingerprints as well (and > they've given me permission to include a copy of their fingerprints > file, too, That was nice of them, but it is important to get more clarification and more explicit permission whenever we include 3rd party code or data into Nmap. I hate dealing with copyright stuff as much as the next guy, but we really need to be very careful about this sort of thing. When they say we can include the DB with Nmap, what does that really mean. Remember that Nmap is open source, so people can incorporate parts into other projects or fork Nmap under a different name. A strict reading of "you may include this file within Nmap" would not allow such things, which would mean that part of Nmap is not open source. Also, a strict reading might mean that we can only include the file and not modify it (create derivative works). In general, we can put third party code/data in with Nmap if it is given to us under one of the following licenses (either via special permission or because the code is already under such a license): o Public domain -- that means people can do whatever they want with it. o BSD-style (includes MIT license, Lua license, etc.) - preferably 2-clause variant. If it has the advertising clause, we need to mention it explicitly in the man page (http://nmap.org/book/man-legal.html#third-party-soft) and potentially other places. o Nmap license - if they're OK with us distributing it under the terms of the Nmap license (http://nmap.org/data/COPYING), that is OK too. So if they let us use the data under one of these licenses, inclusion with Nmap is OK. In any case, the license permission granted needs to be included (described) at the top of the file. We only need license rights to the list of URL paths and descriptions, and not the rest of Yokoso. Note that even when a data file isn't licensed appropriately for distribution with Nmap, we can generally point to it in the documentation (e.g. if it is a URL somewhere) so users can download and use it. > For those who were there (and I know several of you were, because I > was sitting with you :) ), there was a presentation about Yokoso at > Defcon. Haha, yes that was a good talk :). > One thought I had -- http-enum.nse and Yokoso sort of have different > points. http-enum.nse is designed for finding common locations, like > /icons, /scripts, /test, etc, and Yokoso is designed for fingerprinting > common web apps. So, for that reason, it might make sense to put it in a > different script that the user can run separately. Or maybe not. I'm > happy with going either way. I'm not sure, but my gut reaction is that with a good file format (which doesn't have to be too complex), we could probably combine the Yokoso DB with the existing enum DB. There is also a DB by HD Moore (a NASL script he wrote) which I hope to request permission for us to use. If we end up with more URLs than we want to scan by default, we could look at either splitting it up into multiple scripts or introducing NSE arguments to select categories of paths to try. > I plan to move the hardcoded tests from http-enum.nse into their own > file, too, once I'm happy with how it's working. Or maybe one combined file. Thanks again for all your efforts on this! Cheers, -F |
|
From: Ron <ro...@sk...> - 2009-08-18 18:41:03
|
> A diff is always easier, but this works for now. We really appreciate > all the work you are doing! Sure, we can re-send it as a diff if that would help. We're probably going to continue adding to it, so if you think what we've done is ok then you can probably just ignore it for now. > We should put it back as I have seen files that appear on one but not > another based on version, but if we don't know the exact version..... You're probably right. The reason it was removed in ours was a theming thing -- it was replaced with a link to a custom image. At the time, we were considering it from an ANDing perspective, and that one wouldn't work well. We'll put it back. > I like the generic option for the overlap and then split out when needed. Any thoughts on possible formats? We were thinking of simply space-delimited URLs on a single line. Simple and effective. > Something to think about.... Let me know. For now, we'll stick with what we see (en/EN). > Thanks > Kevin Ron |
|
From: Kevin J. <ke...@in...> - 2009-08-18 18:03:11
|
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Aug 17, 2009, at 6:58 PM, Ron wrote:
> Hi all,
>
> Today, me and one of our co-op students (Andrew, who I believe
> joined this list today) made significant improvements to the
> fingerprints file. I've attached the new version as a raw file, but
> let me know if a diff would be easier for you to work with.
A diff is always easier, but this works for now. We really appreciate
all the work you are doing!
> We did the Pre-Auth/Post-Auth of a bunch of our security tools, then
> started scanning our network on Port 80. We found hundreds of
> printers, plus some server utilities, network cameras, network
> drives, network cdrom drives (weird!), and other miscellaneous
> stuff. We barely scratched the surface, though, we should be adding
> more in the near future.
Looking forward to it!
>
> In addition to the stuff we found on our network, we also made a
> couple minor changes to your fingerprints:
> 1) Fixed a spelling mistake ("imags" => "images")
Thanks!
> 2) Removed a check from the Sharepoint section, since it was
> pointing at a file that wasn't present on our install (the remaining
> checks detect our Sharepoints nicely, though)
>
We should put it back as I have seen files that appear on one but not
another based on version, but if we don't know the exact version.....
> I mentioned in my previous reply that fingerprints should maybe all
> be ANDs, not ORs, but after collecting fingerprints I tend to
> disagree with my previous position. There are a lot of cases where
> we had to pick several images, some of which may or may not be
> present on the site page.
Makes sense.
>
> I still agree with the need for an AND scenario, though.
>
> There are some cases where we weren't sure how specific/general to
> get. Mainly, we have every version of HP printer you can imagine.
> For now, we separated them when possible into the different
> fingerprints, but it seems to me that the versions overlap a lot, so
> we might want to do one generic "HP Printer" section. Any thoughts
> on that? I noticed with your fingerprints, they were specific to the
> model of the printer.
I like the generic option for the overlap and then split out when
needed.
>
> Another thing we noticed was that some applications use theming
> based on language. So, the most representative images we could find
> were in /en/ or /EN/. I'm guessing that could also be de/fr/es/etc,
> depending on the language. I'm not sure if you want to handle that
> in any special way, like adding a generic "2-digit-country-code" tag
> or something.
Something to think about....
Thanks
Kevin
Kevin Johnson
Senior Security Analyst
InGuardians, Inc.
office: 202.448.8958
cell: 904.403.8024
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)
iEYEARECAAYFAkqK7MQACgkQGDcWptZ2zmSo1ACgsXS7YK/cslHDehsfawHNR+QD
4SoAn2h9CJH38hbaWaqE4LtsM0+nFDvn
=DHXI
-----END PGP SIGNATURE-----
|
|
From: Ron <ro...@sk...> - 2009-08-18 02:56:22
|
Hi all, I rewrote a huge chunk of the http-enum.nse script today. Hopefully in the process I fixed the bugs/false positives I saw. I've attached the new version, and I uploaded it here: http://www.skullsecurity.org/tmp/http-enum.nse Don't forget to check out my updated fingerprints file! Ron -- Ron Bowes http://www.skullsecurity.org/ |
|
From: Ron <ro...@sk...> - 2009-08-17 22:59:50
|
We tested this script today a little, after adding our new fingerprints, and it seems to work well but we occasionally got false positives. On one server specifically, every folder showed up as present, which is something we were hoping to avoid. We also get errors once in awhile that I'll have to look into. Overall, however, it seems to work well. Let me know what you think! Ron On 08/17/2009 08:13 AM, Justin Searle wrote: > Thanks Ron. I'll check it out later tonight. > > > On Aug 16, 2009, at 10:47 PM, Ron wrote: > >> Hi all, >> >> I just finished adding the ability to parse the yokoso fingerprint >> file to the http-enum.nse script in Nmap. To use it: >> >> 1) Place Yokoso's "fingerprints" file, or a link to it, in Nmap's >> nselib/data directory (/usr/local/share/nmap/nselib/data by default, I >> think.. your mileage may vary). >> >> 2) Replace your copy of http-enum.nse with the copy I've attached (in >> case the attachment gets eaten by the listserv, you can get it from >> http://www.skullsecurity.org/tmp/http-enum.nse ) >> >> 3) Run Nmap with the following command: >> nmap --script=http-enum -p80,443 www.javaop.com >> >> (www.javaop.com is my site, and I put some randomly-chosen test >> folders there) >> >> Here's the output I get: >> - >> $ ./nmap --script=http-enum -p80,443 www.javaop.com >> >> Starting Nmap 5.05BETA1 ( http://nmap.org ) at 2009-08-16 21:44 CDT >> NSE: Script Scanning completed. >> Interesting ports on dsl-208-81-2-52.les.net (208.81.2.52): >> PORT STATE SERVICE >> 80/tcp open http >> | http-enum: /icons/ Icons directory >> | /images/ Images directory >> | /sw/auth/login.aspx Citrix WebTop >> | /images/outlook.jpg Outlook Web Access >> | /nfservlets/servlet/SPSRouterServlet/ netForensics >> |_ /nfservlets/servlet/SPSRouterServlet/ netForensics >> 443/tcp filtered https >> >> Nmap done: 1 IP address (1 host up) scanned in 2.94 seconds >> - >> >> If it doesn't work for you, please run with debug enabled (-d) and >> send me all the output. >> >> If you have any other suggestions, please let me know. I haven't sent >> this to the Nmap list just yet, I wanted to get your opinions/blessing >> first. I don't really know the best way to distribute it. Kevin had >> mentioned he'd like to include the script with Yokoso, and I'd like to >> include your fingerprints file with Nmap, if that's ok. I'm sort of >> worried if we do both, we'll end up with versioning issues. But we'll >> see. >> >> Thanks, looking forward to your thoughts! >> Ron >> >> -- >> Ron Bowes >> http://www.skullsecurity.org/ >> <http-enum.nse>------------------------------------------------------------------------------ >> >> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 >> 30-Day >> trial. Simplify your report design, integration and deployment - and >> focus on >> what you do best, core application coding. Discover what's new with >> Crystal Reports now. >> http://p.sf.net/sfu/bobj-july_______________________________________________ >> >> Yokoso-devel mailing list >> Yok...@li... >> https://lists.sourceforge.net/lists/listinfo/yokoso-devel > > Justin Searle > Senior Security Analyst - InGuardians, Inc. > ju...@in... > Direct: 801-784-2052 > Fax: 202-318-0235 > -- Ron Bowes http://www.skullsecurity.org/ |
|
From: Ron <ro...@sk...> - 2009-08-17 22:58:35
|
Hi all,
Today, me and one of our co-op students (Andrew, who I believe joined
this list today) made significant improvements to the fingerprints file.
I've attached the new version as a raw file, but let me know if a diff
would be easier for you to work with. We did the Pre-Auth/Post-Auth of a
bunch of our security tools, then started scanning our network on Port
80. We found hundreds of printers, plus some server utilities, network
cameras, network drives, network cdrom drives (weird!), and other
miscellaneous stuff. We barely scratched the surface, though, we should
be adding more in the near future.
In addition to the stuff we found on our network, we also made a couple
minor changes to your fingerprints:
1) Fixed a spelling mistake ("imags" => "images")
2) Removed a check from the Sharepoint section, since it was pointing at
a file that wasn't present on our install (the remaining checks detect
our Sharepoints nicely, though)
I mentioned in my previous reply that fingerprints should maybe all be
ANDs, not ORs, but after collecting fingerprints I tend to disagree with
my previous position. There are a lot of cases where we had to pick
several images, some of which may or may not be present on the site page.
I still agree with the need for an AND scenario, though.
There are some cases where we weren't sure how specific/general to get.
Mainly, we have every version of HP printer you can imagine. For now, we
separated them when possible into the different fingerprints, but it
seems to me that the versions overlap a lot, so we might want to do one
generic "HP Printer" section. Any thoughts on that? I noticed with your
fingerprints, they were specific to the model of the printer.
Another thing we noticed was that some applications use theming based on
language. So, the most representative images we could find were in /en/
or /EN/. I'm guessing that could also be de/fr/es/etc, depending on the
language. I'm not sure if you want to handle that in any special way,
like adding a generic "2-digit-country-code" tag or something.
Ron
--
Ron Bowes
http://www.skullsecurity.org/
|
|
From: Ron <ro...@sk...> - 2009-08-17 17:45:10
|
On 08/17/2009 12:26 PM, Kevin Johnson wrote: > The plan so far was to OR them, but I can see the need for an AND. > > We need to think about the architecture of the fingerprints... for > example some of them are on a default port outside of 80. Maybe we could > put multiple fingerprints on a line for ANDs? > > ideas? > > Kevin I don't like the idea of complicating it any more than necessary. Honestly, I don't see a purpose for ORing them at all. When do you think it ever be preferable to ANDing them? I can personally see two reasons, but neither of them seem compelling to me. 1) Different versions of software might change a link or two 2) It's faster to short circuit after one hit I don't think either is a good enough reason, though. I'd say moving to AND would be the best in the end. Ron |
|
From: Kevin J. <ke...@in...> - 2009-08-17 17:26:24
|
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Aug 17, 2009, at 12:32 PM, Ron wrote:
> Me and one of my co-op students were looking at some of our web apps
> to
> get fingerprints. Specifically, we were looking at "Secunia". The
> issue
> with Secunia is that all the files are very generic ("left-
> border.gif",
> "default.css", etc). While each file is generic, the combination of
> them
> is likely unique.
>
> But that made me realize, are your signatures intended to be ANDed or
> ORed? Right now, I implemented it with an OR -- if any one signature
> matches, it's a match. Implementing it as an AND, where all files must
> be present, would be somewhat more difficult.
>
> Which way do you suggest?
The plan so far was to OR them, but I can see the need for an AND.
We need to think about the architecture of the fingerprints... for
example some of them are on a default port outside of 80. Maybe we
could put multiple fingerprints on a line for ANDs?
ideas?
Kevin
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)
iEYEARECAAYFAkqJkqkACgkQGDcWptZ2zmRoogCdG28lSqXofEoWb08OQxlfU3nP
Me0AniMZjrQLHk7Hj6MF3WMIVtr8ypkH
=lRYm
-----END PGP SIGNATURE-----
|
|
From: Ron <ro...@sk...> - 2009-08-17 16:32:29
|
Me and one of my co-op students were looking at some of our web apps to
get fingerprints. Specifically, we were looking at "Secunia". The issue
with Secunia is that all the files are very generic ("left-border.gif",
"default.css", etc). While each file is generic, the combination of them
is likely unique.
But that made me realize, are your signatures intended to be ANDed or
ORed? Right now, I implemented it with an OR -- if any one signature
matches, it's a match. Implementing it as an AND, where all files must
be present, would be somewhat more difficult.
Which way do you suggest?
Ron
On 08/17/2009 08:13 AM, Justin Searle wrote:
> Thanks Ron. I'll check it out later tonight.
>
>
> On Aug 16, 2009, at 10:47 PM, Ron wrote:
>
>> Hi all,
>>
>> I just finished adding the ability to parse the yokoso fingerprint
>> file to the http-enum.nse script in Nmap. To use it:
>>
>> 1) Place Yokoso's "fingerprints" file, or a link to it, in Nmap's
>> nselib/data directory (/usr/local/share/nmap/nselib/data by default, I
>> think.. your mileage may vary).
>>
>> 2) Replace your copy of http-enum.nse with the copy I've attached (in
>> case the attachment gets eaten by the listserv, you can get it from
>> http://www.skullsecurity.org/tmp/http-enum.nse )
>>
>> 3) Run Nmap with the following command:
>> nmap --script=http-enum -p80,443 www.javaop.com
>>
>> (www.javaop.com is my site, and I put some randomly-chosen test
>> folders there)
>>
>> Here's the output I get:
>> -
>> $ ./nmap --script=http-enum -p80,443 www.javaop.com
>>
>> Starting Nmap 5.05BETA1 ( http://nmap.org ) at 2009-08-16 21:44 CDT
>> NSE: Script Scanning completed.
>> Interesting ports on dsl-208-81-2-52.les.net (208.81.2.52):
>> PORT STATE SERVICE
>> 80/tcp open http
>> | http-enum: /icons/ Icons directory
>> | /images/ Images directory
>> | /sw/auth/login.aspx Citrix WebTop
>> | /images/outlook.jpg Outlook Web Access
>> | /nfservlets/servlet/SPSRouterServlet/ netForensics
>> |_ /nfservlets/servlet/SPSRouterServlet/ netForensics
>> 443/tcp filtered https
>>
>> Nmap done: 1 IP address (1 host up) scanned in 2.94 seconds
>> -
>>
>> If it doesn't work for you, please run with debug enabled (-d) and
>> send me all the output.
>>
>> If you have any other suggestions, please let me know. I haven't sent
>> this to the Nmap list just yet, I wanted to get your opinions/blessing
>> first. I don't really know the best way to distribute it. Kevin had
>> mentioned he'd like to include the script with Yokoso, and I'd like to
>> include your fingerprints file with Nmap, if that's ok. I'm sort of
>> worried if we do both, we'll end up with versioning issues. But we'll
>> see.
>>
>> Thanks, looking forward to your thoughts!
>> Ron
>>
>> --
>> Ron Bowes
>> http://www.skullsecurity.org/
>> <http-enum.nse>------------------------------------------------------------------------------
>>
>> Let Crystal Reports handle the reporting - Free Crystal Reports 2008
>> 30-Day
>> trial. Simplify your report design, integration and deployment - and
>> focus on
>> what you do best, core application coding. Discover what's new with
>> Crystal Reports now.
>> http://p.sf.net/sfu/bobj-july_______________________________________________
>>
>> Yokoso-devel mailing list
>> Yok...@li...
>> https://lists.sourceforge.net/lists/listinfo/yokoso-devel
>
> Justin Searle
> Senior Security Analyst - InGuardians, Inc.
> ju...@in...
> Direct: 801-784-2052
> Fax: 202-318-0235
>
|
|
From: Justin S. <ju...@in...> - 2009-08-17 13:14:09
|
Thanks Ron. I'll check it out later tonight. On Aug 16, 2009, at 10:47 PM, Ron wrote: > Hi all, > > I just finished adding the ability to parse the yokoso fingerprint > file to the http-enum.nse script in Nmap. To use it: > > 1) Place Yokoso's "fingerprints" file, or a link to it, in Nmap's > nselib/data directory (/usr/local/share/nmap/nselib/data by default, > I think.. your mileage may vary). > > 2) Replace your copy of http-enum.nse with the copy I've attached > (in case the attachment gets eaten by the listserv, you can get it > from http://www.skullsecurity.org/tmp/http-enum.nse ) > > 3) Run Nmap with the following command: > nmap --script=http-enum -p80,443 www.javaop.com > > (www.javaop.com is my site, and I put some randomly-chosen test > folders there) > > Here's the output I get: > - > $ ./nmap --script=http-enum -p80,443 www.javaop.com > > Starting Nmap 5.05BETA1 ( http://nmap.org ) at 2009-08-16 21:44 CDT > NSE: Script Scanning completed. > Interesting ports on dsl-208-81-2-52.les.net (208.81.2.52): > PORT STATE SERVICE > 80/tcp open http > | http-enum: /icons/ Icons directory > | /images/ Images directory > | /sw/auth/login.aspx Citrix WebTop > | /images/outlook.jpg Outlook Web Access > | /nfservlets/servlet/SPSRouterServlet/ netForensics > |_ /nfservlets/servlet/SPSRouterServlet/ netForensics > 443/tcp filtered https > > Nmap done: 1 IP address (1 host up) scanned in 2.94 seconds > - > > If it doesn't work for you, please run with debug enabled (-d) and > send me all the output. > > If you have any other suggestions, please let me know. I haven't > sent this to the Nmap list just yet, I wanted to get your opinions/ > blessing first. I don't really know the best way to distribute it. > Kevin had mentioned he'd like to include the script with Yokoso, and > I'd like to include your fingerprints file with Nmap, if that's ok. > I'm sort of worried if we do both, we'll end up with versioning > issues. But we'll see. > > Thanks, looking forward to your thoughts! > Ron > > -- > Ron Bowes > http://www.skullsecurity.org/ > <http- > enum > .nse > > > ------------------------------------------------------------------------------ > Let Crystal Reports handle the reporting - Free Crystal Reports 2008 > 30-Day > trial. Simplify your report design, integration and deployment - and > focus on > what you do best, core application coding. Discover what's new with > Crystal Reports now. http://p.sf.net/sfu/bobj-july_______________________________________________ > Yokoso-devel mailing list > Yok...@li... > https://lists.sourceforge.net/lists/listinfo/yokoso-devel Justin Searle Senior Security Analyst - InGuardians, Inc. ju...@in... Direct: 801-784-2052 Fax: 202-318-0235 |
|
From: Ron <ro...@sk...> - 2009-08-17 02:48:08
|
Hi all, I just finished adding the ability to parse the yokoso fingerprint file to the http-enum.nse script in Nmap. To use it: 1) Place Yokoso's "fingerprints" file, or a link to it, in Nmap's nselib/data directory (/usr/local/share/nmap/nselib/data by default, I think.. your mileage may vary). 2) Replace your copy of http-enum.nse with the copy I've attached (in case the attachment gets eaten by the listserv, you can get it from http://www.skullsecurity.org/tmp/http-enum.nse ) 3) Run Nmap with the following command: nmap --script=http-enum -p80,443 www.javaop.com (www.javaop.com is my site, and I put some randomly-chosen test folders there) Here's the output I get: - $ ./nmap --script=http-enum -p80,443 www.javaop.com Starting Nmap 5.05BETA1 ( http://nmap.org ) at 2009-08-16 21:44 CDT NSE: Script Scanning completed. Interesting ports on dsl-208-81-2-52.les.net (208.81.2.52): PORT STATE SERVICE 80/tcp open http | http-enum: /icons/ Icons directory | /images/ Images directory | /sw/auth/login.aspx Citrix WebTop | /images/outlook.jpg Outlook Web Access | /nfservlets/servlet/SPSRouterServlet/ netForensics |_ /nfservlets/servlet/SPSRouterServlet/ netForensics 443/tcp filtered https Nmap done: 1 IP address (1 host up) scanned in 2.94 seconds - If it doesn't work for you, please run with debug enabled (-d) and send me all the output. If you have any other suggestions, please let me know. I haven't sent this to the Nmap list just yet, I wanted to get your opinions/blessing first. I don't really know the best way to distribute it. Kevin had mentioned he'd like to include the script with Yokoso, and I'd like to include your fingerprints file with Nmap, if that's ok. I'm sort of worried if we do both, we'll end up with versioning issues. But we'll see. Thanks, looking forward to your thoughts! Ron -- Ron Bowes http://www.skullsecurity.org/ |