From: <ul...@us...> - 2009-08-04 20:57:37
|
Revision: 5 http://adc.svn.sourceforge.net/adc/?rev=5&view=rev Author: ullner Date: 2009-08-04 20:57:28 +0000 (Tue, 04 Aug 2009) Log Message: ----------- Added TS as MSG parameter. Modified Paths: -------------- trunk/ADC-EXT.txt Modified: trunk/ADC-EXT.txt =================================================================== --- trunk/ADC-EXT.txt 2008-05-02 12:33:52 UTC (rev 4) +++ trunk/ADC-EXT.txt 2009-08-04 20:57:28 UTC (rev 5) @@ -1,178 +1,186 @@ -= ADC Extensions -Jacek Sieka <arn...@gm...> -1.0, May 2008 - -== Abstract -These are the official extensions to ADC. This document is based on the -information contained in the ADC wiki - spefications from there are moved here -when they are mature and stable enough. - -== Version history -The latest draft of the next version of this document as well as intermediate -and older versions can be downloaded from -$URL$. - -This version correspods to $Revision$. - -=== Version 1.0 -* Initial release created from original ADC 1.0 text -* Added PING extension - -== Extensions - -=== TIGR - Tiger tree hash support - -==== General - -This extension adds Tiger tree hash support to the base protocol. It is -intended to be used both for identifying files and for purposes such as CID -generation and password negotiation - -==== TIGR for shared files -All files shared by TIGR supporting clients must have been hashed using Merkle -Hash trees, as defined by -http://www.open-content.net/specs/draft-jchapweske-thex-02.html. The Tiger -algorithm, as specified by http://www.cs.technion.ac.il/~biham/Reports/Tiger/, -functions as the hash algorithm. A base segment size of 1024 bytes must be -used when generating the tree, but clients may then discard parts of the tree -as long as at least 7 levels are kept or a block granularity of 64 KiB is -achieved. - -Generally, the root of the tree (TTH) serves to identify a file uniquely. -Searches use it and it must be present in the file list. Further, the root of -the file list must also be available and discoverable via GFI. A client may -also request the rest of the tree using the normal client-client transfer -procedure. The root must be encoded using base32 encoding when converted to -text. - -In the file list, each File element carries an additional attribute "TTH" -containing the base32-encoded value of the Tiger tree root. - -In the GET/GFI type, the full tree may be accessed using the "tthl" type. - -"tthl" transfers send the largest set of leaves available) as a binary stream -of leaf data, right-to-left, with no spacing in between them. <start_pos> -must be set to 0 and <bytes> to -1 when requesting the data. <bytes> must -contain the total binary size of the leaf stream in SND; by dividing this -length by the individual hash length, the number of leaves, and thus the leaf -level, can be deducted. The received leaves can then be used to reconstruct -the entire tree, and the resulting root must match the root of the file (this -verifies the integrity of the tree itself). Identifier must be a TTH root -value from the "TTH/" root. - -In the GET/GFI namespace, files are identified by -"TTH/<base32-encoded tree root>". - -In SCH and GFI, the following attributes are added: - -[separator="|"] -``_ -TR | Tiger tree Hash root, encoded with base32. -TD | Tree depth, index of the highest level of tree data available, root-only = 0, first level (2 leaves) = 1, second level = 2, etc... -___ - -=== BZIP – File list compressed with bzip2 -This extension adds a special file "files.xml.bz2" in the unnamed root of the -share which contains "files.xml" compressed with bzip2 1.0.3+ (www.bzip.org). - -=== ZLIB - Compressed communication -There are two variants of zlib support, FULL and GET, and only one should be -used on a each communications channel set up. - -==== ZLIB-FULL -If, during SUP negotiation, a peer sends "ZLIF" in its support string, it must -accept two additional commands, ZON and ZOF. Upon reception of ZON the peer -must start decompressing the incoming stream of data with zlib before -interpreting it, and stop doing so after ZOF is received (in the compressed -stream). The compressing end must partially flush the zlib buffer after each -chunk of data to allow for decompression by the peer. - -==== ZLIB-GET -The alternative is to send "ZLIG" to indicate that zlib is supported for -binary transfers using the GET command, but not otherwise. A flag "ZL1" is -added to the to the SND command to indicate that the data will come -compressed, and the client receiving requests it by adding the same flag to -GET (the sending client may ignore a request for a compressed transfer, but -may also use it even when not requested by the receiver). The <bytes> -parameter of the GET and SND commands is to be interpreted as the number of -uncompressed bytes to be transferred. - -=== PING - Pinger extension -Added 2008-03-14, based on ADC 1.0 - -This extension can be supported by both clients and hubs, and when present, if hub -supports it, it must send additional information to the client ( otherwise normal -base client). - -It's purpose is to send to hublist pingers additional information about the hub - that otherwise it would be impossible to get as a normal user (eg. minimum share, - maximum user count, etc). - -==== INF - -Contexts : F - -When the client supporting the PING extension connects, the hub must send its -normal INF along with the following added fields ( none mandatory, if not present, -it means hub has no restrictions in that matter, or non existent): - -[separator="|"] -```_ -Code | Type | Description -___ -HH | string | Hub Host address ( DNS or IP ) -WS | url | Hub Website -NE | string | Hub Network -OW | string | Hub Owner name -UC | integer | Current User count -SS | integer | Total share size -SF | integer | Total files shared -MS | integer | Minimum share required to enter hub ( bytes ) -XS | integer | Maximum share for entering hub ( bytes ) -ML | integer | Minimum slots required to enter hub -XL | integer | Maximum slots for entering hub -MU | integer | Minimum hubs connected where clients can be users -MR | integer | Minimum hubs connected where client can be registered -MO | integer | Minimum hubs connected where client can be operators -XU | integer | Maximum hubs connected where clients can be users -XR | integer | Maximum hubs connected where client can be registered -XO | integer | Maximum hubs connected where client can be operators -MC | integer | Maximum possible clients ( users ) who can connect -UP | integer | Hub uptime (seconds) -NI | string | Hub name (from BASE) -DE | string | Hub description (from BASE) -VE | string | Hub software version (from BASE) -___ - -The hub must continue to send the user list as for a normal user (move to -NORMAL state). The pinger may decide to go through or disconnect (eg. if it -doesn't require additional information about the users). - -===== Example ----- - -pinger- HSUP ADBASE ADPING AD.. - -hub- ISUP ADBASE ADPING AD.. - -hub- ISID .. - -hub- IINF NIhubname DEcurrent\stopic VE.. HHmyhub.no-ip.org:555 WShttp://myhub.no-ip.org/ OWmyname UC2231 SS.. SF.. MS0 ML0 MC5000 - - (pinger may disconnect) - -.. ----- - -==== Hub - Hublist communication - -The same extension goes for hub- hublist communication. This way, the hub -takes the role of the client and the hublist of the server. - -The hublist may send INF about itself with NI field which would become hublist -name and WS hublist web address. - -===== Example ----- - -hub- HSUP ADBASE ADPING AD.. - -hublist- ISUP ADBASE ADPING AD.. - -hublist- IINF NIhublist_name WShublist_address - -hub- HINF NIhubname DEcurrent\stopic VE.. HHmyhub.no-ip.org:555 WShttp://myhub.no-ip.org/ OWmyname UC2231 SS.. SF.. MS0 ML0 MC5000 - -( disconnect ) ----- - -// vim: set syntax=asciidoc: += ADC Extensions +Jacek Sieka <arn...@gm...> +1.0, May 2008 +Fredrik Ullner <ul...@gm...> +1.0.1 August 2009 + +== Abstract +These are the official extensions to ADC. This document is based on the +information contained in the ADC wiki - spefications from there are moved here +when they are mature and stable enough. + +== Version history +The latest draft of the next version of this document as well as intermediate +and older versions can be downloaded from +$URL$. + +This version correspods to $Revision$. + +=== Version 1.0 +* Initial release created from original ADC 1.0 text +* Added PING extension + +=== Version 1.0.1 +* Added TS as additional MSG parameter + +== Extensions + +=== TIGR - Tiger tree hash support + +==== General + +This extension adds Tiger tree hash support to the base protocol. It is +intended to be used both for identifying files and for purposes such as CID +generation and password negotiation + +==== TIGR for shared files +All files shared by TIGR supporting clients must have been hashed using Merkle +Hash trees, as defined by +http://www.open-content.net/specs/draft-jchapweske-thex-02.html. The Tiger +algorithm, as specified by http://www.cs.technion.ac.il/~biham/Reports/Tiger/, +functions as the hash algorithm. A base segment size of 1024 bytes must be +used when generating the tree, but clients may then discard parts of the tree +as long as at least 7 levels are kept or a block granularity of 64 KiB is +achieved. + +Generally, the root of the tree (TTH) serves to identify a file uniquely. +Searches use it and it must be present in the file list. Further, the root of +the file list must also be available and discoverable via GFI. A client may +also request the rest of the tree using the normal client-client transfer +procedure. The root must be encoded using base32 encoding when converted to +text. + +In the file list, each File element carries an additional attribute "TTH" +containing the base32-encoded value of the Tiger tree root. + +In the GET/GFI type, the full tree may be accessed using the "tthl" type. + +"tthl" transfers send the largest set of leaves available) as a binary stream +of leaf data, right-to-left, with no spacing in between them. <start_pos> +must be set to 0 and <bytes> to -1 when requesting the data. <bytes> must +contain the total binary size of the leaf stream in SND; by dividing this +length by the individual hash length, the number of leaves, and thus the leaf +level, can be deducted. The received leaves can then be used to reconstruct +the entire tree, and the resulting root must match the root of the file (this +verifies the integrity of the tree itself). Identifier must be a TTH root +value from the "TTH/" root. + +In the GET/GFI namespace, files are identified by +"TTH/<base32-encoded tree root>". + +In SCH and GFI, the following attributes are added: + +[separator="|"] +``_ +TR | Tiger tree Hash root, encoded with base32. +TD | Tree depth, index of the highest level of tree data available, root-only = 0, first level (2 leaves) = 1, second level = 2, etc... +___ + +=== BZIP – File list compressed with bzip2 +This extension adds a special file "files.xml.bz2" in the unnamed root of the +share which contains "files.xml" compressed with bzip2 1.0.3+ (www.bzip.org). + +=== ZLIB - Compressed communication +There are two variants of zlib support, FULL and GET, and only one should be +used on a each communications channel set up. + +==== ZLIB-FULL +If, during SUP negotiation, a peer sends "ZLIF" in its support string, it must +accept two additional commands, ZON and ZOF. Upon reception of ZON the peer +must start decompressing the incoming stream of data with zlib before +interpreting it, and stop doing so after ZOF is received (in the compressed +stream). The compressing end must partially flush the zlib buffer after each +chunk of data to allow for decompression by the peer. + +==== ZLIB-GET +The alternative is to send "ZLIG" to indicate that zlib is supported for +binary transfers using the GET command, but not otherwise. A flag "ZL1" is +added to the to the SND command to indicate that the data will come +compressed, and the client receiving requests it by adding the same flag to +GET (the sending client may ignore a request for a compressed transfer, but +may also use it even when not requested by the receiver). The <bytes> +parameter of the GET and SND commands is to be interpreted as the number of +uncompressed bytes to be transferred. + +=== PING - Pinger extension +Added 2008-03-14, based on ADC 1.0 + +This extension can be supported by both clients and hubs, and when present, if hub +supports it, it must send additional information to the client ( otherwise normal +base client). + +It's purpose is to send to hublist pingers additional information about the hub + that otherwise it would be impossible to get as a normal user (eg. minimum share, + maximum user count, etc). + +==== INF + +Contexts : F + +When the client supporting the PING extension connects, the hub must send its +normal INF along with the following added fields ( none mandatory, if not present, +it means hub has no restrictions in that matter, or non existent): + +[separator="|"] +```_ +Code | Type | Description +___ +HH | string | Hub Host address ( DNS or IP ) +WS | url | Hub Website +NE | string | Hub Network +OW | string | Hub Owner name +UC | integer | Current User count +SS | integer | Total share size +SF | integer | Total files shared +MS | integer | Minimum share required to enter hub ( bytes ) +XS | integer | Maximum share for entering hub ( bytes ) +ML | integer | Minimum slots required to enter hub +XL | integer | Maximum slots for entering hub +MU | integer | Minimum hubs connected where clients can be users +MR | integer | Minimum hubs connected where client can be registered +MO | integer | Minimum hubs connected where client can be operators +XU | integer | Maximum hubs connected where clients can be users +XR | integer | Maximum hubs connected where client can be registered +XO | integer | Maximum hubs connected where client can be operators +MC | integer | Maximum possible clients ( users ) who can connect +UP | integer | Hub uptime (seconds) +NI | string | Hub name (from BASE) +DE | string | Hub description (from BASE) +VE | string | Hub software version (from BASE) +___ + +The hub must continue to send the user list as for a normal user (move to +NORMAL state). The pinger may decide to go through or disconnect (eg. if it +doesn't require additional information about the users). + +===== Example +---- + -pinger- HSUP ADBASE ADPING AD.. + -hub- ISUP ADBASE ADPING AD.. + -hub- ISID .. + -hub- IINF NIhubname DEcurrent\stopic VE.. HHmyhub.no-ip.org:555 WShttp://myhub.no-ip.org/ OWmyname UC2231 SS.. SF.. MS0 ML0 MC5000 + - (pinger may disconnect) + -.. +---- + +==== Hub - Hublist communication + +The same extension goes for hub- hublist communication. This way, the hub +takes the role of the client and the hublist of the server. + +The hublist may send INF about itself with NI field which would become hublist +name and WS hublist web address. + +===== Example +---- + -hub- HSUP ADBASE ADPING AD.. + -hublist- ISUP ADBASE ADPING AD.. + -hublist- IINF NIhublist_name WShublist_address + -hub- HINF NIhubname DEcurrent\stopic VE.. HHmyhub.no-ip.org:555 WShttp://myhub.no-ip.org/ OWmyname UC2231 SS.. SF.. MS0 ML0 MC5000 + -( disconnect ) +---- + +=== TS - Timestamp in MSG +Timestamp of the moment when the message was sent, expressed in seconds since the Unix Epoch (January 1 1970 00:00:00 GMT). + +// vim: set syntax=asciidoc: This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <ul...@us...> - 2009-08-04 21:07:57
|
Revision: 6 http://adc.svn.sourceforge.net/adc/?rev=6&view=rev Author: ullner Date: 2009-08-04 21:07:42 +0000 (Tue, 04 Aug 2009) Log Message: ----------- Added DFAV as well... Modified Paths: -------------- trunk/ADC-EXT.txt Modified: trunk/ADC-EXT.txt =================================================================== --- trunk/ADC-EXT.txt 2009-08-04 20:57:28 UTC (rev 5) +++ trunk/ADC-EXT.txt 2009-08-04 21:07:42 UTC (rev 6) @@ -22,6 +22,7 @@ === Version 1.0.1 * Added TS as additional MSG parameter +* Added DFAV == Extensions @@ -76,7 +77,7 @@ TD | Tree depth, index of the highest level of tree data available, root-only = 0, first level (2 leaves) = 1, second level = 2, etc... ___ -=== BZIP – File list compressed with bzip2 +=== BZIP \x96 File list compressed with bzip2 This extension adds a special file "files.xml.bz2" in the unnamed root of the share which contains "files.xml" compressed with bzip2 1.0.3+ (www.bzip.org). @@ -183,4 +184,24 @@ === TS - Timestamp in MSG Timestamp of the moment when the message was sent, expressed in seconds since the Unix Epoch (January 1 1970 00:00:00 GMT). +=== DFAV - Distributed Favorites +The idea behind this extension is to generate a public hublist from the users favorite hublist. Implementations should seperate between public and private hubs in the favorite hublist of an user, in order not to distribute private hubs where one can not connect to anyway. + +==== GFA + +Contexts: T, C + +Asks all users within the same hub with the correct feature to send all publicly available hubs, in their favorite hub list to the requesting client. + +==== RFA + +Contexts: C + +Response of a client. + +HA | Hub address + +LG | Last succesfull login time ( number of seconds since the epoch (1970), (UTC) ) + + // vim: set syntax=asciidoc: This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <ul...@us...> - 2009-08-04 21:30:34
|
Revision: 7 http://adc.svn.sourceforge.net/adc/?rev=7&view=rev Author: ullner Date: 2009-08-04 21:30:26 +0000 (Tue, 04 Aug 2009) Log Message: ----------- Added "All INF fields from BASE are inherited." for DFAV. Modified Paths: -------------- trunk/ADC-EXT.txt Modified: trunk/ADC-EXT.txt =================================================================== --- trunk/ADC-EXT.txt 2009-08-04 21:07:42 UTC (rev 6) +++ trunk/ADC-EXT.txt 2009-08-04 21:30:26 UTC (rev 7) @@ -185,7 +185,7 @@ Timestamp of the moment when the message was sent, expressed in seconds since the Unix Epoch (January 1 1970 00:00:00 GMT). === DFAV - Distributed Favorites -The idea behind this extension is to generate a public hublist from the users favorite hublist. Implementations should seperate between public and private hubs in the favorite hublist of an user, in order not to distribute private hubs where one can not connect to anyway. +The idea behind this extension is to generate a public hublist from the users favorite hublist. Implementations should separate between public and private hubs in the favorite hublist of an user, in order not to distribute private hubs where one can not connect to anyway. ==== GFA @@ -203,5 +203,7 @@ LG | Last succesfull login time ( number of seconds since the epoch (1970), (UTC) ) +All INF fields from BASE are inherited. + // vim: set syntax=asciidoc: This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <ul...@us...> - 2009-08-04 21:36:22
|
Revision: 8 http://adc.svn.sourceforge.net/adc/?rev=8&view=rev Author: ullner Date: 2009-08-04 21:36:10 +0000 (Tue, 04 Aug 2009) Log Message: ----------- DFAV could inherit from PING extension... Modified Paths: -------------- trunk/ADC-EXT.txt Modified: trunk/ADC-EXT.txt =================================================================== --- trunk/ADC-EXT.txt 2009-08-04 21:30:26 UTC (rev 7) +++ trunk/ADC-EXT.txt 2009-08-04 21:36:10 UTC (rev 8) @@ -203,7 +203,7 @@ LG | Last succesfull login time ( number of seconds since the epoch (1970), (UTC) ) -All INF fields from BASE are inherited. +All INF fields from BASE are inherited. All INF fields from PING extension are inherited. // vim: set syntax=asciidoc: This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <ul...@us...> - 2009-08-04 21:38:10
|
Revision: 9 http://adc.svn.sourceforge.net/adc/?rev=9&view=rev Author: ullner Date: 2009-08-04 21:38:01 +0000 (Tue, 04 Aug 2009) Log Message: ----------- Last changes were for some reason not included... Modified Paths: -------------- trunk/ADC-EXT.txt Modified: trunk/ADC-EXT.txt =================================================================== --- trunk/ADC-EXT.txt 2009-08-04 21:36:10 UTC (rev 8) +++ trunk/ADC-EXT.txt 2009-08-04 21:38:01 UTC (rev 9) @@ -199,8 +199,6 @@ Response of a client. -HA | Hub address - LG | Last succesfull login time ( number of seconds since the epoch (1970), (UTC) ) All INF fields from BASE are inherited. All INF fields from PING extension are inherited. This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <ul...@us...> - 2010-06-27 15:01:24
|
Revision: 14 http://adc.svn.sourceforge.net/adc/?rev=14&view=rev Author: ullner Date: 2010-06-27 15:01:18 +0000 (Sun, 27 Jun 2010) Log Message: ----------- Fix superscript etc. Modified Paths: -------------- trunk/ADC-EXT.txt Modified: trunk/ADC-EXT.txt =================================================================== --- trunk/ADC-EXT.txt 2010-06-26 11:10:34 UTC (rev 13) +++ trunk/ADC-EXT.txt 2010-06-27 15:01:18 UTC (rev 14) @@ -305,7 +305,7 @@ ==== Probability |===== -|p == (1 - (1 - 1 / m)^(k * n))^k |False positives +|p == (1 - (1 - 1 / m)\^(k * n))^k |False positives |p == 0 |False negatives |===== @@ -338,7 +338,7 @@ Once the hub has received the bloom filter bit array, for each search command it processes, if it contains a hash search term, it can skip broadcasting the search to a particular client if at least one of the k bits in that clients bit array is "0", calculating positions as the client does when setting bits to "1". The hub has to evaluate the filter for each client that it has a bloom filter for, for each search. ==== Probability calculations -p = (1 - (1 - 1 / m)^(k * n))^k, thus p becomes smaller as m grows and larger as n grows. Larger m means more bits to transfer but also fewer false positives. The optimum value for k given m and n is (m / n) * ln 2. The largest k supported by a hash of a certain size is b / h, so if the hub wants the smallest p possible, it should choose the smallest possible h which gives the largest k, and then calculate m = k * n/ln 2, checking that the resulting m < 2^h. 2^h should much be larger than m (at least 3-4 times), because of how the modulo operator works. Also, with m grows the required bandwidth to transfer the bloom filter, so the hub may wish to cap m. In that case, it should still choose k according to m / n * ln 2, but send an h as big as possible to alleviate biasing problems. +p = (1 - (1 - 1 / m)\^(k * n))\^k, thus p becomes smaller as m grows and larger as n grows. Larger m means more bits to transfer but also fewer false positives. The optimum value for k given m and n is (m / n) * ln 2. The largest k supported by a hash of a certain size is b / h, so if the hub wants the smallest p possible, it should choose the smallest possible h which gives the largest k, and then calculate m = k * n/ln 2, checking that the resulting m < 2\^h. 2^h should much be larger than m (at least 3-4 times), because of how the modulo operator works. Also, with m grows the required bandwidth to transfer the bloom filter, so the hub may wish to cap m. In that case, it should still choose k according to m / n * ln 2, but send an h as big as possible to alleviate biasing problems. ==== Sample implementations ===== Tiger This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <ul...@us...> - 2010-09-26 18:06:25
|
Revision: 32 http://adc.svn.sourceforge.net/adc/?rev=32&view=rev Author: ullner Date: 2010-09-26 18:06:19 +0000 (Sun, 26 Sep 2010) Log Message: ----------- Convinced that we don't need to re-create the URI syntax... Modified Paths: -------------- trunk/ADC-EXT.txt Modified: trunk/ADC-EXT.txt =================================================================== --- trunk/ADC-EXT.txt 2010-09-26 17:51:23 UTC (rev 31) +++ trunk/ADC-EXT.txt 2010-09-26 18:06:19 UTC (rev 32) @@ -44,7 +44,6 @@ === Version 1.0.6 UNRELEASED * Added KEYP extension for providing certificate substitution protection in ADCS. -* Added URI specification for ADC == Extensions @@ -552,17 +551,7 @@ ==== Example ==== -adcs://example.com:1234/kp?SHA256/G3PJC4F4MQ5KOXGE2MPYJW5EW63IC6M7RN7OS663JLLWN2M5I6FQ +adcs://example.com:1234/?kp=SHA256/G3PJC4F4MQ5KOXGE2MPYJW5EW63IC6M7RN7OS663JLLWN2M5I6FQ ==== -=== URI specification -ADC URIs shall be specified in the following (extended) form; - -==== -adc://example.com:1234/foo?bar -==== -Where 'foo' is the identifyable or parsable string for the URI and where 'bar' is the content or value. - -Extensions creating a new URI (say, the adcs:// URI), should also adopt this general structure. - // vim: set syntax=asciidoc: This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <ul...@us...> - 2010-11-23 22:55:53
|
Revision: 69 http://adc.svn.sourceforge.net/adc/?rev=69&view=rev Author: ullner Date: 2010-11-23 22:55:47 +0000 (Tue, 23 Nov 2010) Log Message: ----------- Added FS field to INF - free slots in client Modified Paths: -------------- trunk/ADC-EXT.txt Modified: trunk/ADC-EXT.txt =================================================================== --- trunk/ADC-EXT.txt 2010-11-08 23:16:20 UTC (rev 68) +++ trunk/ADC-EXT.txt 2010-11-23 22:55:47 UTC (rev 69) @@ -50,6 +50,7 @@ * Added 'FEED' extension for RSS feeds. * Added grouping of file extensions to SCH. * Added failover hub addresses to the hub's INF. +* Added free slots to the client's INF. == Extensions @@ -171,7 +172,7 @@ |UP |integer |Hub uptime (seconds) |NI |string |Hub name (from BASE) |DE |string |Hub description (from BASE) -VE |string |Hub software version (from BASE) +|VE |string |Hub software version (from BASE) |===== The hub must continue to send the user list as for a normal user (move to @@ -573,7 +574,7 @@ |KY | 16 byte encryption key in BASE32. 128 bit AES encryption shall be used. |===== -For example, a SCH command will result in 29 Bytes of overhead ("<space>KY"+26 Bytes Base32 encoded key). +For example, a SCH command will result in 29 Bytes of overhead ("<separator>KY"+26 Bytes Base32 encoded key). A client that has a response for the command can now encrypt the response message by prepending 16 bytes of random data and afterwards encrypting it with AES/CBC/PKCS5Padding (Cipher/Blockmode/Padding) using 16 zero bytes as IV for CBC. @@ -613,7 +614,7 @@ |===== === FEED - RSS feeds -The extension adds RSS feed support. See http://en.wikipedia.org/wiki/RSS for a description. +The extension adds http://en.wikipedia.org/wiki/RSS[RSS feed] support. Signal FEED in SUP and the INF's SU field. @@ -657,7 +658,7 @@ ==== === GR - Grouping of file extensions in SCH -In BASE, clients add EX fields to SCH to denote which extension files should have. This can lead to a situation where the large bulk of extensions are of similar "type", e.g. audio files or documents. This extension intend to add a field GR which groups multiple extensions. In addition, the field RX shall be used for group-exclusion; if all extensions in a group but one are desired, field RX will be used to exclude those group items. +In BASE, clients add EX fields to SCH to denote which extension files should have. This can lead to a situation where the large bulk of extensions are of similar "type", e.g. audio files or documents. This extension intend to add a field GR which groups multiple extensions. In addition, the field RX shall be used for group-exclusion; if all extensions in a group but two are desired, field RX will be used to exclude those group items. Field GR values, where multiple groups are specified by adding the numbers together: [options="autowidth"] @@ -691,4 +692,13 @@ |FO |Failover hub addresses. Specify well formed ADC or ADCS URI addresses, with multiple addresses separated with a comma. Example; FOadc://example.com:1234,adc://example.net:1234 |===== +=== FS - Free slots in client +This is an extension that will broadcast the amount of free slots the client has available. + +Field FS in the client's INF: +[options="autowidth"] +|===== +|FS |Free slots available. +|===== + // vim: set syntax=asciidoc: This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <ul...@us...> - 2014-01-19 16:27:06
|
Revision: 118 http://sourceforge.net/p/adc/code/118 Author: ullner Date: 2014-01-19 16:27:04 +0000 (Sun, 19 Jan 2014) Log Message: ----------- Added 'RDEX' for extended redirecting capabilities. Modified Paths: -------------- trunk/ADC-EXT.txt Modified: trunk/ADC-EXT.txt =================================================================== --- trunk/ADC-EXT.txt 2013-12-21 21:48:05 UTC (rev 117) +++ trunk/ADC-EXT.txt 2014-01-19 16:27:04 UTC (rev 118) @@ -24,6 +24,7 @@ * Added 'Size' attribute in file list for directories. * Added 'Children' attribute in file list for directories. * Added downloaded progress report for uploaders in GET. +* Added 'RDEX' for extended redirecting capabilities. === Version 1.0.7 Fredrik Ullner <ul...@gm...>, 2012-11-22 @@ -948,4 +949,75 @@ |DB |Downloaded (and verified) bytes. |===== +=== RDEX - Redirects Extended + +The goal of this extension is to enhance the redirect capabilities provided by BASE by adding support for two new redirect operations and a way for hubs and clients to communicate accepted redirect protocols. It also attempts to make the concept of redirects more unambiguous and less implementation defined. The two new types of redirects added are a multiple choice redirect and so-called permanent redirect as outlined below. These new redirect types are not necessarily mutually exclusive. + +==== Redirect Security + +Hubs should always aim to include as much information pertaining to a redirect as possible using other fields defined in BASE. Clients may, for example, rely on the availability of such information to check the origin of a redirect before making irreversible decisions, such as accepting a redirect as permanent as defined below. + +Clients should not attempt to connect to an unreachable or otherwise invalid destination resources indefinitely or in otherwise abusive manner when responding to redirects. Whether clients respond to redirects using default user information or the same information as used on the hub that initiated the redirect is implementation defined, however, in the case that the same information is used careful consideration should be taken when it comes to automatically sending sensitive information such as passwords to other hubs especially if it provides less security to the user than the original hub. + +Both clients and hubs should also take note that the TL field, as defined by BASE, should only impact the current hub and its connection (if re-established) thus it should not be relied on to trigger a delayed response to a redirect. + +==== Accepted Redirect Protocols + +Support for this extension is indicated by the presence of an RP field in the INF, which defines the accepted redirect protocols for the hub or a client as a bitmask using any combination of following values. Additional compatible values may be defined by other extensions. + +[options="header"] +|===== +|Protocol |URI Syntax |Flag +|User action (exclusive) |any supported, as defined below |0 +|ADC (BASE) |adc://adress:port |1 +|<<anchor-ADCS,ADCS Extension>> |adcs://address:port |2 +|http://nmdc.sf.net[NeoModus Direct Connect] |dchub://address[:port] |4 +|===== + +Hubs should treat the absence of the RP field in clients INF as the equivalent of RP with the value of one, thus implying support for BASE. If the ADCS feature is included in the clients feature list this may also be amended to include ADCS. If a client does not define an RP field the hub may choose to omit the RDEX syntax from redirects, although this extension retains full backwards compatibility with BASE. If a hub defines an RP field for itself it should prevent its users from redirecting others to protocols it did not define, including redirects to destinations without an explicit protocol handler. Hubs should always aim to use fully qualified resource identifiers for all redirects, regardless of whether an RP field is defined or not, to avoid ambiguous destinations due to implementation defined default handling. + +Clients should refrain from removing support of a previously signalled redirect protocol from their RP field. It is up to the hub to decide a corresponding course of action should the client do so. Additionally a special case is defined for clients as an RP field with the value of zero to indicate that the client may not react to redirects as defined by BASE or this extension at all or may do so only as a result of user action. As such clients defining a non-zero RP field should follow redirects to accepted protocols automatically and conversely any attempts to redirect clients that define RP as zero should result in normal disconnection as per BASE. + +==== Multiple Choice Redirect + +Multiple choice redirect is defined as a redirect where instead of a single destination resource a set of resource identifiers is provided. The methodology that is used to select the final destination resource for a multiple choice redirect is implementation defined. However, only one final destination resource may be chosen. For example a particular implementation may prefer one protocol over another, encrypted connection over an unencrypted one or even simply choose one at random. + +This type of redirect is signalled by the presence of an additional RX field in the QUI as defined for redirects by BASE. The value of the RX field is a comma separated list of resource identifiers to use in the redirect. The RD field should be defined for maximum compatibility and clients should add its value to the list of identifiers defined in RX when selecting the final destination resource. + +==== Permanent Redirect + +Permanent redirect is defined as a redirect with reference updating. In the case of a permanent redirect, if and only if a successful connection is made to the destination resource, the connecting party will then update all relevant references of the original resource to the destination resource. Examples of such references may be hublist records for pingers or favorite entries for clients. + +This type of redirect is signalled by the presence of an additional PT field in the QUI, as defined for redirects by BASE, with the value of one. Upon receiving such redirect clients will proceed to update references as outlined above provided the conditions are met. + +==== Examples + +Multiple Choice Redirect (BASE compliant) +[options="header"] +|===== +|Example |Description +|IQUI BBBB IDAAAA RDadc://example.com:1001 RXadcs://example.com:1002,adc://example.org:1020 |Here user BBBB is presented with a choice between three destinations Two on example.com and one on example.org. Old and non-supporting clients will be redirected to adc://example.com:1001. User AAAA is the originator for this redirect. +|===== + +Multiple Choice Redirect (RDEX only, not compatible with old clients thus not recommended) +[options="header"] +|===== +|Example |Description +|IQUI BBBB IDAAAA RXadc://example.com:1001,adcs://example.com:1002,adc://example.org:1020 |For clients supporting RDEX this is the same as sending the first example. Old and non-supporting clients will be disconnected and they might reconnect to the original hub after an implementation defined time. +|===== + +Permanent Redirect +[options="header"] +|===== +|Example |Description +|IQUI BBBB IDAAAA MSWe're\smoving! RDadc://example.com:1001 PT1 |Here user BBBB will be redirected to adc://example.com:1001 with the message of "We're moving!" and AAAA as the originator. BBBB should (see section Redirect Security) connect directly to adc://example.com:1001 in the future. +|===== + +Permanent Multiple Choice Redirect +[options="header"] +|===== +|Example |Description +|IQUI BBBB IDAAAA MSWe\shave\sADCS\snow! RDadc://example.com:1001 RXadcs://example.com:1002 PT1 TL0 |Here user BBBB is redirected permanently with a choice between ADC and ADCS URI, if and only if either of the URI's matches the URI of the current hub connection and the matching URI is chosen BBBB will reconnect immediately. Sending a redirect like this to all connected clients (once) can be used to attempt to migrate them to ADCS instance of a hub. +|===== + // vim: set syntax=asciidoc: This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <ul...@us...> - 2014-02-11 21:28:04
|
Revision: 121 http://sourceforge.net/p/adc/code/121 Author: ullner Date: 2014-02-11 21:27:46 +0000 (Tue, 11 Feb 2014) Log Message: ----------- Release of ADC-EXT.txt, now at 1.0.8 Modified Paths: -------------- trunk/ADC-EXT.txt Modified: trunk/ADC-EXT.txt =================================================================== --- trunk/ADC-EXT.txt 2014-02-05 20:53:35 UTC (rev 120) +++ trunk/ADC-EXT.txt 2014-02-11 21:27:46 UTC (rev 121) @@ -1,5 +1,6 @@ = ADC Extensions -1.0.8, UNRELEASED +Fredrik Ullner, ul...@gm... +1.0.8, February 2014 == Abstract These are the official extensions to ADC. This document is based on the @@ -13,7 +14,8 @@ This version corresponds to $Revision$. -=== Version 1.0.8, UNRELEASED +=== Version 1.0.8 +Fredrik Ullner <ul...@gm...>, 2014-02-11 * Improved 'NATT' documentation, as according to the original paper. * Added 'ONID' extension to provide online service integration. This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <ul...@us...> - 2010-04-04 13:34:58
|
Revision: 10 http://adc.svn.sourceforge.net/adc/?rev=10&view=rev Author: ullner Date: 2010-04-04 13:34:49 +0000 (Sun, 04 Apr 2010) Log Message: ----------- Create ADC-Ext 1.0.2 version with; Added UCMD support. Modified Paths: -------------- trunk/ADC-EXT.txt Modified: trunk/ADC-EXT.txt =================================================================== --- trunk/ADC-EXT.txt 2009-08-04 21:38:01 UTC (rev 9) +++ trunk/ADC-EXT.txt 2010-04-04 13:34:49 UTC (rev 10) @@ -3,6 +3,8 @@ 1.0, May 2008 Fredrik Ullner <ul...@gm...> 1.0.1 August 2009 +Fredrik Ullner <ul...@gm...> +1.0.2 April 2010 == Abstract These are the official extensions to ADC. This document is based on the @@ -24,6 +26,9 @@ * Added TS as additional MSG parameter * Added DFAV +=== Version 1.0.2 +* Added UCMD extension + == Extensions === TIGR - Tiger tree hash support @@ -199,9 +204,73 @@ Response of a client. +HA | Hub address + LG | Last succesfull login time ( number of seconds since the epoch (1970), (UTC) ) All INF fields from BASE are inherited. All INF fields from PING extension are inherited. +==== UCMD +CMD name + +Contexts: F + +States: NORMAL + +User commands are used to send hub-specific commands to the client which provide useful shortcuts for the user. These commands contain strings which must be sent back to the hub and keyword substitutions in the strings. Each user command has a display name, a string to be sent to the hub, and one or more categories where it may appear. The strings passed to the hub must first be passed through a dictionary replacement that replaces all keywords in the string and then through the equivalent of the C standard function "strftime", with the current time. + +Name uniquely (per hub) identifies a particular user command. The name may contain "/" to indicate a logical structure on the viewing client, where each "/" introduces a new submenu level. Other than name, the command also has a number of flags that further detail what to do with it. + +[separator="|"] +``_ +RM | 1 = Remove Command +CT | Message Category, 1 = Hub command, client parameters only, 2 = User list command, client and user parameters, 4 = Search result command, client, user and file parameters, 8 = File list command, client, user and file parameters. +TT | The full text to be sent to hub, including FOURCC, parameters and keywords. +CO | 1 = Constrained, when sending this command on multiple users (for example in search results), constrain it to once per CID only +SP | 1 = Insert separator instead of command name (name must still be present to uniquely identify the command). + +===== Keywords +Keywords are specified using "%[keyword]". Unknown keywords must be replaced by the empty string. Additionally, all %-substitutions of the C function "strftime" must be supported. + +The following tables specify the keywords that must be supported. + +Client parameters +[separator="|"] +``_ +myCID | Client CID +mySID | ClientSID +myXX | One for each flag on that particular hub; for example, myI4 and myNI + +User parameters +[separator="|"] +``_ +userCID | User CID +userSID | User SID +userXX | One for each flag on the user sent; for example, userI4 and userNI + +File parameters +[separator="|"] +``_ +hubXX | One for each flag of the hub; for example, hubNI and hubVE + +Hub parameters +[separator="|"] +``_ + +The following tables specify the keywords that are optional. + +User parameters +[separator="|"] +``_ +line:info | Prompts the user for input where 'info' is the displayed text description for the user input + +===== Example +ICMD ADCH++/Hub\smanagement/Register\snick TTHMSG\s+regnick\\s%[userNI]\\s%[line:Password\\s(leave\\sempty\\sto\\sun-reg)]\\s%[line:Level\\s(facultative;\\sdefaults\\sto\\syour\\sown\\slevel\\sminus\\sone)]\n CT2 +ICMD ADCH++/Hub\smanagement/Reload\sbans TTHMSG\s+loadbans\n CT3 +ICMD ADCH++/Hub\smanagement/Reload\sscripts TTHMSG\s+reload\n CT3 +ICMD ADCH++/Info TTHMSG\s+info\\s%[userNI]\n CT2 +ICMD ADCH++/Info TTHMSG\s+info\n CT1 + + // vim: set syntax=asciidoc: This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <ul...@us...> - 2010-04-04 13:54:45
|
Revision: 11 http://adc.svn.sourceforge.net/adc/?rev=11&view=rev Author: ullner Date: 2010-04-04 13:54:38 +0000 (Sun, 04 Apr 2010) Log Message: ----------- Fix minor things so that we can generate an HTML document.. Modified Paths: -------------- trunk/ADC-EXT.txt Modified: trunk/ADC-EXT.txt =================================================================== --- trunk/ADC-EXT.txt 2010-04-04 13:34:49 UTC (rev 10) +++ trunk/ADC-EXT.txt 2010-04-04 13:54:38 UTC (rev 11) @@ -1,10 +1,6 @@ = ADC Extensions -Jacek Sieka <arn...@gm...> -1.0, May 2008 Fredrik Ullner <ul...@gm...> -1.0.1 August 2009 -Fredrik Ullner <ul...@gm...> -1.0.2 April 2010 +1.0.2, April 2010 == Abstract These are the official extensions to ADC. This document is based on the @@ -16,7 +12,7 @@ and older versions can be downloaded from $URL$. -This version correspods to $Revision$. +This version corresponds to $Revision$. === Version 1.0 * Initial release created from original ADC 1.0 text @@ -82,7 +78,7 @@ TD | Tree depth, index of the highest level of tree data available, root-only = 0, first level (2 leaves) = 1, second level = 2, etc... ___ -=== BZIP \x96 File list compressed with bzip2 +=== BZIP - File list compressed with bzip2 This extension adds a special file "files.xml.bz2" in the unnamed root of the share which contains "files.xml" compressed with bzip2 1.0.3+ (www.bzip.org). @@ -210,9 +206,9 @@ All INF fields from BASE are inherited. All INF fields from PING extension are inherited. -==== UCMD +=== UCMD -CMD name + CMD name Contexts: F @@ -225,12 +221,13 @@ [separator="|"] ``_ RM | 1 = Remove Command -CT | Message Category, 1 = Hub command, client parameters only, 2 = User list command, client and user parameters, 4 = Search result command, client, user and file parameters, 8 = File list command, client, user and file parameters. -TT | The full text to be sent to hub, including FOURCC, parameters and keywords. +CT | Message Category, 1 = Hub command, client parameters only, 2 = User list command, client and user parameters, 4 = Search result command, client, user and file parameters, 8 = File list command, client, user and file parameters. Multiple types are specified by adding the numbers together. +TT | The full text to be sent to hub, including FOURCC, parameters and keywords. CO | 1 = Constrained, when sending this command on multiple users (for example in search results), constrain it to once per CID only SP | 1 = Insert separator instead of command name (name must still be present to uniquely identify the command). +___ -===== Keywords +==== Keywords Keywords are specified using "%[keyword]". Unknown keywords must be replaced by the empty string. Additionally, all %-substitutions of the C function "strftime" must be supported. The following tables specify the keywords that must be supported. @@ -241,6 +238,7 @@ myCID | Client CID mySID | ClientSID myXX | One for each flag on that particular hub; for example, myI4 and myNI +___ User parameters [separator="|"] @@ -248,15 +246,19 @@ userCID | User CID userSID | User SID userXX | One for each flag on the user sent; for example, userI4 and userNI +___ File parameters [separator="|"] ``_ -hubXX | One for each flag of the hub; for example, hubNI and hubVE +fileXX | One for each flag contained within a search result or file list entry (see RES) +___ Hub parameters [separator="|"] ``_ +hubXX | One for each flag of the hub; for example, hubNI and hubVE +___ The following tables specify the keywords that are optional. @@ -264,13 +266,13 @@ [separator="|"] ``_ line:info | Prompts the user for input where 'info' is the displayed text description for the user input +___ -===== Example +==== Example ICMD ADCH++/Hub\smanagement/Register\snick TTHMSG\s+regnick\\s%[userNI]\\s%[line:Password\\s(leave\\sempty\\sto\\sun-reg)]\\s%[line:Level\\s(facultative;\\sdefaults\\sto\\syour\\sown\\slevel\\sminus\\sone)]\n CT2 ICMD ADCH++/Hub\smanagement/Reload\sbans TTHMSG\s+loadbans\n CT3 ICMD ADCH++/Hub\smanagement/Reload\sscripts TTHMSG\s+reload\n CT3 ICMD ADCH++/Info TTHMSG\s+info\\s%[userNI]\n CT2 ICMD ADCH++/Info TTHMSG\s+info\n CT1 - - -// vim: set syntax=asciidoc: + +// vim: set syntax=asciidoc: This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <ul...@us...> - 2010-04-04 14:24:38
|
Revision: 12 http://adc.svn.sourceforge.net/adc/?rev=12&view=rev Author: ullner Date: 2010-04-04 14:24:32 +0000 (Sun, 04 Apr 2010) Log Message: ----------- Additional formatting stuff for ASCIIDoc. Modified Paths: -------------- trunk/ADC-EXT.txt Modified: trunk/ADC-EXT.txt =================================================================== --- trunk/ADC-EXT.txt 2010-04-04 13:54:38 UTC (rev 11) +++ trunk/ADC-EXT.txt 2010-04-04 14:24:32 UTC (rev 12) @@ -269,10 +269,12 @@ ___ ==== Example -ICMD ADCH++/Hub\smanagement/Register\snick TTHMSG\s+regnick\\s%[userNI]\\s%[line:Password\\s(leave\\sempty\\sto\\sun-reg)]\\s%[line:Level\\s(facultative;\\sdefaults\\sto\\syour\\sown\\slevel\\sminus\\sone)]\n CT2 -ICMD ADCH++/Hub\smanagement/Reload\sbans TTHMSG\s+loadbans\n CT3 -ICMD ADCH++/Hub\smanagement/Reload\sscripts TTHMSG\s+reload\n CT3 -ICMD ADCH++/Info TTHMSG\s+info\\s%[userNI]\n CT2 -ICMD ADCH++/Info TTHMSG\s+info\n CT1 +`_ +ICMD ADCH++/Hub{backslash}smanagement/Register{backslash}snick TTHMSG{backslash}s+regnick{backslash}s%[userNI]{backslash}s%[line:Password{backslash}s(leave{backslash}sempty{backslash}sto{backslash}sun-reg)]{backslash}s%[line:Level{backslash}s(facultative;{backslash}sdefaults{backslash}sto{backslash}syour{backslash}sown{backslash}slevel{backslash}sminus{backslash}sone)]{backslash}n CT2 +ICMD ADCH++/Hub{backslash}smanagement/Reload{backslash}sbans TTHMSG{backslash}s+loadbans{backslash}n CT3 +ICMD ADCH++/Hub{backslash}smanagement/Reload{backslash}sscripts TTHMSG{backslash}s+reload{backslash}n CT3 +ICMD ADCH++/Info TTHMSG{backslash}s+info{backslash}s%[userNI]{backslash}n CT2 +ICMD ADCH++/Info TTHMSG{backslash}s+info{backslash}n CT1 +___ // vim: set syntax=asciidoc: This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <ul...@us...> - 2010-06-26 11:10:41
|
Revision: 13 http://adc.svn.sourceforge.net/adc/?rev=13&view=rev Author: ullner Date: 2010-06-26 11:10:34 +0000 (Sat, 26 Jun 2010) Log Message: ----------- Adding BLOM Removing 'optional' from UCMD. Modified Paths: -------------- trunk/ADC-EXT.txt Modified: trunk/ADC-EXT.txt =================================================================== --- trunk/ADC-EXT.txt 2010-04-04 14:24:32 UTC (rev 12) +++ trunk/ADC-EXT.txt 2010-06-26 11:10:34 UTC (rev 13) @@ -1,6 +1,6 @@ -= ADC Extensions += ADC Extensions Fredrik Ullner <ul...@gm...> -1.0.2, April 2010 +1.0.3, June 2010 == Abstract These are the official extensions to ADC. This document is based on the @@ -25,6 +25,10 @@ === Version 1.0.2 * Added UCMD extension +=== Version 1.0.3 +* Removed 'optional' keywords from UCMD +* Added BLOM extension + == Extensions === TIGR - Tiger tree hash support @@ -246,6 +250,7 @@ userCID | User CID userSID | User SID userXX | One for each flag on the user sent; for example, userI4 and userNI +line:info | Prompts the user for input where 'info' is the displayed text description for the user input ___ File parameters @@ -260,14 +265,6 @@ hubXX | One for each flag of the hub; for example, hubNI and hubVE ___ -The following tables specify the keywords that are optional. - -User parameters -[separator="|"] -``_ -line:info | Prompts the user for input where 'info' is the displayed text description for the user input -___ - ==== Example `_ ICMD ADCH++/Hub{backslash}smanagement/Register{backslash}snick TTHMSG{backslash}s+regnick{backslash}s%[userNI]{backslash}s%[line:Password{backslash}s(leave{backslash}sempty{backslash}sto{backslash}sun-reg)]{backslash}s%[line:Level{backslash}s(facultative;{backslash}sdefaults{backslash}sto{backslash}syour{backslash}sown{backslash}slevel{backslash}sminus{backslash}sone)]{backslash}n CT2 @@ -277,4 +274,76 @@ ICMD ADCH++/Info TTHMSG{backslash}s+info{backslash}n CT1 ___ + +=== BLOM + +Bloom filters allow the hub to filter certain searches using bitmap that represents the hashes of the files in the users share. BLOM is an extension that allows hub software to create a map (bloom filter) of the shared files on the hub, but with minimal effort, e.g. the hub doesn't keep a list of files, but a filter that never produces false negatives but only possible false positives. This can potentially save bandwidth and effort on the client side. When the user updates the share, the client must send an INF containing the flag SF. The hub may at any time request that the client sends an updated version of its bloom filter by sending a GET command to the client. The client will then respond using SND and send the bloom filter binary data. + +==== Legend + +|===== +|b |Number of bits used for file hashes +|n |Number of files in the user's share +|m |Size of the bloom filter in bits +|k |Number of sub-hashes constructed from the file hash +|h |Number of bits to use for each sub-hash +|p |Propability of a false positive +|===== + +The hub chooses k, h and m. + +==== Restrictions + +[width="15%"] +|===== +|k * h < = b +|h < = 64 +|2^h > m +|m mod 64 == 0 +|===== + +==== Probability + +|===== +|p == (1 - (1 - 1 / m)^(k * n))^k |False positives +|p == 0 |False negatives +|===== + +==== Protocol changes +Signal BLOM in SUP. + +For the SND type, adds H as message type. + +For the GET type, adds I as message type. + +For the GET type, adds "blom" as type. + +For the GET type, "/" shall be used as namespace. + +For the GET type, 0 (zero) shall be used as start position. + +For the GET type, m / 8 shall be used as byte amount. + +Updates GET with the following flags; + +[width="15%"] +|===== +|BK |Specify 'k'. +|BH |Specify 'h'. +|===== + +==== Algorithm +The client constructs the bloom filter by creating a bit array of m bits set to 0 (zero). For each file it then sets to "1" k positions constructed from the file hash. Seeing the file hash as a stream of bits (starting from the lowest bit of the first byte, ending at the highest bit of the last byte), the client should use h bits starting at the first bit of the first byte to create an integer and apply modulo m to get the position in the bit array, then redo the process k times advancing the starting position by h each time. + +Once the hub has received the bloom filter bit array, for each search command it processes, if it contains a hash search term, it can skip broadcasting the search to a particular client if at least one of the k bits in that clients bit array is "0", calculating positions as the client does when setting bits to "1". The hub has to evaluate the filter for each client that it has a bloom filter for, for each search. + +==== Probability calculations +p = (1 - (1 - 1 / m)^(k * n))^k, thus p becomes smaller as m grows and larger as n grows. Larger m means more bits to transfer but also fewer false positives. The optimum value for k given m and n is (m / n) * ln 2. The largest k supported by a hash of a certain size is b / h, so if the hub wants the smallest p possible, it should choose the smallest possible h which gives the largest k, and then calculate m = k * n/ln 2, checking that the resulting m < 2^h. 2^h should much be larger than m (at least 3-4 times), because of how the modulo operator works. Also, with m grows the required bandwidth to transfer the bloom filter, so the hub may wish to cap m. In that case, it should still choose k according to m / n * ln 2, but send an h as big as possible to alleviate biasing problems. + +==== Sample implementations +===== Tiger +For TTH roots, b is 192 and a reasonable value for h is 24, giving a maximum k = 8 which means that m = 8 * n / ln 2 ≈ 11.5 * n. The required bandwidth then becomes 11.5 * n / 8 bytes, so approximately 1.44 bytes per file in the users share. For 20000 files, m should then be 230016 (taking into account the modulo 64 requirement), giving a p = 0.004, in other words ~99.6% of all searches that don't match a users share would not be sent, saving valuable upload bandwidth for the hub. The client calculates i valid positions, if x is an array of bytes containing the hash of the file, on a little-endian machine, by doing pos = x[0+i*h/8] | (x[1+i*h/8] << 8) | (x[2+i*h/8] << 16) for i = [0;k). This is of course a special case where h % 8 = 0, the actual algorithm has to take into consideration values for h where the integer crosses byte boundaries. + +For test vectors, see the ADC wiki talk page; http://www.adcportal.com/wiki/index.php/Talk:BLOM + // vim: set syntax=asciidoc: This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <ul...@us...> - 2010-07-11 21:28:23
|
Revision: 15 http://adc.svn.sourceforge.net/adc/?rev=15&view=rev Author: ullner Date: 2010-07-11 21:28:16 +0000 (Sun, 11 Jul 2010) Log Message: ----------- Begin 1.0.4; * fileMN for magnet links in UCMD Modified Paths: -------------- trunk/ADC-EXT.txt Modified: trunk/ADC-EXT.txt =================================================================== --- trunk/ADC-EXT.txt 2010-06-27 15:01:18 UTC (rev 14) +++ trunk/ADC-EXT.txt 2010-07-11 21:28:16 UTC (rev 15) @@ -1,6 +1,6 @@ = ADC Extensions Fredrik Ullner <ul...@gm...> -1.0.3, June 2010 +1.0.4 UNRELEASED, July 2010 == Abstract These are the official extensions to ADC. This document is based on the @@ -29,6 +29,9 @@ * Removed 'optional' keywords from UCMD * Added BLOM extension +=== Version 1.0.4 UNRELEASED +* Added magnet link extension to UCMD + == Extensions === TIGR - Tiger tree hash support @@ -257,6 +260,7 @@ [separator="|"] ``_ fileXX | One for each flag contained within a search result or file list entry (see RES) +fileMN | Specify magnet link. Magnet links <http://en.wikipedia.org/wiki/Magnet_link> are used to reference files across networks and applications. Clients may ignore parameters it does not understand, but are free to pass on the parameters to other programs. ___ Hub parameters This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <ul...@us...> - 2010-07-12 09:53:42
|
Revision: 18 http://adc.svn.sourceforge.net/adc/?rev=18&view=rev Author: ullner Date: 2010-07-12 09:53:35 +0000 (Mon, 12 Jul 2010) Log Message: ----------- Add basic NATT (will get a slight update in its text later on). Removed old style tables. Made proper hyper links in the document. General cleanup in terms of document structure and style. Modified Paths: -------------- trunk/ADC-EXT.txt Modified: trunk/ADC-EXT.txt =================================================================== --- trunk/ADC-EXT.txt 2010-07-11 21:42:47 UTC (rev 17) +++ trunk/ADC-EXT.txt 2010-07-12 09:53:35 UTC (rev 18) @@ -4,7 +4,7 @@ == Abstract These are the official extensions to ADC. This document is based on the -information contained in the ADC wiki - spefications from there are moved here +information contained in the ADC wiki and ADC forum - spefications from there are moved here when they are mature and stable enough. == Version history @@ -26,11 +26,12 @@ * Added UCMD extension === Version 1.0.3 -* Removed 'optional' keywords from UCMD +* Removed optional keywords from UCMD * Added BLOM extension === Version 1.0.4 UNRELEASED -* Added magnet link extension to UCMD +* Added magnet link extension to 'UCMD' +* Added NAT traversal extension 'NATT' == Extensions @@ -79,11 +80,10 @@ In SCH and GFI, the following attributes are added: -[separator="|"] -``_ -TR | Tiger tree Hash root, encoded with base32. -TD | Tree depth, index of the highest level of tree data available, root-only = 0, first level (2 leaves) = 1, second level = 2, etc... -___ +|===== +|TR |Tiger tree Hash root, encoded with base32. +|TD |Tree depth, index of the highest level of tree data available, root-only = 0, first level (2 leaves) = 1, second level = 2, etc... +|===== === BZIP - File list compressed with bzip2 This extension adds a special file "files.xml.bz2" in the unnamed root of the @@ -112,8 +112,6 @@ uncompressed bytes to be transferred. === PING - Pinger extension -Added 2008-03-14, based on ADC 1.0 - This extension can be supported by both clients and hubs, and when present, if hub supports it, it must send additional information to the client ( otherwise normal base client). @@ -130,33 +128,32 @@ normal INF along with the following added fields ( none mandatory, if not present, it means hub has no restrictions in that matter, or non existent): -[separator="|"] -```_ -Code | Type | Description -___ -HH | string | Hub Host address ( DNS or IP ) -WS | url | Hub Website -NE | string | Hub Network -OW | string | Hub Owner name -UC | integer | Current User count -SS | integer | Total share size -SF | integer | Total files shared -MS | integer | Minimum share required to enter hub ( bytes ) -XS | integer | Maximum share for entering hub ( bytes ) -ML | integer | Minimum slots required to enter hub -XL | integer | Maximum slots for entering hub -MU | integer | Minimum hubs connected where clients can be users -MR | integer | Minimum hubs connected where client can be registered -MO | integer | Minimum hubs connected where client can be operators -XU | integer | Maximum hubs connected where clients can be users -XR | integer | Maximum hubs connected where client can be registered -XO | integer | Maximum hubs connected where client can be operators -MC | integer | Maximum possible clients ( users ) who can connect -UP | integer | Hub uptime (seconds) -NI | string | Hub name (from BASE) -DE | string | Hub description (from BASE) -VE | string | Hub software version (from BASE) -___ +[options="header"] +|===== +|Code | Type | Description +|HH |string |Hub Host address ( DNS or IP ) +|WS |url |Hub Website +|NE |string |Hub Network +|OW |string |Hub Owner name +|UC |integer |Current User count +|SS |integer |Total share size +|SF |integer |Total files shared +|MS |integer |Minimum share required to enter hub ( bytes ) +|XS |integer |Maximum share for entering hub ( bytes ) +|ML |integer |Minimum slots required to enter hub +|XL |integer |Maximum slots for entering hub +|MU |integer |Minimum hubs connected where clients can be users +|MR |integer |Minimum hubs connected where client can be registered +|MO |integer |Minimum hubs connected where client can be operators +|XU |integer |Maximum hubs connected where clients can be users +|XR |integer |Maximum hubs connected where client can be registered +|XO |integer |Maximum hubs connected where client can be operators +|MC |integer |Maximum possible clients ( users ) who can connect +|UP |integer |Hub uptime (seconds) +|NI |string |Hub name (from BASE) +|DE |string |Hub description (from BASE) +VE |string |Hub software version (from BASE) +|===== The hub must continue to send the user list as for a normal user (move to NORMAL state). The pinger may decide to go through or disconnect (eg. if it @@ -196,24 +193,27 @@ The idea behind this extension is to generate a public hublist from the users favorite hublist. Implementations should separate between public and private hubs in the favorite hublist of an user, in order not to distribute private hubs where one can not connect to anyway. ==== GFA + GFA Contexts: T, C Asks all users within the same hub with the correct feature to send all publicly available hubs, in their favorite hub list to the requesting client. ==== RFA + RFA Contexts: C Response of a client. -HA | Hub address +|===== +|HA |Hub address +|LG |Last succesfull login time ( number of seconds since the epoch (1970), (UTC) ) +|===== -LG | Last succesfull login time ( number of seconds since the epoch (1970), (UTC) ) - All INF fields from BASE are inherited. All INF fields from PING extension are inherited. -=== UCMD +=== UCMD - User commands CMD name @@ -225,14 +225,13 @@ Name uniquely (per hub) identifies a particular user command. The name may contain "/" to indicate a logical structure on the viewing client, where each "/" introduces a new submenu level. Other than name, the command also has a number of flags that further detail what to do with it. -[separator="|"] -``_ -RM | 1 = Remove Command -CT | Message Category, 1 = Hub command, client parameters only, 2 = User list command, client and user parameters, 4 = Search result command, client, user and file parameters, 8 = File list command, client, user and file parameters. Multiple types are specified by adding the numbers together. -TT | The full text to be sent to hub, including FOURCC, parameters and keywords. -CO | 1 = Constrained, when sending this command on multiple users (for example in search results), constrain it to once per CID only -SP | 1 = Insert separator instead of command name (name must still be present to uniquely identify the command). -___ +|===== +|RM |1 = Remove Command +|CT |Message Category, 1 = Hub command, client parameters only, 2 = User list command, client and user parameters, 4 = Search result command, client, user and file parameters, 8 = File list command, client, user and file parameters. Multiple types are specified by adding the numbers together. +|TT |The full text to be sent to hub, including FOURCC, parameters and keywords. +|CO |1 = Constrained, when sending this command on multiple users (for example in search results), constrain it to once per CID only +|SP |1 = Insert separator instead of command name (name must still be present to uniquely identify the command). +|===== ==== Keywords Keywords are specified using "%[keyword]". Unknown keywords must be replaced by the empty string. Additionally, all %-substitutions of the C function "strftime" must be supported. @@ -240,46 +239,41 @@ The following tables specify the keywords that must be supported. Client parameters -[separator="|"] -``_ -myCID | Client CID -mySID | ClientSID -myXX | One for each flag on that particular hub; for example, myI4 and myNI -___ +|===== +|myCID |Client CID +|mySID |ClientSID +|myXX |One for each flag on that particular hub; for example, myI4 and myNI +|===== User parameters -[separator="|"] -``_ -userCID | User CID -userSID | User SID -userXX | One for each flag on the user sent; for example, userI4 and userNI -line:info | Prompts the user for input where 'info' is the displayed text description for the user input -___ +|===== +|userCID |User CID +|userSID |User SID +|userXX |One for each flag on the user sent; for example, userI4 and userNI +|line:info |Prompts the user for input where 'info' is the displayed text description for the user input +|===== File parameters -[separator="|"] -``_ -fileXX | One for each flag contained within a search result or file list entry (see RES) -fileMN | Specify magnet link. Magnet links <http://en.wikipedia.org/wiki/Magnet_link> are used to reference files across networks and applications. Clients may ignore parameters it does not understand, but are free to pass on the parameters to other programs. -___ +|===== +|fileXX |One for each flag contained within a search result or file list entry (see RES) +|fileMN |Specify magnet link. http://en.wikipedia.org/wiki/Magnet_link[Magnet links] are used to reference files across networks and applications. Clients may ignore parameters it does not understand, but are free to pass on the parameters to other programs. +|===== Hub parameters -[separator="|"] -``_ -hubXX | One for each flag of the hub; for example, hubNI and hubVE -___ +|===== +|hubXX |One for each flag of the hub; for example, hubNI and hubVE +|===== ==== Example -`_ -ICMD ADCH++/Hub{backslash}smanagement/Register{backslash}snick TTHMSG{backslash}s+regnick{backslash}s%[userNI]{backslash}s%[line:Password{backslash}s(leave{backslash}sempty{backslash}sto{backslash}sun-reg)]{backslash}s%[line:Level{backslash}s(facultative;{backslash}sdefaults{backslash}sto{backslash}syour{backslash}sown{backslash}slevel{backslash}sminus{backslash}sone)]{backslash}n CT2 -ICMD ADCH++/Hub{backslash}smanagement/Reload{backslash}sbans TTHMSG{backslash}s+loadbans{backslash}n CT3 -ICMD ADCH++/Hub{backslash}smanagement/Reload{backslash}sscripts TTHMSG{backslash}s+reload{backslash}n CT3 -ICMD ADCH++/Info TTHMSG{backslash}s+info{backslash}s%[userNI]{backslash}n CT2 -ICMD ADCH++/Info TTHMSG{backslash}s+info{backslash}n CT1 -___ - +|===== +|ICMD ADCH++/Hub\smanagement/Register\snick TTHMSG\s+regnick\s%[userNI]\s%[line:Password\s(leave\sempty\sto\sun-reg)]\s%[line:Level\s(facultative;\sdefaults\sto\syour\sown\slevel\sminus\sone)]\n CT2 +|ICMD ADCH++/Hub\smanagement/Reload\sbans TTHMSG\s+loadbans\n CT3 +|ICMD ADCH++/Hub\smanagement/Reload\sscripts TTHMSG\s+reload\n CT3 +|ICMD ADCH++/Info TTHMSG\s+info\s%[userNI]\n CT2 +|ICMD ADCH++/Info TTHMSG\s+info\n CT1 +|===== -=== BLOM +=== BLOM- Bloom filter Bloom filters allow the hub to filter certain searches using bitmap that represents the hashes of the files in the users share. BLOM is an extension that allows hub software to create a map (bloom filter) of the shared files on the hub, but with minimal effort, e.g. the hub doesn't keep a list of files, but a filter that never produces false negatives but only possible false positives. This can potentially save bandwidth and effort on the client side. When the user updates the share, the client must send an INF containing the flag SF. The hub may at any time request that the client sends an updated version of its bloom filter by sending a GET command to the client. The client will then respond using SND and send the bloom filter binary data. @@ -348,6 +342,45 @@ ===== Tiger For TTH roots, b is 192 and a reasonable value for h is 24, giving a maximum k = 8 which means that m = 8 * n / ln 2 ≈ 11.5 * n. The required bandwidth then becomes 11.5 * n / 8 bytes, so approximately 1.44 bytes per file in the users share. For 20000 files, m should then be 230016 (taking into account the modulo 64 requirement), giving a p = 0.004, in other words ~99.6% of all searches that don't match a users share would not be sent, saving valuable upload bandwidth for the hub. The client calculates i valid positions, if x is an array of bytes containing the hash of the file, on a little-endian machine, by doing pos = x[0+i*h/8] | (x[1+i*h/8] << 8) | (x[2+i*h/8] << 16) for i = [0;k). This is of course a special case where h % 8 = 0, the actual algorithm has to take into consideration values for h where the integer crosses byte boundaries. -For test vectors, see the ADC wiki talk page; http://www.adcportal.com/wiki/index.php/Talk:BLOM +For test vectors, see the http://www.adcportal.com/wiki/index.php/Talk:BLOM[ADC wiki talk page]. +=== NATT - NAT traversal +NAT traversal allow two passive clients to connect to each other. For more information about NAT traversal, see https://dcpp.wordpress.com/2010/02/13/passive-mode-c-c-connections-and-nat-traversal[Passive Mode C-C Connections and NAT Traversal]. + +If a client does not support TCP4 or TCP6, it will send an RCM to the client it is trying to connect to. If the other client also doesn't support TCP4 (or TCP6 correspondingly), NAT traversal may instead be used. The feature should be signalled in SUP as NATT. + +Do note that the hub must forward I4 or I6 for respective clients' INF. + +==== BASE RCM updates +When receiving an RCM and the client does not support TCP4 or TCP6, and if NAT-T is supported in the remote client, a NAT command should be sent repeating the protocol and token. The port shall be the outbound port to the connected hub. + +==== NAT + NAT protocol port token + +Contexts: T + +States: NORMAL + +Upon receiving this, try and connect to the specified port. An RNT command should be sent repeating the protocol and token. The port shall be the outbound port to the connected hub. Upon receiving this, try and connect to the specified port. + +==== RNT + RNT protocol port token + +Contexts: T + +States: NORMAL + +Upon receiving this, try and connect to the specified port. + +==== Example +Client A is connected to hub A with the outbound port 1000 and client B is connected to hub A with the outbound port 2000. Client A has the SID AAAA and client B has the SID BBBB. + +---- +Client A: DRCM AAAA BBBB ADC/1.0 foobar +Client B: DNAT BBBB AAAA ADC/1.0 2000 foobar +<Client A connects to client B's IP address and port 2000> +Client A: DRNT AAAA BBBB ADC/1.0 1000 foobar +<Client B connects to client A's IP address and port 1000> +---- + // vim: set syntax=asciidoc: This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <ul...@us...> - 2010-07-12 12:19:16
|
Revision: 19 http://adc.svn.sourceforge.net/adc/?rev=19&view=rev Author: ullner Date: 2010-07-12 12:19:09 +0000 (Mon, 12 Jul 2010) Log Message: ----------- Style fix for postscript Modified Paths: -------------- trunk/ADC-EXT.txt Modified: trunk/ADC-EXT.txt =================================================================== --- trunk/ADC-EXT.txt 2010-07-12 09:53:35 UTC (rev 18) +++ trunk/ADC-EXT.txt 2010-07-12 12:19:09 UTC (rev 19) @@ -278,7 +278,6 @@ Bloom filters allow the hub to filter certain searches using bitmap that represents the hashes of the files in the users share. BLOM is an extension that allows hub software to create a map (bloom filter) of the shared files on the hub, but with minimal effort, e.g. the hub doesn't keep a list of files, but a filter that never produces false negatives but only possible false positives. This can potentially save bandwidth and effort on the client side. When the user updates the share, the client must send an INF containing the flag SF. The hub may at any time request that the client sends an updated version of its bloom filter by sending a GET command to the client. The client will then respond using SND and send the bloom filter binary data. ==== Legend - |===== |b |Number of bits used for file hashes |n |Number of files in the user's share @@ -291,19 +290,16 @@ The hub chooses k, h and m. ==== Restrictions - -[width="15%"] |===== |k * h < = b |h < = 64 -|2^h > m +|2^h^ > m |m mod 64 == 0 |===== ==== Probability - |===== -|p == (1 - (1 - 1 / m)\^(k * n))^k |False positives +|p == (1 - (1 - 1 / m)^(k * n)^)^k^ |False positives |p == 0 |False negatives |===== @@ -323,8 +319,6 @@ For the GET type, m / 8 shall be used as byte amount. Updates GET with the following flags; - -[width="15%"] |===== |BK |Specify 'k'. |BH |Specify 'h'. @@ -336,7 +330,7 @@ Once the hub has received the bloom filter bit array, for each search command it processes, if it contains a hash search term, it can skip broadcasting the search to a particular client if at least one of the k bits in that clients bit array is "0", calculating positions as the client does when setting bits to "1". The hub has to evaluate the filter for each client that it has a bloom filter for, for each search. ==== Probability calculations -p = (1 - (1 - 1 / m)\^(k * n))\^k, thus p becomes smaller as m grows and larger as n grows. Larger m means more bits to transfer but also fewer false positives. The optimum value for k given m and n is (m / n) * ln 2. The largest k supported by a hash of a certain size is b / h, so if the hub wants the smallest p possible, it should choose the smallest possible h which gives the largest k, and then calculate m = k * n/ln 2, checking that the resulting m < 2\^h. 2^h should much be larger than m (at least 3-4 times), because of how the modulo operator works. Also, with m grows the required bandwidth to transfer the bloom filter, so the hub may wish to cap m. In that case, it should still choose k according to m / n * ln 2, but send an h as big as possible to alleviate biasing problems. +p = (1 - (1 - 1 / m)^(k * n)^)^k^, thus p becomes smaller as m grows and larger as n grows. Larger m means more bits to transfer but also fewer false positives. The optimum value for k given m and n is (m / n) * ln 2. The largest k supported by a hash of a certain size is b / h, so if the hub wants the smallest p possible, it should choose the smallest possible h which gives the largest k, and then calculate m = k * n/ln 2, checking that the resulting m < 2^h^. 2^h^ should much be larger than m (at least 3-4 times), because of how the modulo operator works. Also, with m grows the required bandwidth to transfer the bloom filter, so the hub may wish to cap m. In that case, it should still choose k according to m / n * ln 2, but send an h as big as possible to alleviate biasing problems. ==== Sample implementations ===== Tiger This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <ul...@us...> - 2010-07-12 12:58:27
|
Revision: 20 http://adc.svn.sourceforge.net/adc/?rev=20&view=rev Author: ullner Date: 2010-07-12 12:58:20 +0000 (Mon, 12 Jul 2010) Log Message: ----------- Added autowidth to tables to so they text fit appropriately to columns. Changed level 1 titles to real example use. Modified Paths: -------------- trunk/ADC-EXT.txt Modified: trunk/ADC-EXT.txt =================================================================== --- trunk/ADC-EXT.txt 2010-07-12 12:19:09 UTC (rev 19) +++ trunk/ADC-EXT.txt 2010-07-12 12:58:20 UTC (rev 20) @@ -80,6 +80,7 @@ In SCH and GFI, the following attributes are added: +[options="autowidth"] |===== |TR |Tiger tree Hash root, encoded with base32. |TD |Tree depth, index of the highest level of tree data available, root-only = 0, first level (2 leaves) = 1, second level = 2, etc... @@ -128,7 +129,7 @@ normal INF along with the following added fields ( none mandatory, if not present, it means hub has no restrictions in that matter, or non existent): -[options="header"] +[options="header, autowidth"] |===== |Code | Type | Description |HH |string |Hub Host address ( DNS or IP ) @@ -160,14 +161,13 @@ doesn't require additional information about the users). ===== Example ----- +==== -pinger- HSUP ADBASE ADPING AD.. -hub- ISUP ADBASE ADPING AD.. -hub- ISID .. -hub- IINF NIhubname DEcurrent\stopic VE.. HHmyhub.no-ip.org:555 WShttp://myhub.no-ip.org/ OWmyname UC2231 SS.. SF.. MS0 ML0 MC5000 - (pinger may disconnect) - -.. ----- +==== ==== Hub - Hublist communication @@ -178,13 +178,13 @@ name and WS hublist web address. ===== Example ----- +==== -hub- HSUP ADBASE ADPING AD.. -hublist- ISUP ADBASE ADPING AD.. -hublist- IINF NIhublist_name WShublist_address -hub- HINF NIhubname DEcurrent\stopic VE.. HHmyhub.no-ip.org:555 WShttp://myhub.no-ip.org/ OWmyname UC2231 SS.. SF.. MS0 ML0 MC5000 -( disconnect ) ----- +==== === TS - Timestamp in MSG Timestamp of the moment when the message was sent, expressed in seconds since the Unix Epoch (January 1 1970 00:00:00 GMT). @@ -206,6 +206,7 @@ Response of a client. +[options="autowidth"] |===== |HA |Hub address |LG |Last succesfull login time ( number of seconds since the epoch (1970), (UTC) ) @@ -225,6 +226,7 @@ Name uniquely (per hub) identifies a particular user command. The name may contain "/" to indicate a logical structure on the viewing client, where each "/" introduces a new submenu level. Other than name, the command also has a number of flags that further detail what to do with it. +[options="autowidth"] |===== |RM |1 = Remove Command |CT |Message Category, 1 = Hub command, client parameters only, 2 = User list command, client and user parameters, 4 = Search result command, client, user and file parameters, 8 = File list command, client, user and file parameters. Multiple types are specified by adding the numbers together. @@ -239,6 +241,7 @@ The following tables specify the keywords that must be supported. Client parameters +[options="autowidth"] |===== |myCID |Client CID |mySID |ClientSID @@ -246,6 +249,7 @@ |===== User parameters +[options="autowidth"] |===== |userCID |User CID |userSID |User SID @@ -254,30 +258,37 @@ |===== File parameters +[options="autowidth"] |===== |fileXX |One for each flag contained within a search result or file list entry (see RES) |fileMN |Specify magnet link. http://en.wikipedia.org/wiki/Magnet_link[Magnet links] are used to reference files across networks and applications. Clients may ignore parameters it does not understand, but are free to pass on the parameters to other programs. |===== Hub parameters +[options="autowidth"] |===== |hubXX |One for each flag of the hub; for example, hubNI and hubVE |===== ==== Example -|===== -|ICMD ADCH++/Hub\smanagement/Register\snick TTHMSG\s+regnick\s%[userNI]\s%[line:Password\s(leave\sempty\sto\sun-reg)]\s%[line:Level\s(facultative;\sdefaults\sto\syour\sown\slevel\sminus\sone)]\n CT2 -|ICMD ADCH++/Hub\smanagement/Reload\sbans TTHMSG\s+loadbans\n CT3 -|ICMD ADCH++/Hub\smanagement/Reload\sscripts TTHMSG\s+reload\n CT3 -|ICMD ADCH++/Info TTHMSG\s+info\s%[userNI]\n CT2 -|ICMD ADCH++/Info TTHMSG\s+info\n CT1 -|===== +==== +ICMD ADCH++/Hub\smanagement/Register\snick TTHMSG\s+regnick\s%[userNI]\s%[line:Password\s(leave\sempty\sto\sun-reg)]\s%[line:Level\s(facultative;\sdefaults\sto\syour\sown\slevel\sminus\sone)]\n CT2 +ICMD ADCH++/Hub\smanagement/Reload\sbans TTHMSG\s+loadbans\n CT3 + +ICMD ADCH++/Hub\smanagement/Reload\sscripts TTHMSG\s+reload\n CT3 + +ICMD ADCH++/Info TTHMSG\s+info\s%[userNI]\n CT2 + +ICMD ADCH++/Info TTHMSG\s+info\n CT1 +==== + === BLOM- Bloom filter Bloom filters allow the hub to filter certain searches using bitmap that represents the hashes of the files in the users share. BLOM is an extension that allows hub software to create a map (bloom filter) of the shared files on the hub, but with minimal effort, e.g. the hub doesn't keep a list of files, but a filter that never produces false negatives but only possible false positives. This can potentially save bandwidth and effort on the client side. When the user updates the share, the client must send an INF containing the flag SF. The hub may at any time request that the client sends an updated version of its bloom filter by sending a GET command to the client. The client will then respond using SND and send the bloom filter binary data. ==== Legend +[options="autowidth"] |===== |b |Number of bits used for file hashes |n |Number of files in the user's share @@ -290,6 +301,7 @@ The hub chooses k, h and m. ==== Restrictions +[options="autowidth"] |===== |k * h < = b |h < = 64 @@ -298,6 +310,7 @@ |===== ==== Probability +[options="autowidth"] |===== |p == (1 - (1 - 1 / m)^(k * n)^)^k^ |False positives |p == 0 |False negatives @@ -319,6 +332,7 @@ For the GET type, m / 8 shall be used as byte amount. Updates GET with the following flags; +[options="autowidth"] |===== |BK |Specify 'k'. |BH |Specify 'h'. @@ -369,12 +383,16 @@ ==== Example Client A is connected to hub A with the outbound port 1000 and client B is connected to hub A with the outbound port 2000. Client A has the SID AAAA and client B has the SID BBBB. ----- +==== Client A: DRCM AAAA BBBB ADC/1.0 foobar + Client B: DNAT BBBB AAAA ADC/1.0 2000 foobar + <Client A connects to client B's IP address and port 2000> + Client A: DRNT AAAA BBBB ADC/1.0 1000 foobar + <Client B connects to client A's IP address and port 1000> ----- +==== // vim: set syntax=asciidoc: This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <ul...@us...> - 2010-07-13 10:16:07
|
Revision: 21 http://adc.svn.sourceforge.net/adc/?rev=21&view=rev Author: ullner Date: 2010-07-13 10:15:57 +0000 (Tue, 13 Jul 2010) Log Message: ----------- Added RF field to STA. Modified Paths: -------------- trunk/ADC-EXT.txt Modified: trunk/ADC-EXT.txt =================================================================== --- trunk/ADC-EXT.txt 2010-07-12 12:58:20 UTC (rev 20) +++ trunk/ADC-EXT.txt 2010-07-13 10:15:57 UTC (rev 21) @@ -32,6 +32,7 @@ === Version 1.0.4 UNRELEASED * Added magnet link extension to 'UCMD' * Added NAT traversal extension 'NATT' +* Added referral field to STA == Extensions @@ -395,4 +396,19 @@ <Client B connects to client A's IP address and port 1000> ==== +=== RF - Referrer notification +Extends the RF field of the INF to STA, allowing the client to notify a hub upon SUP-negotiation from where it was redirected. + +[options="autowidth"] +|===== +|RF |URL of referer (hub in case of redirect, web page) +|===== + +==== Example +==== +CSUP ADBASE (...) + +CSTA 000 referrer RFadc://example.com:1234 +==== + // vim: set syntax=asciidoc: This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <ul...@us...> - 2010-07-13 10:18:18
|
Revision: 22 http://adc.svn.sourceforge.net/adc/?rev=22&view=rev Author: ullner Date: 2010-07-13 10:18:12 +0000 (Tue, 13 Jul 2010) Log Message: ----------- Minor phrasing fix for RF. Modified Paths: -------------- trunk/ADC-EXT.txt Modified: trunk/ADC-EXT.txt =================================================================== --- trunk/ADC-EXT.txt 2010-07-13 10:15:57 UTC (rev 21) +++ trunk/ADC-EXT.txt 2010-07-13 10:18:12 UTC (rev 22) @@ -397,11 +397,11 @@ ==== === RF - Referrer notification -Extends the RF field of the INF to STA, allowing the client to notify a hub upon SUP-negotiation from where it was redirected. +Extends the RF field of the INF to STA, allowing a client to notify a client upon SUP-negotiation from where the C-C originated from. [options="autowidth"] |===== -|RF |URL of referer (hub in case of redirect, web page) +|RF |URL of referrer |===== ==== Example This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <ul...@us...> - 2010-07-13 10:21:28
|
Revision: 23 http://adc.svn.sourceforge.net/adc/?rev=23&view=rev Author: ullner Date: 2010-07-13 10:21:22 +0000 (Tue, 13 Jul 2010) Log Message: ----------- RF can be a general STA command, regardless of hub or client... Modified Paths: -------------- trunk/ADC-EXT.txt Modified: trunk/ADC-EXT.txt =================================================================== --- trunk/ADC-EXT.txt 2010-07-13 10:18:12 UTC (rev 22) +++ trunk/ADC-EXT.txt 2010-07-13 10:21:22 UTC (rev 23) @@ -397,7 +397,7 @@ ==== === RF - Referrer notification -Extends the RF field of the INF to STA, allowing a client to notify a client upon SUP-negotiation from where the C-C originated from. +Extends the RF field of the INF to STA, allowing a client to notify clients and hubs upon SUP-negotiation from where the C-C or C-H originated from. [options="autowidth"] |===== This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <ul...@us...> - 2010-07-13 14:14:35
|
Revision: 24 http://adc.svn.sourceforge.net/adc/?rev=24&view=rev Author: ullner Date: 2010-07-13 14:14:29 +0000 (Tue, 13 Jul 2010) Log Message: ----------- Added PFSR (partial file sharing) Modified Paths: -------------- trunk/ADC-EXT.txt Modified: trunk/ADC-EXT.txt =================================================================== --- trunk/ADC-EXT.txt 2010-07-13 10:21:22 UTC (rev 23) +++ trunk/ADC-EXT.txt 2010-07-13 14:14:29 UTC (rev 24) @@ -33,6 +33,8 @@ * Added magnet link extension to 'UCMD' * Added NAT traversal extension 'NATT' * Added referral field to STA +* Added upload queue field to STA +* Added partial file sharing extension 'PFSR' == Extensions @@ -411,4 +413,42 @@ CSTA 000 referrer RFadc://example.com:1234 ==== +=== QP - Upload queue notification +This extension's purpose is creating a queue on a client, when multiple other clients want to download from it, but they have no slots. Currently, when a slot is being freed, the first connecting client gets it. Other clients that don't have the luck of getting in time to attempt to download, have to wait again. The client who creates a queue must have a ticket number for each connecting client, which must be kept internally , and a difference between current connecting client's queue number and the currently uploading client's be provided to the connecting client, so that the clients are being deserved in the order they originally connected. The client could have a ticket incrementing starting from 1 for each session. Connecting slients must use the same token as they used when originally connected. + +[options="autowidth"] +|===== +|QP |Queue number, representing how many others are in front in the queue. +|===== + +==== Example +The following example will notify that the client's slots are full and that there are three uploads in the queue. +==== +CSTA 253 No\sslots\savailable QP3 +==== + +=== PFSR - Partial file sharing +Partial File Sharing allows sharing of files which are available in user's download queue or in finished downloads list. As a result of this, new files will be spread much faster over whole network. + +When client receives search request (SCH), it looks into shared list to see whether requested hash is available between shared files. If it's found, everything is processed as normally. In other case, it looks into download queue (then, into finished downloads list) and receives a list of available chunks for requested hash. It mustn't take every chunk, but only that ones which are fully downloaded and has already been verified (e.g. against TTH leaves). + + The feature should be signalled in SUP as PFSR. + +==== PSR + PSR + +Contexts: F, U + +States: NORMAL + +Information about downloaded and verified chunks is sent to requester with this command. When the requester is active command is sent over UDP (with U context) else it is sent over hub (with D context). +[options="autowidth"] +|==== +|U4 |UDP IPv4 port or 0 if we don't want receive a response. If UDP port is not zero (it must be when we are passive user), a PSR receiver must respond us with PSR including his available (and verified) chunks. The client must also set UDP port to zero to avoid infinite PSR sending. +|U6 | Same as U4 but with IPv6 port. +|TR |File hash +|PC |Count of downloaded (and verified) chunks the user has. +|PI |Comma separated list of start and end index of chunks the user has. If a user has a continous block of chunks, the intermittent start and end indexes may be omitted. For example, if a user has chunk 0 and 4 of total 5 chunks, then PI is "0,1,4,5". If the user has chunk 0, 1 and 4, then PI may be "0,1,1,2,4,5" or "0,2,4,5". +|==== + // vim: set syntax=asciidoc: This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <ul...@us...> - 2010-08-29 09:50:20
|
Revision: 25 http://adc.svn.sourceforge.net/adc/?rev=25&view=rev Author: ullner Date: 2010-08-29 09:50:14 +0000 (Sun, 29 Aug 2010) Log Message: ----------- Updating the spec to 1.0.4 status; * Added magnet link extension to 'UCMD' * Added NAT traversal extension 'NATT' * Added referral field to STA * Added upload queue field to STA * Added partial file sharing extension 'PFSR' Modified Paths: -------------- trunk/ADC-EXT.txt Modified: trunk/ADC-EXT.txt =================================================================== --- trunk/ADC-EXT.txt 2010-07-13 14:14:29 UTC (rev 24) +++ trunk/ADC-EXT.txt 2010-08-29 09:50:14 UTC (rev 25) @@ -1,6 +1,6 @@ = ADC Extensions Fredrik Ullner <ul...@gm...> -1.0.4 UNRELEASED, July 2010 +1.0.4, August 2010 == Abstract These are the official extensions to ADC. This document is based on the @@ -29,7 +29,7 @@ * Removed optional keywords from UCMD * Added BLOM extension -=== Version 1.0.4 UNRELEASED +=== Version 1.0.4 * Added magnet link extension to 'UCMD' * Added NAT traversal extension 'NATT' * Added referral field to STA @@ -432,7 +432,7 @@ When client receives search request (SCH), it looks into shared list to see whether requested hash is available between shared files. If it's found, everything is processed as normally. In other case, it looks into download queue (then, into finished downloads list) and receives a list of available chunks for requested hash. It mustn't take every chunk, but only that ones which are fully downloaded and has already been verified (e.g. against TTH leaves). - The feature should be signalled in SUP as PFSR. +The feature should be signalled in SUP as PFSR. ==== PSR PSR @@ -441,7 +441,7 @@ States: NORMAL -Information about downloaded and verified chunks is sent to requester with this command. When the requester is active command is sent over UDP (with U context) else it is sent over hub (with D context). +Information about downloaded and verified chunks is sent to requester with this command. When the requester is active command is sent over UDP (with U type) else it is sent over hub (with D type). [options="autowidth"] |==== |U4 |UDP IPv4 port or 0 if we don't want receive a response. If UDP port is not zero (it must be when we are passive user), a PSR receiver must respond us with PSR including his available (and verified) chunks. The client must also set UDP port to zero to avoid infinite PSR sending. This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <ul...@us...> - 2010-09-03 19:45:18
|
Revision: 26 http://adc.svn.sourceforge.net/adc/?rev=26&view=rev Author: ullner Date: 2010-09-03 19:45:11 +0000 (Fri, 03 Sep 2010) Log Message: ----------- Starting 1.0.5 with; User locale notification in INF Modified Paths: -------------- trunk/ADC-EXT.txt Modified: trunk/ADC-EXT.txt =================================================================== --- trunk/ADC-EXT.txt 2010-08-29 09:50:14 UTC (rev 25) +++ trunk/ADC-EXT.txt 2010-09-03 19:45:11 UTC (rev 26) @@ -1,6 +1,6 @@ = ADC Extensions Fredrik Ullner <ul...@gm...> -1.0.4, August 2010 +1.0.5, September 2010 == Abstract These are the official extensions to ADC. This document is based on the @@ -36,6 +36,9 @@ * Added upload queue field to STA * Added partial file sharing extension 'PFSR' +=== Version 1.0.5 UNRELEASED +* Added locale field to INF + == Extensions === TIGR - Tiger tree hash support @@ -414,7 +417,7 @@ ==== === QP - Upload queue notification -This extension's purpose is creating a queue on a client, when multiple other clients want to download from it, but they have no slots. Currently, when a slot is being freed, the first connecting client gets it. Other clients that don't have the luck of getting in time to attempt to download, have to wait again. The client who creates a queue must have a ticket number for each connecting client, which must be kept internally , and a difference between current connecting client's queue number and the currently uploading client's be provided to the connecting client, so that the clients are being deserved in the order they originally connected. The client could have a ticket incrementing starting from 1 for each session. Connecting slients must use the same token as they used when originally connected. +This extension's purpose is creating a queue on a client, when multiple other clients want to download from it, but they have no slots. Currently, when a slot is being freed, the first connecting client gets it. Other clients that don't have the luck of getting in time to attempt to download, have to wait again. The client who creates a queue must have a ticket number for each connecting client, which must be kept internally , and a difference between current connecting client's queue number and the currently uploading client's be provided to the connecting client, so that the clients are being deserved in the order they originally connected. The client could have a ticket incrementing starting from 1 for each session. Connecting clients must use the same token as they used when originally connected. [options="autowidth"] |===== @@ -451,4 +454,16 @@ |PI |Comma separated list of start and end index of chunks the user has. If a user has a continous block of chunks, the intermittent start and end indexes may be omitted. For example, if a user has chunk 0 and 4 of total 5 chunks, then PI is "0,1,4,5". If the user has chunk 0, 1 and 4, then PI may be "0,1,1,2,4,5" or "0,2,4,5". |==== +=== LC - Locale specification +This extension's purpose is to notify the hub which user locale the client is using as well as the default locale for the hub. This allows hubs to customize text sent to clients, depending on language, left-to-right or right-to-left and more. If the hub does not directly support the client's locale, it should attempt to fall back to the same language group (e.g. hub supports en-US but not en-AU, so falls back to en-US), and if this is not available, then fall back to the hub's own locale. + +http://tools.ietf.org/html/bcp47[BCP47] should be used as reference for locale structure. + +[options="autowidth"] +|===== +|LC |User locale +|===== + +Note that the standard suggest that the language should be in lowercase and the country in upper case. Note that the country code may be more than two characters. Additionally, dash ('-') and underscore ('_') are acceptable seperators. + // vim: set syntax=asciidoc: This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <ul...@us...> - 2010-09-05 14:34:41
|
Revision: 27 http://adc.svn.sourceforge.net/adc/?rev=27&view=rev Author: ullner Date: 2010-09-05 14:34:35 +0000 (Sun, 05 Sep 2010) Log Message: ----------- Modified line in UCMD to handle multiple inputs. Modified Paths: -------------- trunk/ADC-EXT.txt Modified: trunk/ADC-EXT.txt =================================================================== --- trunk/ADC-EXT.txt 2010-09-03 19:45:11 UTC (rev 26) +++ trunk/ADC-EXT.txt 2010-09-05 14:34:35 UTC (rev 27) @@ -38,6 +38,7 @@ === Version 1.0.5 UNRELEASED * Added locale field to INF +* Modified user parameter 'line' in 'UCMD' to handle multiple inputs. == Extensions @@ -260,7 +261,7 @@ |userCID |User CID |userSID |User SID |userXX |One for each flag on the user sent; for example, userI4 and userNI -|line:info |Prompts the user for input where 'info' is the displayed text description for the user input +|line:info |Prompts the user for input where 'info' is the displayed text description for the user input. 'info' can be used for multiple values (combobox etc), and should be structured as "general info/default selection/value0/value1/.../valueN". The general info can be viewed as the caption of the user input dialog. Default selection is an integer k which signifies the default value to be used. Note that 0<=k<=N and N>=1. Note that values are 0-index based. Values are separated with a forward slash ('/'). If a forward slash is part of a value, it should be escaped by using an additional forward slash ('//'). |===== File parameters This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |