From: Brian F. <Bri...@qu...> - 2017-05-12 09:28:55
|
I think there might be a problem with ns_urlencode on Windows Naviserver. I just ran this example (taken directly from the documentation): ns_urlencode http://www.aolserver.com/redirect.adp?url=http://www.aol.com&t=1,2,3 And the result was unexpected: http://www.aolserver.com/redirect.adp?url%3dhttp://www.aol.com%26t%3d1%2c2%2c The correct result is http%3a%2f%2fwww%2eaolserver%2ecom%2fredirect%2eadp%3furl%3dhttp%3a%2f%2fwww%2eaol%2ecom%26t%3d1%2c2%2c3 This is breaking some of our existing AOLserver code during migration testing. Any suggestions? Brian Fenton Senior Developer Phone: +353 1 679 9933 grantmanagementsoftware.com<http://www.grantmanagementsoftware.com> [AIMS Grant and Programme Management]<http://www.grantmanagementsoftware.com/> [Twitter]<https://twitter.com/AIMS_qcl> [LinkedIn]<https://www.linkedin.com/grp/home?gid=2978658> Quest Computing Ltd. Ushers Court | 31-33 Ushers Quay | Dublin 8, Ireland This e-mail transmission may contain confidential information that is intended for the individual or entity named on the e-mail address. If you are not the intended recipient, please reply to the sender so that Quest Computing Ltd can arrange for the proper delivery and then please delete the message from your inbox. If you have received this e-mail in error, you are hereby notified that any disclosure, copying, distribution, or reliance upon the contents of this e-mail is strictly prohibited. Quest Computing Ltd., Ushers Court, 31-33 Ushers Quay, Dublin 8, Ireland. Quest Computing Ltd. is registered in Ireland No. 146435. |
From: Maurizio M. <Mau...@sp...> - 2017-05-12 11:31:40
|
Dear Brian, Is this really a "Windows Naviserver" issue or is it a matter of a new implementation of the function, different from what was previously done in Aolserver? The two implementations seem to me quite different. Thanks a lot, Maurizio From: Brian Fenton [mailto:Bri...@qu...] Sent: 12 May 2017 11:15 To: 'nav...@li...' <nav...@li...> Subject: [naviserver-devel] ns_urlencode on Windows Naviserver I think there might be a problem with ns_urlencode on Windows Naviserver. I just ran this example (taken directly from the documentation): ns_urlencode http://www.aolserver.com/redirect.adp?url=http://www.aol.com <http://www.aolserver.com/redirect.adp?url=http://www.aol.com&t=1,2,3> &t=1,2,3 And the result was unexpected: http://www.aolserver.com/redirect.adp?url%3dhttp://www.aol.com%26t%3d1%2c2%2 c The correct result is http%3a%2f%2fwww%2eaolserver%2ecom%2fredirect%2eadp%3furl%3dhttp%3a%2f%2fwww %2eaol%2ecom%26t%3d1%2c2%2c3 This is breaking some of our existing AOLserver code during migration testing. Any suggestions? Brian Fenton Senior Developer Phone: +353 1 679 9933 <http://www.grantmanagementsoftware.com> grantmanagementsoftware.com <http://www.grantmanagementsoftware.com/> <https://twitter.com/AIMS_qcl> <https://www.linkedin.com/grp/home?gid=2978658> Quest Computing Ltd. Ushers Court | 31-33 Ushers Quay | Dublin 8, Ireland This e-mail transmission may contain confidential information that is intended for the individual or entity named on the e-mail address. If you are not the intended recipient, please reply to the sender so that Quest Computing Ltd can arrange for the proper delivery and then please delete the message from your inbox. If you have received this e-mail in error, you are hereby notified that any disclosure, copying, distribution, or reliance upon the contents of this e-mail is strictly prohibited. Quest Computing Ltd., Ushers Court, 31-33 Ushers Quay, Dublin 8, Ireland. Quest Computing Ltd. is registered in Ireland No. 146435. |
From: Brian F. <Bri...@qu...> - 2017-05-12 11:33:35
|
Hi Maurizio I really don't know. I don't have access to a Linux installation to check. Are you suggesting that the API may have changed since AOLserver? Brian From: Maurizio Martignano [mailto:Mau...@sp...] Sent: 12 May 2017 12:31 To: nav...@li... Subject: Re: [naviserver-devel] ns_urlencode on Windows Naviserver Dear Brian, Is this really a "Windows Naviserver" issue or is it a matter of a new implementation of the function, different from what was previously done in Aolserver? The two implementations seem to me quite different. Thanks a lot, Maurizio From: Brian Fenton [mailto:Bri...@qu...] Sent: 12 May 2017 11:15 To: 'nav...@li...' <nav...@li...<mailto:nav...@li...>> Subject: [naviserver-devel] ns_urlencode on Windows Naviserver I think there might be a problem with ns_urlencode on Windows Naviserver. I just ran this example (taken directly from the documentation): ns_urlencode http://www.aolserver.com/redirect.adp?url=http://www.aol.com&t=1,2,3 And the result was unexpected: http://www.aolserver.com/redirect.adp?url%3dhttp://www.aol.com%26t%3d1%2c2%2c The correct result is http%3a%2f%2fwww%2eaolserver%2ecom%2fredirect%2eadp%3furl%3dhttp%3a%2f%2fwww%2eaol%2ecom%26t%3d1%2c2%2c3 This is breaking some of our existing AOLserver code during migration testing. Any suggestions? Brian Fenton Senior Developer Phone: +353 1 679 9933 grantmanagementsoftware.com<http://www.grantmanagementsoftware.com> [AIMS Grant and Programme Management]<http://www.grantmanagementsoftware.com/> [Twitter]<https://twitter.com/AIMS_qcl> [LinkedIn]<https://www.linkedin.com/grp/home?gid=2978658> Quest Computing Ltd. Ushers Court | 31-33 Ushers Quay | Dublin 8, Ireland This e-mail transmission may contain confidential information that is intended for the individual or entity named on the e-mail address. If you are not the intended recipient, please reply to the sender so that Quest Computing Ltd can arrange for the proper delivery and then please delete the message from your inbox. If you have received this e-mail in error, you are hereby notified that any disclosure, copying, distribution, or reliance upon the contents of this e-mail is strictly prohibited. Quest Computing Ltd., Ushers Court, 31-33 Ushers Quay, Dublin 8, Ireland. Quest Computing Ltd. is registered in Ireland No. 146435. |
From: David O. <da...@qc...> - 2017-05-12 11:35:55
|
--- Welcome to tlc_erp running at /usr/lib/naviserver/bin/nsd (pid 10339) NaviServer/4.99.16d5 for linux built on May 4 2017 at 16:39:38 Tag: tar-4.99.16d5 tlc_erp:nscp 1> ns_urlencode http://www.aolserver.com/redirect.adp?url=http://www.aol.com&t=1,2,3 http://www.aolserver.com/redirect.adp?url%3dhttp://www.aol.com%26t%3d1%2c2%2c3 ---- On 12 May 2017 at 12:32, Brian Fenton <Bri...@qu...> wrote: > Hi Maurizio > > > > I really don’t know. I don’t have access to a Linux installation to check. > Are you suggesting that the API may have changed since AOLserver? > > > > Brian > > > > > > *From:* Maurizio Martignano [mailto:Mau...@sp...] > *Sent:* 12 May 2017 12:31 > *To:* nav...@li... > *Subject:* Re: [naviserver-devel] ns_urlencode on Windows Naviserver > > > > Dear Brian, > > Is this really a “Windows Naviserver” issue or is it a > matter of a new implementation of the function, different from what was > previously done in Aolserver? > > > > The two implementations seem to me quite different. > > > > Thanks a lot, > > Maurizio > > > > > > *From:* Brian Fenton [mailto:Bri...@qu... <Bri...@qu...>] > > *Sent:* 12 May 2017 11:15 > *To:* 'nav...@li...' <naviserver-devel@lists. > sourceforge.net> > *Subject:* [naviserver-devel] ns_urlencode on Windows Naviserver > > > > I think there might be a problem with ns_urlencode on Windows Naviserver. > > > > I just ran this example (taken directly from the documentation): > > ns_urlencode http://www.aolserver.com/redirect.adp?url=http://www. > aol.com&t=1,2,3 > > > > And the result was unexpected: http://www.aolserver.com/ > redirect.adp?url%3dhttp://www.aol.com%26t%3d1%2c2%2c > > > > The correct result is http%3a%2f%2fwww%2eaolserver% > 2ecom%2fredirect%2eadp%3furl%3dhttp%3a%2f%2fwww%2eaol% > 2ecom%26t%3d1%2c2%2c3 > > > > This is breaking some of our existing AOLserver code during migration > testing. > > > > Any suggestions? > > > > *Brian Fenton* > *Senior Developer* > > Phone: *+353 1 679 9933 <+353%201%20679%209933>* > *grantmanagementsoftware.com* <http://www.grantmanagementsoftware.com> > > [image: AIMS Grant and Programme Management] > <http://www.grantmanagementsoftware.com/> > > > > [image: Twitter] <https://twitter.com/AIMS_qcl> > > [image: LinkedIn] <https://www.linkedin.com/grp/home?gid=2978658> > > Quest Computing Ltd. Ushers Court | 31-33 Ushers Quay | Dublin 8, Ireland > > > > > > This e-mail transmission may contain confidential information that is > intended for the individual or entity named on the e-mail address. If you > are not the intended recipient, please reply to the sender so that Quest > Computing Ltd can arrange for the proper delivery and then please delete > the message from your inbox. > > If you have received this e-mail in error, you are hereby notified that > any disclosure, copying, distribution, or reliance upon the contents of > this e-mail is strictly prohibited. Quest Computing Ltd., Ushers Court, > 31-33 Ushers Quay, Dublin 8, Ireland. Quest Computing Ltd. is registered in > Ireland No. 146435. > > > > > > ------------------------------------------------------------ > ------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > _______________________________________________ > naviserver-devel mailing list > nav...@li... > https://lists.sourceforge.net/lists/listinfo/naviserver-devel > > -- David Osborne Qcode Software Limited http://www.qcode.co.uk T: +44 (0)1463 896484 |
From: Brian F. <Bri...@qu...> - 2017-05-12 11:43:25
|
Thank you David So, assuming the documentation is incorrect https://naviserver.sourceforge.io/n/naviserver/files/ns_urlencode.html it seems that the API has been changed from AOLserver. Brian From: David Osborne [mailto:da...@qc...] Sent: 12 May 2017 12:36 To: nav...@li... Subject: Re: [naviserver-devel] ns_urlencode on Windows Naviserver --- Welcome to tlc_erp running at /usr/lib/naviserver/bin/nsd (pid 10339) NaviServer/4.99.16d5 for linux built on May 4 2017 at 16:39:38 Tag: tar-4.99.16d5 tlc_erp:nscp 1> ns_urlencode http://www.aolserver.com/redirect.adp?url=http://www.aol.com&t=1,2,3 http://www.aolserver.com/redirect.adp?url%3dhttp://www.aol.com%26t%3d1%2c2%2c3 ---- On 12 May 2017 at 12:32, Brian Fenton <Bri...@qu...<mailto:Bri...@qu...>> wrote: Hi Maurizio I really don’t know. I don’t have access to a Linux installation to check. Are you suggesting that the API may have changed since AOLserver? Brian From: Maurizio Martignano [mailto:Mau...@sp...<mailto:Mau...@sp...>] Sent: 12 May 2017 12:31 To: nav...@li...<mailto:nav...@li...> Subject: Re: [naviserver-devel] ns_urlencode on Windows Naviserver Dear Brian, Is this really a “Windows Naviserver” issue or is it a matter of a new implementation of the function, different from what was previously done in Aolserver? The two implementations seem to me quite different. Thanks a lot, Maurizio From: Brian Fenton [mailto:Bri...@qu...] Sent: 12 May 2017 11:15 To: 'nav...@li...<mailto:nav...@li...>' <nav...@li...<mailto:nav...@li...>> Subject: [naviserver-devel] ns_urlencode on Windows Naviserver I think there might be a problem with ns_urlencode on Windows Naviserver. I just ran this example (taken directly from the documentation): ns_urlencode http://www.aolserver.com/redirect.adp?url=http://www.aol.com&t=1,2,3 And the result was unexpected: http://www.aolserver.com/redirect.adp?url%3dhttp://www.aol.com%26t%3d1%2c2%2c The correct result is http%3a%2f%2fwww%2eaolserver%2ecom%2fredirect%2eadp%3furl%3dhttp%3a%2f%2fwww%2eaol%2ecom%26t%3d1%2c2%2c3 This is breaking some of our existing AOLserver code during migration testing. Any suggestions? Brian Fenton Senior Developer Phone: +353 1 679 9933<tel:+353%201%20679%209933> grantmanagementsoftware.com<http://www.grantmanagementsoftware.com> [AIMS Grant and Programme Management]<http://www.grantmanagementsoftware.com/> [Twitter]<https://twitter.com/AIMS_qcl> [LinkedIn]<https://www.linkedin.com/grp/home?gid=2978658> Quest Computing Ltd. Ushers Court | 31-33 Ushers Quay | Dublin 8, Ireland This e-mail transmission may contain confidential information that is intended for the individual or entity named on the e-mail address. If you are not the intended recipient, please reply to the sender so that Quest Computing Ltd can arrange for the proper delivery and then please delete the message from your inbox. If you have received this e-mail in error, you are hereby notified that any disclosure, copying, distribution, or reliance upon the contents of this e-mail is strictly prohibited. Quest Computing Ltd., Ushers Court, 31-33 Ushers Quay, Dublin 8, Ireland. Quest Computing Ltd. is registered in Ireland No. 146435. ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ naviserver-devel mailing list nav...@li...<mailto:nav...@li...> https://lists.sourceforge.net/lists/listinfo/naviserver-devel -- David Osborne Qcode Software Limited http://www.qcode.co.uk T: +44 (0)1463 896484 |
From: Gustaf N. <ne...@wu...> - 2017-05-12 13:51:44
|
Am 12.05.17 um 13:41 schrieb Brian Fenton: > > Thank you David > > So, assuming the documentation is incorrect > https://naviserver.sourceforge.io/n/naviserver/files/ns_urlencode.html > it seems that the API has been changed from AOLserver. > > Brian > Brian, are you worried about the literal comparison in a test case, or is there a deeper background? Some more background: ns_urlencode in AOLserver was always incorrect in respect to the RFCs, which define different encodings for the "path" and the "query" part of an URL. Therefore many years ago, NaviServer added flags "-part" to specify the encoding (implementing an extension of RFC1738 (1994)). In the (unreleased) tip version of NaviServer on bitbucket, i've updated the implementation to be in sync with valid RFCs (RFC 3986 for URIs and RFC 6265, cookie encodings). So if one compares the result of a tip version of NaviServer with the documentation of the released version, the results differ (btw., the example on the NaviServer man page is bad). See about the changes in the tip see e.g. [1] and the following postings. -g [1] https://sourceforge.net/p/naviserver/mailman/naviserver-devel/?limit=50&viewmonth=201703&viewday=22 -- Univ.Prof. Dr. Gustaf Neumann WU Vienna Institute of Information Systems and New Media Welthandelsplatz 1, A-1020 Vienna, Austria |
From: Maurizio M. <Mau...@sp...> - 2017-05-12 11:48:21
|
Thank you very much David! Maurizio From: David Osborne [mailto:da...@qc...] Sent: 12 May 2017 13:36 To: nav...@li... Subject: Re: [naviserver-devel] ns_urlencode on Windows Naviserver --- Welcome to tlc_erp running at /usr/lib/naviserver/bin/nsd (pid 10339) NaviServer/4.99.16d5 for linux built on May 4 2017 at 16:39:38 Tag: tar-4.99.16d5 tlc_erp:nscp 1> ns_urlencode http://www.aolserver.com/redirect.adp?url=http://www.aol.com <http://www.aolserver.com/redirect.adp?url=http://www.aol.com&t=1,2,3> &t=1,2,3 http://www.aolserver.com/redirect.adp?url%3dhttp://www.aol.com%26t%3d1%2c2%2c3 ---- On 12 May 2017 at 12:32, Brian Fenton <Bri...@qu... <mailto:Bri...@qu...> > wrote: Hi Maurizio I really don’t know. I don’t have access to a Linux installation to check. Are you suggesting that the API may have changed since AOLserver? Brian From: Maurizio Martignano [mailto:Mau...@sp... <mailto:Mau...@sp...> ] Sent: 12 May 2017 12:31 To: nav...@li... <mailto:nav...@li...> Subject: Re: [naviserver-devel] ns_urlencode on Windows Naviserver Dear Brian, Is this really a “Windows Naviserver” issue or is it a matter of a new implementation of the function, different from what was previously done in Aolserver? The two implementations seem to me quite different. Thanks a lot, Maurizio From: Brian Fenton [mailto:Bri...@qu...] Sent: 12 May 2017 11:15 To: 'nav...@li... <mailto:nav...@li...> ' <nav...@li... <mailto:nav...@li...> > Subject: [naviserver-devel] ns_urlencode on Windows Naviserver I think there might be a problem with ns_urlencode on Windows Naviserver. I just ran this example (taken directly from the documentation): ns_urlencode http://www.aolserver.com/redirect.adp?url=http://www.aol.com <http://www.aolserver.com/redirect.adp?url=http://www.aol.com&t=1,2,3> &t=1,2,3 And the result was unexpected: http://www.aolserver.com/redirect.adp?url%3dhttp://www.aol.com%26t%3d1%2c2%2c The correct result is http%3a%2f%2fwww%2eaolserver%2ecom%2fredirect%2eadp%3furl%3dhttp%3a%2f%2fwww%2eaol%2ecom%26t%3d1%2c2%2c3 This is breaking some of our existing AOLserver code during migration testing. Any suggestions? Brian Fenton Senior Developer Phone: +353 1 679 9933 <tel:+353%201%20679%209933> <http://www.grantmanagementsoftware.com> grantmanagementsoftware.com <http://www.grantmanagementsoftware.com/> <https://twitter.com/AIMS_qcl> <https://www.linkedin.com/grp/home?gid=2978658> Quest Computing Ltd. Ushers Court | 31-33 Ushers Quay | Dublin 8, Ireland This e-mail transmission may contain confidential information that is intended for the individual or entity named on the e-mail address. If you are not the intended recipient, please reply to the sender so that Quest Computing Ltd can arrange for the proper delivery and then please delete the message from your inbox. If you have received this e-mail in error, you are hereby notified that any disclosure, copying, distribution, or reliance upon the contents of this e-mail is strictly prohibited. Quest Computing Ltd., Ushers Court, 31-33 Ushers Quay, Dublin 8, Ireland. Quest Computing Ltd. is registered in Ireland No. 146435. ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ naviserver-devel mailing list nav...@li... <mailto:nav...@li...> https://lists.sourceforge.net/lists/listinfo/naviserver-devel -- David Osborne Qcode Software Limited http://www.qcode.co.uk T: +44 (0)1463 896484 |
From: Brian F. <Bri...@qu...> - 2017-05-12 14:19:14
|
Thanks Gustaf An issue arose during migration testing which was breaking some of our existing AOLserver code, then when I saw the documentation issue, I thought best to check with the list that maybe there was an issue with the Windows version. I'll dig into our code and see what the problem is exactly. thanks Brian From: Gustaf Neumann [mailto:ne...@wu...] Sent: 12 May 2017 14:52 To: nav...@li... Subject: Re: [naviserver-devel] ns_urlencode on Windows Naviserver Am 12.05.17 um 13:41 schrieb Brian Fenton: Thank you David So, assuming the documentation is incorrect https://naviserver.sourceforge.io/n/naviserver/files/ns_urlencode.html it seems that the API has been changed from AOLserver. Brian Brian, are you worried about the literal comparison in a test case, or is there a deeper background? Some more background: ns_urlencode in AOLserver was always incorrect in respect to the RFCs, which define different encodings for the "path" and the "query" part of an URL. Therefore many years ago, NaviServer added flags "-part" to specify the encoding (implementing an extension of RFC1738 (1994)). In the (unreleased) tip version of NaviServer on bitbucket, i've updated the implementation to be in sync with valid RFCs (RFC 3986 for URIs and RFC 6265, cookie encodings). So if one compares the result of a tip version of NaviServer with the documentation of the released version, the results differ (btw., the example on the NaviServer man page is bad). See about the changes in the tip see e.g. [1] and the following postings. -g [1] https://sourceforge.net/p/naviserver/mailman/naviserver-devel/?limit=50&viewmonth=201703&viewday=22 -- Univ.Prof. Dr. Gustaf Neumann WU Vienna Institute of Information Systems and New Media Welthandelsplatz 1, A-1020 Vienna, Austria |
From: Maurizio M. <Mau...@sp...> - 2017-05-12 15:12:37
|
Dear Brian, a short look into the code, at the two different implementations (and how much they differ) would have avoided this wrong assumption. On top of that, as Gustaf suggested, using the "-part path" option you can obtain the full URL properly converted. The main point, the main difference with the old implementation, is about how "reserved characters", like "/", ":", have to be treated. The old implementation percent-encode reserved characters in any situation. The new, and I guess correct, implementation percent-encode these character only when they are not used as delimiters. This is the relevant RFC 3986 text: " Reserved Characters URIs include components and subcomponents that are delimited by characters in the "reserved" set. These characters are called "reserved" because they may (or may not) be defined as delimiters by the generic syntax, by each scheme-specific syntax, or by the implementation-specific syntax of a URI's dereferencing algorithm. If data for a URI component would conflict with a reserved character's purpose as a delimiter, then the conflicting data must be percent-encoded before the URI is formed. " Hope it helps, Maurizio From: Brian Fenton [mailto:Bri...@qu...] Sent: 12 May 2017 16:18 To: nav...@li... Subject: Re: [naviserver-devel] ns_urlencode on Windows Naviserver Thanks Gustaf An issue arose during migration testing which was breaking some of our existing AOLserver code, then when I saw the documentation issue, I thought best to check with the list that maybe there was an issue with the Windows version. I'll dig into our code and see what the problem is exactly. thanks Brian From: Gustaf Neumann [mailto:ne...@wu...] Sent: 12 May 2017 14:52 To: nav...@li... <mailto:nav...@li...> Subject: Re: [naviserver-devel] ns_urlencode on Windows Naviserver Am 12.05.17 um 13:41 schrieb Brian Fenton: Thank you David So, assuming the documentation is incorrect https://naviserver.sourceforge.io/n/naviserver/files/ns_urlencode.html it seems that the API has been changed from AOLserver. Brian Brian, are you worried about the literal comparison in a test case, or is there a deeper background? Some more background: ns_urlencode in AOLserver was always incorrect in respect to the RFCs, which define different encodings for the "path" and the "query" part of an URL. Therefore many years ago, NaviServer added flags "-part" to specify the encoding (implementing an extension of RFC1738 (1994)). In the (unreleased) tip version of NaviServer on bitbucket, i've updated the implementation to be in sync with valid RFCs (RFC 3986 for URIs and RFC 6265, cookie encodings). So if one compares the result of a tip version of NaviServer with the documentation of the released version, the results differ (btw., the example on the NaviServer man page is bad). See about the changes in the tip see e.g. [1] and the following postings. -g [1] https://sourceforge.net/p/naviserver/mailman/naviserver-devel/?limit=50 <https://sourceforge.net/p/naviserver/mailman/naviserver-devel/?limit=50&vie wmonth=201703&viewday=22> &viewmonth=201703&viewday=22 -- Univ.Prof. Dr. Gustaf Neumann WU Vienna Institute of Information Systems and New Media Welthandelsplatz 1, A-1020 Vienna, Austria |
From: Brian F. <Bri...@qu...> - 2017-06-08 16:51:49
|
Hi all I'm back again with more questions about changes to ns_urlencode in Naviserver. 1. Can someone confirm what is the default for -part, query or path? >From my own tests, it seems to be "query" but I couldn't see it in the docs here https://naviserver.sourceforge.io/n/naviserver/files/ns_urlencode.html 2. in the case where the query portion of a URL, has a value which itself is a url, Naviserver handles this differently than AOLserver e.g. ns_urlencode "url=/wf/task?task_id=2" returns: url%3d/wf/task?task_id%3d2 Under AOLserver, this returns: url%3d%2fwf%2ftask%3ftask%5fid%3d2 This is something widely used in our application, where we have a UUID implementation, which relies on encoding and decoding URLs, and comparing them to cached values. Unfortunately this difference in how Naviserver and AOLserver implement this is giving us some issues (of our own making, admittedly). Has anyone any suggestions for how to obtain the AOLserver behaviour under Naviserver? Would it be difficult to overload ns_urlencode? Would be grateful for any advice. best wishes Brian From: Brian Fenton Sent: 12 May 2017 15:18 To: nav...@li... Subject: RE: [naviserver-devel] ns_urlencode on Windows Naviserver Thanks Gustaf An issue arose during migration testing which was breaking some of our existing AOLserver code, then when I saw the documentation issue, I thought best to check with the list that maybe there was an issue with the Windows version. I'll dig into our code and see what the problem is exactly. thanks Brian From: Gustaf Neumann [mailto:ne...@wu...] Sent: 12 May 2017 14:52 To: nav...@li...<mailto:nav...@li...> Subject: Re: [naviserver-devel] ns_urlencode on Windows Naviserver Am 12.05.17 um 13:41 schrieb Brian Fenton: Thank you David So, assuming the documentation is incorrect https://naviserver.sourceforge.io/n/naviserver/files/ns_urlencode.html it seems that the API has been changed from AOLserver. Brian Brian, are you worried about the literal comparison in a test case, or is there a deeper background? Some more background: ns_urlencode in AOLserver was always incorrect in respect to the RFCs, which define different encodings for the "path" and the "query" part of an URL. Therefore many years ago, NaviServer added flags "-part" to specify the encoding (implementing an extension of RFC1738 (1994)). In the (unreleased) tip version of NaviServer on bitbucket, i've updated the implementation to be in sync with valid RFCs (RFC 3986 for URIs and RFC 6265, cookie encodings). So if one compares the result of a tip version of NaviServer with the documentation of the released version, the results differ (btw., the example on the NaviServer man page is bad). See about the changes in the tip see e.g. [1] and the following postings. -g [1] https://sourceforge.net/p/naviserver/mailman/naviserver-devel/?limit=50&viewmonth=201703&viewday=22 -- Univ.Prof. Dr. Gustaf Neumann WU Vienna Institute of Information Systems and New Media Welthandelsplatz 1, A-1020 Vienna, Austria |
From: Gustaf N. <ne...@wu...> - 2017-06-08 19:29:54
|
Brian, It is certainly possible to overload every command, but i would not recommend it, since this can break code in various ways, and i would not recommend to rely in general on the non-standard-confoming AOLserver encoding.I would rather recommend to implement a new function (e.g. ns_urlencode_aol) which can be used in your UUID implementation. Would this be an acceptable approach for your system? i could dig out the AOLserver code and send you such an implementation, in case that would be of use for you. -g PS: there are many - also pretty standard - ways to compute an UUID. An UUID should not depend on an encoding representation, which is just intended as a temporary representation for transit, internally everything should work on the decoded values. Am 08.06.17 um 18:50 schrieb Brian Fenton: > > Hi all > > I'm back again with more questions about changes to ns_urlencode in > Naviserver. > > 1.Can someone confirm what is the default for -part, query or path? > From my own tests, it seems to be "query" but I couldn't see it in the > docs here > https://naviserver.sourceforge.io/n/naviserver/files/ns_urlencode.html > > 2. in the case where the query portion of a URL, has a value which > itself is a url, Naviserver handles this differently than AOLserver e.g. > > ns_urlencode "url=/wf/task?task_id=2" returns: > > url%3d/wf/task?task_id%3d2 > > Under AOLserver, this returns: > > url%3d%2fwf%2ftask%3ftask%5fid%3d2 > > This is something widely used in our application, where we have a UUID > implementation, which relies on encoding and decoding URLs, and > comparing them to cached values. Unfortunately this difference in how > Naviserver and AOLserver implement this is giving us some issues (of > our own making, admittedly). > > Has anyone any suggestions for how to obtain the AOLserver behaviour > under Naviserver? Would it be difficult to overload ns_urlencode? > > Would be grateful for any advice. > > best wishes > > Brian > > *From:*Brian Fenton > *Sent:* 12 May 2017 15:18 > *To:* nav...@li... > *Subject:* RE: [naviserver-devel] ns_urlencode on Windows Naviserver > > Thanks Gustaf > > An issue arose during migration testing which was breaking some of our > existing AOLserver code, then when I saw the documentation issue, I > thought best to check with the list that maybe there was an issue with > the Windows version. > I’ll dig into our code and see what the problem is exactly. > > thanks > > Brian > > *From:*Gustaf Neumann [mailto:ne...@wu...] > *Sent:* 12 May 2017 14:52 > *To:* nav...@li... > <mailto:nav...@li...> > *Subject:* Re: [naviserver-devel] ns_urlencode on Windows Naviserver > > Am 12.05.17 um 13:41 schrieb Brian Fenton: > > Thank you David > > So, assuming the documentation is incorrect > https://naviserver.sourceforge.io/n/naviserver/files/ns_urlencode.html > it seems that the API has been changed from AOLserver. > > Brian > > Brian, are you worried about the literal comparison in a test case, or > is there a deeper background? > > Some more background: ns_urlencode in AOLserver was always incorrect > in respect to the RFCs, which define different encodings for the > "path" and the "query" part of an URL. Therefore many years ago, > NaviServer added flags "-part" to specify the encoding (implementing > an extension of RFC1738 (1994)). > > In the (unreleased) tip version of NaviServer on bitbucket, i've > updated the implementation to be in sync with valid RFCs (RFC 3986 for > URIs and RFC 6265, cookie encodings). So if one compares the result > of a tip version of NaviServer with the documentation of the released > version, the results differ (btw., the example on the NaviServer man > page is bad). See about the changes in the tip see e.g. [1] and the > following postings. > > -g > > [1] > https://sourceforge.net/p/naviserver/mailman/naviserver-devel/?limit=50&viewmonth=201703&viewday=22 > > -- > Univ.Prof. Dr. Gustaf Neumann > WU Vienna > Institute of Information Systems and New Media > Welthandelsplatz 1, A-1020 Vienna, Austria > > > ------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > > > _______________________________________________ > naviserver-devel mailing list > nav...@li... > https://lists.sourceforge.net/lists/listinfo/naviserver-devel -- Univ.Prof. Dr. Gustaf Neumann WU Vienna Institute of Information Systems and New Media Welthandelsplatz 1, A-1020 Vienna, Austria |
From: Brian F. <Bri...@qu...> - 2017-06-12 14:28:42
|
Hi Gustaf Many thanks for your kind offer. For now, Enrique has found a good workaround using the TCL ::http::formatQuery command, so we'll use that. All the best Brian From: Gustaf Neumann [mailto:ne...@wu...] Sent: 08 June 2017 20:30 To: nav...@li... Subject: Re: [naviserver-devel] ns_urlencode on Windows Naviserver Brian, It is certainly possible to overload every command, but i would not recommend it, since this can break code in various ways, and i would not recommend to rely in general on the non-standard-confoming AOLserver encoding.I would rather recommend to implement a new function (e.g. ns_urlencode_aol) which can be used in your UUID implementation. Would this be an acceptable approach for your system? i could dig out the AOLserver code and send you such an implementation, in case that would be of use for you. -g PS: there are many - also pretty standard - ways to compute an UUID. An UUID should not depend on an encoding representation, which is just intended as a temporary representation for transit, internally everything should work on the decoded values. Am 08.06.17 um 18:50 schrieb Brian Fenton: Hi all I'm back again with more questions about changes to ns_urlencode in Naviserver. 1. Can someone confirm what is the default for -part, query or path? From my own tests, it seems to be "query" but I couldn't see it in the docs here https://naviserver.sourceforge.io/n/naviserver/files/ns_urlencode.html 2. in the case where the query portion of a URL, has a value which itself is a url, Naviserver handles this differently than AOLserver e.g. ns_urlencode "url=/wf/task?task_id=2" returns: url%3d/wf/task?task_id%3d2 Under AOLserver, this returns: url%3d%2fwf%2ftask%3ftask%5fid%3d2 This is something widely used in our application, where we have a UUID implementation, which relies on encoding and decoding URLs, and comparing them to cached values. Unfortunately this difference in how Naviserver and AOLserver implement this is giving us some issues (of our own making, admittedly). Has anyone any suggestions for how to obtain the AOLserver behaviour under Naviserver? Would it be difficult to overload ns_urlencode? Would be grateful for any advice. best wishes Brian From: Brian Fenton Sent: 12 May 2017 15:18 To: nav...@li...<mailto:nav...@li...> Subject: RE: [naviserver-devel] ns_urlencode on Windows Naviserver Thanks Gustaf An issue arose during migration testing which was breaking some of our existing AOLserver code, then when I saw the documentation issue, I thought best to check with the list that maybe there was an issue with the Windows version. I'll dig into our code and see what the problem is exactly. thanks Brian From: Gustaf Neumann [mailto:ne...@wu...] Sent: 12 May 2017 14:52 To: nav...@li...<mailto:nav...@li...> Subject: Re: [naviserver-devel] ns_urlencode on Windows Naviserver Am 12.05.17 um 13:41 schrieb Brian Fenton: Thank you David So, assuming the documentation is incorrect https://naviserver.sourceforge.io/n/naviserver/files/ns_urlencode.html it seems that the API has been changed from AOLserver. Brian Brian, are you worried about the literal comparison in a test case, or is there a deeper background? Some more background: ns_urlencode in AOLserver was always incorrect in respect to the RFCs, which define different encodings for the "path" and the "query" part of an URL. Therefore many years ago, NaviServer added flags "-part" to specify the encoding (implementing an extension of RFC1738 (1994)). In the (unreleased) tip version of NaviServer on bitbucket, i've updated the implementation to be in sync with valid RFCs (RFC 3986 for URIs and RFC 6265, cookie encodings). So if one compares the result of a tip version of NaviServer with the documentation of the released version, the results differ (btw., the example on the NaviServer man page is bad). See about the changes in the tip see e.g. [1] and the following postings. -g [1] https://sourceforge.net/p/naviserver/mailman/naviserver-devel/?limit=50&viewmonth=201703&viewday=22 -- Univ.Prof. Dr. Gustaf Neumann WU Vienna Institute of Information Systems and New Media Welthandelsplatz 1, A-1020 Vienna, Austria ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ naviserver-devel mailing list nav...@li...<mailto:nav...@li...> https://lists.sourceforge.net/lists/listinfo/naviserver-devel -- Univ.Prof. Dr. Gustaf Neumann WU Vienna Institute of Information Systems and New Media Welthandelsplatz 1, A-1020 Vienna, Austria |
From: Enrique C. <ec...@ec...> - 2017-06-09 13:08:24
|
Hi, Here is a good test case of Brian's point that shows more details about the problem which is when the URL has another URL as a value of an url var, check this out: % set url "/comments/view?object_id=45&return_url=/ticket/view?x=456&ticket_id=90" if we run "ns_urlencode -part query $url", in the naviserver implementation that follows the RFC #1738 as per https://naviserver.sourceforge.io/n/naviserver/files/ns_urlencode.html , we get this result: /comments/view?object_id%3d45%26return_url%3d/ticket/view?x%3d456%26ticket_id%3d90 ( it doesn't encode the / and ? in the value of return_url ). However, if we use the TCL implementation, "::http::formatQuery $url", which seems to follow a different RFC, RFC #2718, as per https://www.tcl.tk/man/tcl8.6/TclCmd/http.htm#M11 , we get this result: %2Fcomments%2Fview%3Fobject_id%3D45%26return_url%3D%2Fticket%2Fview%3Fx%3D456%26ticket_id%3D90 We understand the importance of having the flag -part to encode separately the path or the query, this is great but our concern is that when using -query, it doesn't seem to encode the / and ? when used as value of a variable. Enrique. On Thu, Jun 8, 2017 at 9:29 PM, Gustaf Neumann <ne...@wu...> wrote: > Brian, > It is certainly possible to overload every command, but i would not > recommend it, since this can break code in various ways, and i would not > recommend to rely in general on the non-standard-confoming AOLserver > encoding.I would rather recommend to implement a new function (e.g. > ns_urlencode_aol) which can be used in your UUID implementation. Would > this be an acceptable approach for your system? i could dig out the > AOLserver code and send you such an implementation, in case that would be > of use for you. > > -g > PS: there are many - also pretty standard - ways to compute an UUID. An > UUID should not depend on an encoding representation, which is just > intended as a temporary representation for transit, internally everything > should work on the decoded values. > > > Am 08.06.17 um 18:50 schrieb Brian Fenton: > > Hi all > > > > I'm back again with more questions about changes to ns_urlencode in > Naviserver. > > > > 1. Can someone confirm what is the default for -part, query or > path? From my own tests, it seems to be "query" but I couldn't see it in > the docs here https://naviserver.sourceforge.io/n/naviserver/ > files/ns_urlencode.html > > > > 2. in the case where the query portion of a URL, has a value which itself > is a url, Naviserver handles this differently than AOLserver e.g. > > ns_urlencode "url=/wf/task?task_id=2" returns: > > url%3d/wf/task?task_id%3d2 > > > > Under AOLserver, this returns: > > url%3d%2fwf%2ftask%3ftask%5fid%3d2 > > > > This is something widely used in our application, where we have a UUID > implementation, which relies on encoding and decoding URLs, and comparing > them to cached values. Unfortunately this difference in how Naviserver and > AOLserver implement this is giving us some issues (of our own making, > admittedly). > > > > Has anyone any suggestions for how to obtain the AOLserver behaviour under > Naviserver? Would it be difficult to overload ns_urlencode? > > Would be grateful for any advice. > > > > best wishes > > Brian > > > > *From:* Brian Fenton > *Sent:* 12 May 2017 15:18 > *To:* nav...@li... > *Subject:* RE: [naviserver-devel] ns_urlencode on Windows Naviserver > > > > Thanks Gustaf > > > > An issue arose during migration testing which was breaking some of our > existing AOLserver code, then when I saw the documentation issue, I thought > best to check with the list that maybe there was an issue with the Windows > version. > I’ll dig into our code and see what the problem is exactly. > > > > thanks > > Brian > > > > *From:* Gustaf Neumann [mailto:ne...@wu... <ne...@wu...>] > *Sent:* 12 May 2017 14:52 > *To:* nav...@li... > *Subject:* Re: [naviserver-devel] ns_urlencode on Windows Naviserver > > > > Am 12.05.17 um 13:41 schrieb Brian Fenton: > > Thank you David > > > > So, assuming the documentation is incorrect https://naviserver. > sourceforge.io/n/naviserver/files/ns_urlencode.html it seems that the > API has been changed from AOLserver. > > > > Brian > > Brian, are you worried about the literal comparison in a test case, or is > there a deeper background? > > Some more background: ns_urlencode in AOLserver was always incorrect in > respect to the RFCs, which define different encodings for the "path" and > the "query" part of an URL. Therefore many years ago, NaviServer added > flags "-part" to specify the encoding (implementing an extension of RFC1738 > (1994)). > > In the (unreleased) tip version of NaviServer on bitbucket, i've updated > the implementation to be in sync with valid RFCs (RFC 3986 for URIs and > RFC 6265, cookie encodings). So if one compares the result of a tip version > of NaviServer with the documentation of the released version, the results > differ (btw., the example on the NaviServer man page is bad). See about the > changes in the tip see e.g. [1] and the following postings. > > -g > > [1] https://sourceforge.net/p/naviserver/mailman/naviserver- > devel/?limit=50&viewmonth=201703&viewday=22 > > -- > > Univ.Prof. Dr. Gustaf Neumann > > WU Vienna > > Institute of Information Systems and New Media > > Welthandelsplatz 1, A-1020 Vienna, Austria > > > > ------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > > > > _______________________________________________ > naviserver-devel mailing lis...@li...https://lists.sourceforge.net/lists/listinfo/naviserver-devel > > > -- > Univ.Prof. Dr. Gustaf Neumann > WU Vienna > Institute of Information Systems and New Media > Welthandelsplatz 1, A-1020 Vienna, Austria > > > ------------------------------------------------------------ > ------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > _______________________________________________ > naviserver-devel mailing list > nav...@li... > https://lists.sourceforge.net/lists/listinfo/naviserver-devel > > |
From: Gustaf N. <ne...@wu...> - 2017-06-12 20:04:29
|
Am 09.06.17 um 14:35 schrieb Enrique Catalan: > Here is a good test case of Brian's point that shows more details > about the problem which is when the URL has another URL as a value of > an url var, check this out: > > % set > url "/comments/view?object_id=45&return_url=/ticket/view?x=456&ticket_id=90" > > if we run "ns_urlencode -part query $url", in the naviserver > implementation that follows the RFC #1738 as per > https://naviserver.sourceforge.io/n/naviserver/files/ns_urlencode.html , Dear Enrique, Since you are using the windows version of NaviServer that Maurizio provides, you are not using NaviServer 4.99.15, to which this man page refers, but some newer version (check with [ns_info patchlevel] and/or [ns_info tag]). The version on bitbucket conforms to RFC 3986 (see [1] for details). > we get this result: > > /comments/view?object_id%3d45%26return_url%3d/ticket/view?x%3d456%26ticket_id%3d90 > ( it doesn't encode the / and ? in the value of return_url ). > > However, if we use the TCL implementation, "::http::formatQuery $url", > which seems to follow a different RFC, RFC #2718, as per > https://www.tcl.tk/man/tcl8.6/TclCmd/http.htm#M11 , we get this result: > > %2Fcomments%2Fview%3Fobject_id%3D45%26return_url%3D%2Fticket%2Fview%3Fx%3D456%26ticket_id%3D90 > > We understand the importance of having the flag -part to encode > separately the path or the query, this is great but our concern is > that when using -query, it doesn't seem to encode the / and ? when > used as value of a variable. RFC 3986 updates RFC 1738 and obsoletes RFCs 2732, 2396, 1808 The detailed definition of the encoding of a query RFC 3986 is summarized in the source code [2]. The definition says clearly query = *( pchar / "/" / "?" ) which means, that slash and question mark should be unprotected. The query part is after the "?" which separates the path from the query. In a path, these characters have to be encoded. Why are you concerned? -g PS: RFC 2718 is about URL *schemes" and is cited in the Tcl manpage about the usage of UTF-8. [1] https://sourceforge.net/p/naviserver/mailman/message/35747431/ [2] https://bitbucket.org/naviserver/naviserver/src/15ad74bc59a51bb53f6feef0135049576289a388/nsd/urlencode.c?at=default&fileviewer=file-view-default#urlencode.c-232 > > Enrique. > > > > > On Thu, Jun 8, 2017 at 9:29 PM, Gustaf Neumann <ne...@wu... > <mailto:ne...@wu...>> wrote: > > Brian, > It is certainly possible to overload every command, but i would > not recommend it, since this can break code in various ways, and i > would not recommend to rely in general on the > non-standard-confoming AOLserver encoding.I would rather recommend > to implement a new function (e.g. ns_urlencode_aol) which can be > used in your UUID implementation. Would this be an acceptable > approach for your system? i could dig out the AOLserver code and > send you such an implementation, in case that would be of use for you. > > -g > PS: there are many - also pretty standard - ways to compute an > UUID. An UUID should not depend on an encoding representation, > which is just intended as a temporary representation for transit, > internally everything should work on the decoded values. > > > Am 08.06.17 um 18:50 schrieb Brian Fenton: >> >> Hi all >> >> I'm back again with more questions about changes to ns_urlencode >> in Naviserver. >> >> 1.Can someone confirm what is the default for -part, query or >> path? From my own tests, it seems to be "query" but I couldn't >> see it in the docs here >> https://naviserver.sourceforge.io/n/naviserver/files/ns_urlencode.html >> <https://naviserver.sourceforge.io/n/naviserver/files/ns_urlencode.html> >> >> 2. in the case where the query portion of a URL, has a value >> which itself is a url, Naviserver handles this differently than >> AOLserver e.g. >> >> ns_urlencode "url=/wf/task?task_id=2" returns: >> >> url%3d/wf/task?task_id%3d2 >> >> Under AOLserver, this returns: >> >> url%3d%2fwf%2ftask%3ftask%5fid%3d2 >> >> This is something widely used in our application, where we have a >> UUID implementation, which relies on encoding and decoding URLs, >> and comparing them to cached values. Unfortunately this >> difference in how Naviserver and AOLserver implement this is >> giving us some issues (of our own making, admittedly). >> >> Has anyone any suggestions for how to obtain the AOLserver >> behaviour under Naviserver? Would it be difficult to overload >> ns_urlencode? >> >> Would be grateful for any advice. >> >> best wishes >> >> Brian >> >> *From:*Brian Fenton >> *Sent:* 12 May 2017 15:18 >> *To:* nav...@li... >> <mailto:nav...@li...> >> *Subject:* RE: [naviserver-devel] ns_urlencode on Windows Naviserver >> >> Thanks Gustaf >> >> An issue arose during migration testing which was breaking some >> of our existing AOLserver code, then when I saw the documentation >> issue, I thought best to check with the list that maybe there was >> an issue with the Windows version. >> I’ll dig into our code and see what the problem is exactly. >> >> thanks >> >> Brian >> >> *From:*Gustaf Neumann [mailto:ne...@wu...] >> *Sent:* 12 May 2017 14:52 >> *To:* nav...@li... >> <mailto:nav...@li...> >> *Subject:* Re: [naviserver-devel] ns_urlencode on Windows Naviserver >> >> Am 12.05.17 um 13:41 schrieb Brian Fenton: >> >> Thank you David >> >> So, assuming the documentation is incorrect >> https://naviserver.sourceforge.io/n/naviserver/files/ns_urlencode.html >> <https://naviserver.sourceforge.io/n/naviserver/files/ns_urlencode.html> >> it seems that the API has been changed from AOLserver. >> >> Brian >> >> Brian, are you worried about the literal comparison in a test >> case, or is there a deeper background? >> >> Some more background: ns_urlencode in AOLserver was always >> incorrect in respect to the RFCs, which define different >> encodings for the "path" and the "query" part of an URL. >> Therefore many years ago, NaviServer added flags "-part" to >> specify the encoding (implementing an extension of RFC1738 (1994)). >> >> In the (unreleased) tip version of NaviServer on bitbucket, i've >> updated the implementation to be in sync with valid RFCs (RFC >> 3986 for URIs and RFC 6265, cookie encodings). So if one compares >> the result of a tip version of NaviServer with the documentation >> of the released version, the results differ (btw., the example on >> the NaviServer man page is bad). See about the changes in the tip >> see e.g. [1] and the following postings. >> >> -g >> >> [1] >> https://sourceforge.net/p/naviserver/mailman/naviserver-devel/?limit=50&viewmonth=201703&viewday=22 >> <https://sourceforge.net/p/naviserver/mailman/naviserver-devel/?limit=50&viewmonth=201703&viewday=22> >> >> -- >> Univ.Prof. Dr. Gustaf Neumann >> WU Vienna >> Institute of Information Systems and New Media >> Welthandelsplatz 1, A-1020 Vienna, Austria >> >> >> ------------------------------------------------------------------------------ >> Check out the vibrant tech community on one of the world's most >> engaging tech sites, Slashdot.org!http://sdm.link/slashdot >> >> >> _______________________________________________ >> naviserver-devel mailing list >> nav...@li... >> <mailto:nav...@li...> >> https://lists.sourceforge.net/lists/listinfo/naviserver-devel >> <https://lists.sourceforge.net/lists/listinfo/naviserver-devel> > > |