Menu
▾
▴
Re: [tipc-discussion] EXTERNAL: Re: DGRAM/STREAM Crossover on Debian?
|
From: Jon M. <jm...@re...> - 2021-06-09 18:11:37
|
On 6/9/21 1:07 PM, Duzan, Gary D wrote: > For the record, as I got some of my application issues out of the > way and actually started using the STREAM socket, I did encounter some > odd behavior (connection timeout) which disappeared when I switched to > using a different service type. So the presence of a listening STREAM > socket didn't cause trouble for the DGRAM activity (except on > kernel-4.19), but something about the DGRAM activity appeared to cause > trouble for the STREAMs. We are still in the process of evaluating this, and what is a reasonable solution. We want to be as permissive as possible, but of course it has to work "as expected" without surprises. > > As an aside, at least some of our systems threw an EMSGSIZE when I > tried to sendmsg() on a STREAM socket with an oversized message, which > was at least a little surprising. The first message cannot be oversized, as it (in reality a data-carrying SYN) is routed as a regular SEQPACKET/DGRAM message to the server socket. Not sure if this is well documented, though. > Using explicit connect()/writev() worked, but having the implicit > connect work Would Be Nice. This is another case where knowing what is > expected to work would help, as I could always try the sendmsg() first > and fall back to connect()/writev() on EMSGSIZE, but if it is never > expected to work, then there would be no point. It is supposed to work, but I haven't heard about anybody actually using this feature in a product, and it is not part of our regular test suite. So, testing this has probably slipped though our filter at some moment. We will have to look into this too, and add it to our test suite. Regards ///jon > > Thanks. > > Gary Duzan > FIS - GT.M Core > > > ------------------------------------------------------------------------ > *From:* Duzan, Gary D via tipc-discussion > <tip...@li...> > *Sent:* Thursday, June 3, 2021 10:57 AM > *To:* Jon Maloy <jm...@re...>; Xin Long <luc...@gm...> > *Cc:* tip...@li... > <tip...@li...> > *Subject:* Re: [tipc-discussion] EXTERNAL: Re: DGRAM/STREAM Crossover > on Debian? > > I can frankly admit that this is a problem we never came across > before, and that we haven´t really spent much time considering this > kind of issues. > That's why you need crazy developers trying to do weird stuff with > it. 🙂 > > You are right that this needs to be at least clarified in the > documentation. > But we should also run some internal tests to identify what is working > and what is not, and then decide what is the correct approach. > Thank you for making us aware of this. > Thanks for your efforts in making TIPC a great tool for datacenter > communication. > > Gary Duzan > FIS - GT.M Core > > > ///jon > > > > > ________________________________ > From: Jon Maloy <jm...@re...> > Sent: Thursday, June 3, 2021 10:19 AM > To: Duzan, Gary D <Gar...@fi...>; Xin Long > <luc...@gm...> > Cc: tip...@li... > <tip...@li...> > Subject: Re: EXTERNAL: Re: [tipc-discussion] DGRAM/STREAM Crossover on > Debian? > > > > On 6/3/21 9:55 AM, Duzan, Gary D wrote: > > Contrary to UDP vs TCP, TIPC is only one protocol, so you > cannot bind the same service type/instance to different socket types > without risking problems. > The SYN bit will prevent a connection from being established with a > socket of the wrong type, but it will not stop the binding table lookup > from selecting such a socket, since it knows nothing about socket types. > I am actually surprised that this works even on the non-Debian machines. > Maybe the secondary lookup mechanism is saving the day. > > This could of course be fixed without too much effort, but the question > is if that is the right way to go. At least we would have to carefully > consider possible compatibility issues. > > Would it be a problem for you to just choose different service > types/ranges? > > ///jon > > Interesting. So, accidents of implementation aside, you would > expect DGRAM, STREAM, and SEQPACKET sockets to be able to communicate > with each other? I'm not using SEQPACKET (yet), but it sounds like one > might be able to connect() between STREAM and SEQPACKET sockets today, > though presumably the way that send()s on the STREAM side get > "packetized" on the SEQPACKET end is not clearly defined. If this is > what is intended, I think some clarification in the documentation > would be helpful, as would testing to ensure expected behavior across > socket types. It is true that bringing TCP/UDP thinking to TIPC is > going to lead to errors, but it isn't going to be rare. If it is > straightforward to keep communication between different socket types > separate, then I suspect that would produce the least surprise. > > As for my own project, logically I'm dealing with the same service > either way; it is only the DGRAM message size limit that is pushing > the STREAM fallback option. I could require a separate service type > for the STREAM implementation, but that is another piece of > configuration which I'd rather avoid, if practical. I expect I will > keep it the way I have it while I'm in experimental mode, as it seems > to be working as I expect on the target kernels, but I'll need to be > sure I'm on solid ground if we ever manage to get the thing to production. > > Thanks. > > Gary Duzan > FIS - GT.M Core > > I can frankly admit that this is a problem we never came across > before, and that we haven´t really spent much time considering this > kind of issues. > You are right that this needs to be at least clarified in the > documentation. > But we should also run some internal tests to identify what is working > and what is not, and then decide what is the correct approach. > Thank you for making us aware of this. > > ///jon > > > ________________________________ > From: Jon Maloy <jm...@re...><mailto:jm...@re...> > Sent: Wednesday, June 2, 2021 8:56 PM > To: Xin Long <luc...@gm...><mailto:luc...@gm...>; > Duzan, Gary D <Gar...@fi...><mailto:Gar...@fi...> > Cc: > tip...@li...<mailto:tip...@li...> > <tip...@li...><mailto:tip...@li...> > Subject: EXTERNAL: Re: [tipc-discussion] DGRAM/STREAM Crossover on Debian? > > CAUTION: This email originated from outside of the company. Do not > click links or open attachments unless you recognize the sender and > know the content is safe. > > > > On 6/2/21 4:14 PM, Xin Long wrote: > > On Wed, May 26, 2021 at 11:38 AM Duzan, Gary D via tipc-discussion > > > <tip...@li...><mailto:tip...@li...> > wrote: > >> I'm in the process of enhancing a TIPC DGRAM-based RPC-ish > service to include TIPC STREAM transport for larger messages. To > simplify configuration, I have the server process(es) bind() the same > type/range for both DGRAM and STREAM sockets (poll()ing to see which > have incoming requests), then choose which to use on the client. This > seems to work on most of my Linux systems (RHEL-8, Ubuntu 20.04/21.04, > Fedora 34, Debian 11), but on my Debian 10 system (4.19.181-1 kernel) > I am seeing messages from a DGRAM client appearing on an accept()ed > STREAM socket on the server. I have confirmed that the client is not > sending anything on a STREAM socket, and the message received by the > server is formatted as a DGRAM message (without the message framing > header). > > When you start two scoket on the server: DGRAM and STREAM, in the > > client's nametable there will be 2 sockets with different portids: > > # tipc nametable show > > Type Lower Upper Scope Port Node > > 18888 17 17 cluster 4063960415 > > 18888 17 17 cluster 1106254118 > > > > When the client calls sendmsg()/connect() to send msg to the server, > > it will choose one of them by the rule of "local, closest-first or > > round-robin". > > The client doesn't know if the peer is a DGRAM socket or STREAM > > socket. In your case, it should go round-robin. > > > > Without this commit: > > > > commit 25b9221b959483f17c2964d0922869e16caa86b5 > > Author: Jon Maloy > <jon...@er...><mailto:jon...@er...> > > Date: Fri Sep 28 20:23:21 2018 +0200 > > > > tipc: add SYN bit to connection setup messages > > > > The SYN msg for STREAM is no different from the DATA msg for DGRAM. > > that's what you're seeing in kernel-4.19 > > > >> Debian isn't a target platform for production, so I don't need > a specific fix, but it is still surprising and a bit disturbing. Was > this a known problem with the 4.19 kernel? Are there particular > reasons why using this pattern is a bad idea? > > I think it may not work as expected if you create 2 different types of > > TIPC sockets binding to the same address. > > At least on the latest kernel, once the DGRAM client chooses the > > STREAM socket, the DATA msg will be dropped. > > > > Thanks. > Exactly. Contrary to UDP vs TCP, TIPC is only one protocol, so you > cannot bind the same service type/instance to different socket types > without risking problems. > The SYN bit will prevent a connection from being established with a > socket of the wrong type, but it will not stop the binding table lookup > from selecting such a socket, since it knows nothing about socket types. > I am actually surprised that this works even on the non-Debian machines. > Maybe the secondary lookup mechanism is saving the day. > > This could of course be fixed without too much effort, but the question > is if that is the right way to go. At least we would have to carefully > consider possible compatibility issues. > > Would it be a problem for you to just choose different service > types/ranges? > > ///jon > > > > >> Thanks. > >> > >> Gary Duzan > >> FIS - GT.M Core > >> > >> The information contained in this message is proprietary and/or > confidential. If you are not the intended recipient, please: (i) > delete the message and all copies; (ii) do not disclose, distribute or > use the message in any manner; and (iii) notify the sender > immediately. In addition, please be aware that any message addressed > to our domain is subject to archiving and review by persons other than > the intended recipient. Thank you. > >> > >> _______________________________________________ > >> tipc-discussion mailing list > >> > tip...@li...<mailto:tip...@li...> > >> > https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Ftipc-discussion&data=04%7C01%7Cgary.duzan%40fisglobal.com%7Cc5cc8db882484ba033b008d926a229c6%7Ce3ff91d834c84b15a0b418910a6ac575%7C0%7C0%7C637583300194004465%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=0qukqL1DPfuLrcwjZJe3%2BkyQQhVbpO9KSeKv81dkYqc%3D&reserved=0<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Ftipc-discussion&data=04%7C01%7Cgary.duzan%40fisglobal.com%7Cc5cc8db882484ba033b008d926a229c6%7Ce3ff91d834c84b15a0b418910a6ac575%7C0%7C0%7C637583300194004465%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=0qukqL1DPfuLrcwjZJe3%2BkyQQhVbpO9KSeKv81dkYqc%3D&reserved=0 > <https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Ftipc-discussion&data=04%7C01%7Cgary.duzan%40fisglobal.com%7Cc5cc8db882484ba033b008d926a229c6%7Ce3ff91d834c84b15a0b418910a6ac575%7C0%7C0%7C637583300194004465%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=0qukqL1DPfuLrcwjZJe3%2BkyQQhVbpO9KSeKv81dkYqc%3D&reserved=0<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Ftipc-discussion&data=04%7C01%7Cgary.duzan%40fisglobal.com%7Cc5cc8db882484ba033b008d926a229c6%7Ce3ff91d834c84b15a0b418910a6ac575%7C0%7C0%7C637583300194004465%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=0qukqL1DPfuLrcwjZJe3%2BkyQQhVbpO9KSeKv81dkYqc%3D&reserved=0>> > > The information contained in this message is proprietary and/or > confidential. If you are not the intended recipient, please: (i) > delete the message and all copies; (ii) do not disclose, distribute or > use the message in any manner; and (iii) notify the sender > immediately. In addition, please be aware that any message addressed > to our domain is subject to archiving and review by persons other than > the intended recipient. Thank you. > > The information contained in this message is proprietary and/or > confidential. If you are not the intended recipient, please: (i) > delete the message and all copies; (ii) do not disclose, distribute or > use the message in any manner; and (iii) notify the sender > immediately. In addition, please be aware that any message addressed > to our domain is subject to archiving and review by persons other than > the intended recipient. Thank you. > > _______________________________________________ > tipc-discussion mailing list > tip...@li... > https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Ftipc-discussion&data=04%7C01%7Cgary.duzan%40fisglobal.com%7Cc5cc8db882484ba033b008d926a229c6%7Ce3ff91d834c84b15a0b418910a6ac575%7C0%7C0%7C637583300194004465%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=0qukqL1DPfuLrcwjZJe3%2BkyQQhVbpO9KSeKv81dkYqc%3D&reserved=0 > <https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Ftipc-discussion&data=04%7C01%7Cgary.duzan%40fisglobal.com%7Cc5cc8db882484ba033b008d926a229c6%7Ce3ff91d834c84b15a0b418910a6ac575%7C0%7C0%7C637583300194004465%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=0qukqL1DPfuLrcwjZJe3%2BkyQQhVbpO9KSeKv81dkYqc%3D&reserved=0> > The information contained in this message is proprietary and/or > confidential. If you are not the intended recipient, please: (i) > delete the message and all copies; (ii) do not disclose, distribute or > use the message in any manner; and (iii) notify the sender > immediately. In addition, please be aware that any message addressed > to our domain is subject to archiving and review by persons other than > the intended recipient. Thank you. |
