Menu
▾
▴
naviserver-devel
|
From: Zoran V. <zv...@ar...> - 2006-09-25 19:36:20
|
Unix is not Unix as I see... Please note this (interesting) document: http://developers.sun.com/solaris/articles/event_completion.html Mostly interesting, as there is now a very powerful and scalable notification interface on both Solaris and Mac OSX (aka BSD) Unixes. Windows has it as well, with Linux hanging pretty much behind (unfortunately). I wonder if we should start making ifdefs or wrappers to be able to benefit from the corresponding event notification interface available on the current platform. Mainly this would affect poll() usage in the driver thread but can also be used all arround the code where we have to wait for multiple "things" to happen. Does anybody have something to comment about that? Ah yes.., why "brawe new world"? Because Unix OS is diverging and OS vendors are adding new (incompatible) functionality on a monthly basis. I wonder how this will all look in 10 years in future... Cheers Zoran |
|
From: Vlad S. <vl...@cr...> - 2006-09-25 19:42:00
|
We use Linux only, so it will not affect us but if using new API will change the way the program works, how it will be possible to support Solaris, Linux, Windows and MaxOSX at the same time? Zoran Vasiljevic wrote: > Unix is not Unix as I see... > > Please note this (interesting) document: > > http://developers.sun.com/solaris/articles/event_completion.html > > Mostly interesting, as there is now a very powerful and scalable > notification interface on both Solaris and Mac OSX (aka BSD) > Unixes. Windows has it as well, with Linux hanging pretty much > behind (unfortunately). > > I wonder if we should start making ifdefs or wrappers to be able > to benefit from the corresponding event notification interface > available on the current platform. > > Mainly this would affect poll() usage in the driver thread > but can also be used all arround the code where we have to > wait for multiple "things" to happen. > > Does anybody have something to comment about that? > > Ah yes.., why "brawe new world"? Because Unix OS is diverging > and OS vendors are adding new (incompatible) functionality > on a monthly basis. I wonder how this will all look in 10 > years in future... > > Cheers > Zoran > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > naviserver-devel mailing list > nav...@li... > https://lists.sourceforge.net/lists/listinfo/naviserver-devel > -- Vlad Seryakov 571 262-8608 office vl...@cr... http://www.crystalballinc.com/vlad/ |
|
From: Zoran V. <zv...@ar...> - 2006-09-25 20:00:45
|
On 25.09.2006, at 21:42, Vlad Seryakov wrote: > We use Linux only, so it will not affect us but if using new API will > change the way the program works, how it will be possible to support > Solaris, Linux, Windows and MaxOSX at the same time? The same way as it is possible to maintain Windows code in parallel... i.e. by defining wrapping procedures and ifdef's. This is obviously something that we will have to do more and more in the future. |
|
From: Mike <nee...@gm...> - 2006-09-25 22:27:01
|
On 9/25/06, Zoran Vasiljevic <zv...@ar...> wrote: > Unix is not Unix as I see... > > Please note this (interesting) document: > > http://developers.sun.com/solaris/articles/event_completion.html > > Mostly interesting, as there is now a very powerful and scalable > notification interface on both Solaris and Mac OSX (aka BSD) > Unixes. Windows has it as well, with Linux hanging pretty much > behind (unfortunately). > > I wonder if we should start making ifdefs or wrappers to be able > to benefit from the corresponding event notification interface > available on the current platform. > > Mainly this would affect poll() usage in the driver thread > but can also be used all arround the code where we have to > wait for multiple "things" to happen. > > Does anybody have something to comment about that? > > Ah yes.., why "brawe new world"? Because Unix OS is diverging > and OS vendors are adding new (incompatible) functionality > on a monthly basis. I wonder how this will all look in 10 > years in future... I highly recommend a download and read-through of lighttpd sources. They have a very simple abstraction that handles the available interfaces (kqueue, poll, and friends). Since it's also a web server, I imagine the API can be adopted (that, and the fact that the code is very well written and clean make it an ideal candidate to examine, in my opinion). |
|
From: Zoran V. <zv...@ar...> - 2006-09-26 08:08:04
|
On 26.09.2006, at 00:26, Mike wrote: > I highly recommend a download and read-through of lighttpd sources. > They have a very simple abstraction that handles the available > interfaces (kqueue, poll, and friends). Indeed. They use their own abstraction of fdevents which translates to either pool, port, devpool, kqueue or whatever is available on the target system. This is a good starting point for our own endeavours. Thanks for the tip! |
|
From: Stephen D. <sd...@gm...> - 2006-10-06 21:41:15
|
On 9/25/06, Zoran Vasiljevic <zv...@ar...> wrote: > Unix is not Unix as I see... > > Please note this (interesting) document: > > http://developers.sun.com/solaris/articles/event_completion.html > > Mostly interesting, as there is now a very powerful and scalable > notification interface on both Solaris and Mac OSX (aka BSD) > Unixes. Windows has it as well, with Linux hanging pretty much > behind (unfortunately). > > I wonder if we should start making ifdefs or wrappers to be able > to benefit from the corresponding event notification interface > available on the current platform. > > Mainly this would affect poll() usage in the driver thread > but can also be used all arround the code where we have to > wait for multiple "things" to happen. > > Does anybody have something to comment about that? What's the goal? We can certainly abstract socket event IO, with poll() as a fallback. But some of the interfaces you mentioned here can handle waiting on other things, e.g. signals. Is this what you want? There's a start at socket event IO in nsd/event.c. I didn't try wrapping anything other than poll(). My main motivation was to create a nice interface to clean up the main driver loop in nsd/driver.c, and also to make something capable of handling the reader/writer stuff that's in there, which is a little trickier due to the IO being spread among more than one thread. I have some patches for it somewhere. IIRC, the locking isn't flexible enough... But it's code that exists so you can hack on it if you think it's suitable. > Ah yes.., why "brawe new world"? Because Unix OS is diverging > and OS vendors are adding new (incompatible) functionality > on a monthly basis. I wonder how this will all look in 10 > years in future... You'll be using Linux like every one else and it won't matter... :-) |
|
From: Zoran V. <zv...@ar...> - 2006-10-06 22:01:53
|
On 06.10.2006, at 23:41, Stephen Deasey wrote: > > What's the goal? We can certainly abstract socket event IO, with > poll() as a fallback. But some of the interfaces you mentioned here > can handle waiting on other things, e.g. signals. Is this what you > want? At the moment only as replacement for poll as kqueue or solaris port_* are way more scalable. > > There's a start at socket event IO in nsd/event.c. I didn't try > wrapping anything other than poll(). My main motivation was to create > a nice interface to clean up the main driver loop in nsd/driver.c, and > also to make something capable of handling the reader/writer stuff > that's in there, which is a little trickier due to the IO being spread > among more than one thread. > > I have some patches for it somewhere. IIRC, the locking isn't > flexible enough... > > But it's code that exists so you can hack on it if you think it's > suitable. Basically that is what I thought. To get rid of the old-fashioned poll as there are much better options today. > > > You'll be using Linux like every one else and it won't matter... :-) > Ah... lets see... In any case, this is interesting development. |
|
From: Stephen D. <sd...@gm...> - 2006-10-06 22:51:22
|
On 10/6/06, Zoran Vasiljevic <zv...@ar...> wrote: > > On 06.10.2006, at 23:41, Stephen Deasey wrote: > > > > > What's the goal? We can certainly abstract socket event IO, with > > poll() as a fallback. But some of the interfaces you mentioned here > > can handle waiting on other things, e.g. signals. Is this what you > > want? > > At the moment only as replacement for poll as kqueue or solaris > port_* are way more scalable. That's not necessarily the case. This only matters when you have many sockets open simultaneously, where 'many' would be in the thousands, or thereabouts. poll() is slow with many sockets because for each spin the entire array of sockets must be checked, even if poll() woke up because just one of them has an event. It's the scanning of the array which is the bottleneck. But there are techniques you can use to work around this. For example, the old patch I posted to the AOLserver bug tracker which accepts multiple new sockets on every spin. You basically drain the listen backlog. The tests in the paper about this show that poll() based implementation became just as scalable as the fancier event notification mechanisms. Something else we can try is a Linux and BSD socket option which causes a listening socket to only generate a readable event when a new socket arrives *and* there is data to read. Usually, you get one when a new connection arrives, you accept(), then you poll() again for data, then you read it. Anyway, I'm all for using epoll() and so on, I think it's a great idea. But it's probably not going to be the performance hack of the century... |
|
From: Zoran V. <zv...@ar...> - 2006-10-07 14:32:42
|
On 07.10.2006, at 00:51, Stephen Deasey wrote: > Something else we can try is a Linux and BSD socket option which > causes a listening socket to only generate a readable event when a new > socket arrives *and* there is data to read. Usually, you get one when > a new connection arrives, you accept(), then you poll() again for > data, then you read it. > > Anyway, I'm all for using epoll() and so on, I think it's a great > idea. But it's probably not going to be the performance hack of the > century... But there is nothing BAD in doing so? The only "bad" thing is the effort needed to abstract kqueue/port_*/poll/etc into a meaningful API but others have already done that so we may "borrow" from them, or invent our own solution. I do not consider this a high-priority but if we want to claim scalability and performance we should not keep sitting in the grandfathers old chevy, regarldess how "comfortable" it is... |
|
From: Stephen D. <sd...@gm...> - 2006-10-07 15:30:26
|
On 10/7/06, Zoran Vasiljevic <zv...@ar...> wrote: > > On 07.10.2006, at 00:51, Stephen Deasey wrote: > > > Something else we can try is a Linux and BSD socket option which > > causes a listening socket to only generate a readable event when a new > > socket arrives *and* there is data to read. Usually, you get one when > > a new connection arrives, you accept(), then you poll() again for > > data, then you read it. > > > > Anyway, I'm all for using epoll() and so on, I think it's a great > > idea. But it's probably not going to be the performance hack of the > > century... > > But there is nothing BAD in doing so? > The only "bad" thing is the effort needed > to abstract kqueue/port_*/poll/etc into a > meaningful API but others have already done > that so we may "borrow" from them, or invent > our own solution. > > I do not consider this a high-priority but > if we want to claim scalability and performance > we should not keep sitting in the grandfathers > old chevy, regarldess how "comfortable" it is... > No, nothing bad. I think it's a great idea! But some of the newer APIs for this are weird because they handle more than socket IO, and we are worried about portability, right? I'm just saying that to get high-performance they may not be strictly necessary. So, if it proves difficult to port to Windows or OSX... doesn't neccessarily matter, they're not going to suffer. But yeah, we should definately move in this direction. |
|
From: Vlad S. <vl...@cr...> - 2006-10-07 15:35:51
|
Stephen, i checked your patch for AS 4.1 regarding acceptmax, we can adopt it for our driver as well > > No, nothing bad. I think it's a great idea! > > But some of the newer APIs for this are weird because they handle more > than socket IO, and we are worried about portability, right? I'm just > saying that to get high-performance they may not be strictly > necessary. So, if it proves difficult to port to Windows or OSX... > doesn't neccessarily matter, they're not going to suffer. > > But yeah, we should definately move in this direction. > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > naviserver-devel mailing list > nav...@li... > https://lists.sourceforge.net/lists/listinfo/naviserver-devel > > |
|
From: Stephen D. <sd...@gm...> - 2006-10-07 17:02:26
|
We can. It's not that big of a change. I was just generally holding off on adding new things to the driver because there's a lot of code in there now and it's getting hard to manage. On 10/7/06, Vlad Seryakov <vl...@cr...> wrote: > Stephen, > > i checked your patch for AS 4.1 regarding acceptmax, we can adopt it > for our driver as well > > > > No, nothing bad. I think it's a great idea! > > > > But some of the newer APIs for this are weird because they handle more > > than socket IO, and we are worried about portability, right? I'm just > > saying that to get high-performance they may not be strictly > > necessary. So, if it proves difficult to port to Windows or OSX... > > doesn't neccessarily matter, they're not going to suffer. > > > > But yeah, we should definately move in this direction. > > > > ------------------------------------------------------------------------- > > Take Surveys. Earn Cash. Influence the Future of IT > > Join SourceForge.net's Techsay panel and you'll get the chance to share your > > opinions on IT & business topics through brief surveys -- and earn cash > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > > _______________________________________________ > > naviserver-devel mailing list > > nav...@li... > > https://lists.sourceforge.net/lists/listinfo/naviserver-devel > > > > > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys -- and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > naviserver-devel mailing list > nav...@li... > https://lists.sourceforge.net/lists/listinfo/naviserver-devel > |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X