From: Jan B. <jb...@gm...> - 2011-09-30 23:13:59
|
> There've been some changes in websockets, but Yaws hasn't caught up to > them yet. Chrome probably supports the changes whereas Safari doesn't. Chrome 14 implements the hybi 10 version, according to http://en.wikipedia.org/wiki/WebSocket#Browser_support A project team I'm in is looking at adding support for this websocket version. I've already got the handshake working without breaking the hixie 76 support (https://github.com/jbothma/yaws/tree/websocket_hy10) but the frame format has also changed. Supporting the new frames will require tracking the version of each websocket (until there's one standard to support) - I'm not sure I can do that without affecting the api yet. This is to know how to format frames when using yaws_api:websocket_send. Supporting only the new protocol would be much easier, but would break support for websocket on FF6, Chromium 12, whatever else is still on the old protocol. What's the chance of getting an implementation merged into your project? Any chance that you'd allow breaking support for the old ones? JD |
From: Wes J. <com...@gm...> - 2011-10-01 04:29:35
|
Hi! On Fri, Sep 30, 2011 at 5:13 PM, Jan Bothma <jb...@gm...> wrote: >> There've been some changes in websockets, but Yaws hasn't caught up to >> them yet. Chrome probably supports the changes whereas Safari doesn't. > > Chrome 14 implements the hybi 10 version, according to > http://en.wikipedia.org/wiki/WebSocket#Browser_support > > A project team I'm in is looking at adding support for this websocket version. Thanks for doing this work! <snip> When I run this I had to change: [<<"client-connected">>] to ["client-connected"] in this code: websocket_owner() -> ...... {ok, Messages} -> case Messages of ["client-connected"] -> yaws_api:websocket_setopts(WebSocket, [{active, true}]), echo_server(WebSocket); in this file: https://github.com/jbothma/yaws/blob/websocket_hy10/www/websockets_example_endpoint.yaws To get it to work properly. -wes |
From: Wes J. <com...@gm...> - 2011-10-01 05:05:51
|
On Fri, Sep 30, 2011 at 10:29 PM, Wes James <com...@gm...> wrote: > Hi! > > On Fri, Sep 30, 2011 at 5:13 PM, Jan Bothma <jb...@gm...> wrote: >>> There've been some changes in websockets, but Yaws hasn't caught up to >>> them yet. Chrome probably supports the changes whereas Safari doesn't. >> >> Chrome 14 implements the hybi 10 version, according to >> http://en.wikipedia.org/wiki/WebSocket#Browser_support Also, I've noticed that there is a binary or base64 process in some of the examples I've seen. I.e. in the javascript: if (window.chrome) var socket = new WebSocket(host, 'base64') // chrome 14 else var socket = new WebSocket(host) // safari, chrome 13 and in the handshake I've seen ... "Sec-WebSocket-Accept: ", ...., "\r\n", "Sec-WebSocket-Protocol: base64\r\n\r\n", the "Sec-WebSocket-Protocol:", but not in your implementation. When I send data back to the websocket right now it's just closing the websocket after the call and my browser just gets the message the websocket closed. in the endpoint: ... yaws_api:websocket_send(WebSocket, Data), echo_server(WebSocket); ... -wes |
From: Jan B. <jb...@gm...> - 2011-10-01 06:55:10
|
On 1 October 2011 07:05, Wes James <com...@gm...> wrote: > On Fri, Sep 30, 2011 at 10:29 PM, Wes James <com...@gm...> wrote: >> Hi! >> >> On Fri, Sep 30, 2011 at 5:13 PM, Jan Bothma <jb...@gm...> wrote: >>>> There've been some changes in websockets, but Yaws hasn't caught up to >>>> them yet. Chrome probably supports the changes whereas Safari doesn't. >>> >>> Chrome 14 implements the hybi 10 version, according to >>> http://en.wikipedia.org/wiki/WebSocket#Browser_support > [<<"client-connected">>] > > to > > ["client-connected"] Thanks, fixed. > Also, I've noticed that there is a binary or base64 process in some of > the examples I've seen. I.e. in the javascript: > > if (window.chrome) > var socket = new WebSocket(host, 'base64') // chrome 14 > else > var socket = new WebSocket(host) // safari, chrome 13 > Good point! I haven't seen anything about websocket subprotocol selection in the older versions. So far I don't pay attention to Sec-WebSocket-Protocol headers, and the fact that they exist means upgrading websocket in yaws will be a bit more work than simply changing the handshake and framing. Thanks for the feedback. Regarding maintaining support... According to http://blog.chromium.org/search/label/websockets, "The WebSocket protocol specification is now largely stable, having solved previous security concerns." which suggests that other vendors are likely to add support for the new protocol (hybi 10) and enable it by default, while the old protocol (hixie 76) is what prompted some to disable it by default. If the security concerns are dealt with, I think there's little argument for keeping the old protocols and a lot of argument for supporting the new. JD |
From: Steve V. <vi...@ie...> - 2011-10-01 22:07:50
|
On Sat, Oct 1, 2011 at 2:55 AM, Jan Bothma <jb...@gm...> wrote: > > Regarding maintaining support... According to > http://blog.chromium.org/search/label/websockets, "The WebSocket > protocol specification is now largely stable, having solved previous > security concerns." which suggests that other vendors are likely to > add support for the new protocol (hybi 10) and enable it by default, > while the old protocol (hixie 76) is what prompted some to disable it > by default. > > If the security concerns are dealt with, I think there's little > argument for keeping the old protocols and a lot of argument for > supporting the new. I believe most folks here using WebSockets today understand it's a moving target, and expect to have to modify their code to keep up with evolution of the specification. The primary issue isn't so much Yaws, as we can fix that, but rather browser support, but of course people can just use whatever Yaws version supports the flavor of WebSockets they need most. So, my own preference is not to try to support old and new versions of WebSockets at the same time, but rather would like to just upgrade Yaws to the latest spec and drop support for old WebSockets versions. I can't speak for Klacke, but if he has a preference here, don't worry, he'll let us know. And yes, all patches and help to bring things up to date are appreciated. --steve |
From: Wes J. <com...@gm...> - 2011-10-02 02:48:41
|
Here is an example in python. It looks like there is a hybi decode and encode: https://github.com/kanaka/websockify -wes |
From: Wes J. <com...@gm...> - 2011-10-02 03:47:44
|
Hmm. I've set everything to use base64 javascript var socket = new WebSocket(host, 'base64') added this to the handshake: "Sec-WebSocket-Protocol: base64\r\n", and the echo_server seems to send the data, but chrome doesn't receive the data. I have message(socket.readyState) in the javascript and it sends the connect and gets connected but never receives the data sent back. It seems the python code is just base64 encoding the data and prepping the frame....?? -wes |
From: Jan B. <jb...@gm...> - 2011-10-02 21:38:10
|
Sending text messages shorter than 126 bytes are working for Chrome 14, and it seems like whatever worked in previous browsers should work for now. https://github.com/jbothma/yaws/commit/05956b9316c5aa515fe3d8d6721d5ff9c505be2f This is really rough, but it's a start. On 2 October 2011 05:47, Wes James <com...@gm...> wrote: > var socket = new WebSocket(host, 'base64') > > added this to the handshake: > > "Sec-WebSocket-Protocol: base64\r\n", > Ah, just realised the handshake parts for the old versions echoes back whatever the client sends in its protocol header. The stuff I added for chrome 14 doesn't. You could add a header to yaws_websocket:handshake/6 if you want it before I get to it. Eventually the endpoint could pass a list of its supported protocols to handshake/3 as fourth argument, letting the handshake send back the highest priority supported subprotocol requested by the client. http://tools.ietf.org/html/draft-ietf-hybi-thewebsocketprotocol-17#section-1.9 I implemented sending text hybi 10 frames shorter than 126 bytes in https://github.com/jbothma/yaws/commit/05956b9316c5aa515fe3d8d6721d5ff9c505be2f JD |
From: Jan B. <jb...@gm...> - 2011-10-02 22:03:49
|
On 2 October 2011 23:37, Jan Bothma <jb...@gm...> wrote: > > I implemented sending text hybi 10 frames shorter than 126 bytes in > https://github.com/jbothma/yaws/commit/05956b9316c5aa515fe3d8d6721d5ff9c505be2f > Forgot to mention, this changes some of the yaws_api:websocket... functions so existing endpoints will need modification, other than the example which I updated in the same changeset. JD |
From: Wes J. <com...@gm...> - 2011-10-04 22:58:53
|
On Sun, Oct 2, 2011 at 3:37 PM, Jan Bothma <jb...@gm...> wrote: > Sending text messages shorter than 126 bytes are working for Chrome > 14, and it seems like whatever worked in previous browsers should work > for now. I've been experimenting with: websocket_send(Socket, ProtocolVersion, IoList) -> io:format("~n lenth iolist: ~p ~n",[length(IoList)]), case length(IoList) >=125 of true -> B1= <<129,126,0,125,49,98,108,97,104,32,98,108,97,104,32,98,108,97,104,32,98, 108,97,104,32,98,108,97,104,32,98,108,97,104,32,98,108,97,104,32,98,108, 97,104,32,98,108,97,104,32,98,108,97,104,32,98,108,97,104,32,98,108,97, 104,32,98,108,97,104,32,98,108,97,104,32,98,108,97,104,32,98,108,97,104, 32,98,108,97,104,32,98,108,97,104,32,98,108,97,104,32,98,108,97,104,32, 98,108,97,104,32,98,108,97,104,32,98,108,97,104,32,98,108,97,104,32,98, 108,97,104>>, B2= <<129,8,97,100,100,32,116,104,105,115>>, gen_tcp:send(Socket, B1), timer:sleep(3000), gen_tcp:send(Socket, B2); _ -> DataFrame = yaws_websockets:frame(ProtocolVersion, IoList), case Socket of {sslsocket,_,_} -> ssl:send(Socket, DataFrame); _ -> gen_tcp:send(Socket, DataFrame) end end. and all I get is the B2 in my web browser. I did $('#cssid').append(m.data) and it did work. I can't figure out how to get all the data sent back to the web socket. I've event tried B3 = << B1/binary, B2/binary >> and do gen_tcp:send(B3), but only the data for B2 shows up in the DIV (#cssid div content). B1 and B2 are made up like this from another test: frame(_, Data) -> %io:format("~n data frame: ~s ~n",[Data]), %FIN=true because we're not fragmenting. %OPCODE=1 for text FirstByte = 128 bor 1, ByteList = Data, DataB=list_to_binary(ByteList), ByteList2="add this", DataB2=list_to_binary(ByteList2), Length1 = length(ByteList), Length2 = length(ByteList2), Length = Length1 + Length2, case Length =< 125 of true -> SecondByte = Length1, << FirstByte, SecondByte, DataB:Length1/binary >>; _ -> SecondByte = 126, ThirdByte = Length1 bsr 8, FourthByte = Length1 band 255, B1= << FirstByte, SecondByte, ThirdByte, FourthByte, DataB:Length1/binary >>, SecondByte2 = Length2, B2= << FirstByte, SecondByte2, DataB2:Length2/binary >>, io:format("~n B1 ~p ~n",[B1]), io:format("~n B2 ~p ~n",[B2]), B1, << B1/binary, B2/binary>> end. but only B2 shows up. I guess I don't know how to push the packets properly. I thought B1 was made up so that somewhere the system would wait for the rest which is B2, finish and send all the data accumulated. Based on this snippet: http://stackoverflow.com/questions/7087522/php-websocket-server-hybi10 -wes |
From: Jan B. <jb...@gm...> - 2011-10-06 20:36:33
|
On 6 October 2011 22:35, Jan Bothma <jb...@gm...> wrote: > On 5 October 2011 00:58, Wes James <com...@gm...> wrote: > >> B1= <<129,126,0,125,49,98,108,97,104,32,98,108,97,104,32,98,108,97,104,32,98, >> 108,97,104,32,98,108,97,104,32,98,108,97,104,32,98,108,97,104,32,98,108, >> 97,104,32,98,108,97,104,32,98,108,97,104,32,98,108,97,104,32,98,108,97, >> 104,32,98,108,97,104,32,98,108,97,104,32,98,108,97,104,32,98,108,97,104, >> 32,98,108,97,104,32,98,108,97,104,32,98,108,97,104,32,98,108,97,104,32, >> 98,108,97,104,32,98,108,97,104,32,98,108,97,104,32,98,108,97,104,32,98, >> 108,97,104>>, >> >> B2= <<129,8,97,100,100,32,116,104,105,115>>, >> gen_tcp:send(Socket, B1), >> timer:sleep(3000), >> gen_tcp:send(Socket, B2); >> > ... >> >> and all I get is the B2 in my web browser. > > Haven't looked in detail, but if you're trying to send two frames with > the data fragmented between them, the first bytes should be different > because of the Fin bit. > > Anwyay, I've improved long messages (although there's a bug for > multibyte characters I think) > https://github.com/jbothma/yaws/commit/9d34f72ffde4ea6ec33c399b203c74e986c83c0d > so all your data might work with my updated frame and unframe. > > Once I've fixed the bug I'll add binary message support which I think > would change the api for sending and receiving again (to indicate > opcode). > > Regards, > JD > Forgot the list, sorry |
From: Wes J. <com...@gm...> - 2011-10-06 20:44:38
|
I also noticed when I do socket.close() in javascript that the echo_server doesn't receive it as {tcp_close, _, _}, but gets sent as a msg that needs to be unframed. The message is: <<136,128,122,11,29,0>> which turns out empty but has opcode 8. According to the spec, that means socket closed. I'll take a look at your changes. Thanks, -wes On Thu, Oct 6, 2011 at 2:35 PM, Jan Bothma <jb...@gm...> wrote: > On 5 October 2011 00:58, Wes James <com...@gm...> wrote: > >> B1= <<129,126,0,125,49,98,108,97,104,32,98,108,97,104,32,98,108,97,104,32,98, >> 108,97,104,32,98,108,97,104,32,98,108,97,104,32,98,108,97,104,32,98,108, >> 97,104,32,98,108,97,104,32,98,108,97,104,32,98,108,97,104,32,98,108,97, >> 104,32,98,108,97,104,32,98,108,97,104,32,98,108,97,104,32,98,108,97,104, >> 32,98,108,97,104,32,98,108,97,104,32,98,108,97,104,32,98,108,97,104,32, >> 98,108,97,104,32,98,108,97,104,32,98,108,97,104,32,98,108,97,104,32,98, >> 108,97,104>>, >> >> B2= <<129,8,97,100,100,32,116,104,105,115>>, >> gen_tcp:send(Socket, B1), >> timer:sleep(3000), >> gen_tcp:send(Socket, B2); >> > ... >> >> and all I get is the B2 in my web browser. > > Haven't looked in detail, but if you're trying to send two frames with > the data fragmented between them, the first bytes should be different > because of the Fin bit. > > Anwyay, I've improved long messages (although there's a bug for > multibyte characters I think) > https://github.com/jbothma/yaws/commit/9d34f72ffde4ea6ec33c399b203c74e986c83c0d > so all your data might work with my updated frame and unframe. > > Once I've fixed the bug I'll add binary message support which I think > would change the api for sending and receiving again (to indicate > opcode). > > Regards, > JD > |
From: Wes J. <com...@gm...> - 2011-10-06 23:24:41
|
On Thu, Oct 6, 2011 at 2:35 PM, Jan Bothma <jb...@gm...> wrote: <snip> > Haven't looked in detail, but if you're trying to send two frames with > the data fragmented between them, the first bytes should be different > because of the Fin bit. > > Anwyay, I've improved long messages (although there's a bug for > multibyte characters I think) > https://github.com/jbothma/yaws/commit/9d34f72ffde4ea6ec33c399b203c74e986c83c0d > so all your data might work with my updated frame and unframe. This works. I had messed with the code a bit to loop through the data and send it back to the client. Your's is doing a similar thing. I have to accumulate the data in a javascript temp_var and then check for </table> in the incoming message and then transfer in to div via $('#id').html(temp_var). If I don't, just the last chunk gets put in the div. I thought it would come back as one big chunk like the other websocket protocols. I guess not. thanks, -wes |
From: Wes J. <com...@gm...> - 2011-10-06 23:38:32
|
ignore the last message..... I forgot to go back to the regular functions in yaws_api.erl. Your work is doing what I was wanting all along. It gets the whole chunk back to the client properly! Awesome!! Thanks Jan! -wes On Thu, Oct 6, 2011 at 5:24 PM, Wes James <com...@gm...> wrote: > On Thu, Oct 6, 2011 at 2:35 PM, Jan Bothma <jb...@gm...> wrote: > <snip> > >> Haven't looked in detail, but if you're trying to send two frames with >> the data fragmented between them, the first bytes should be different >> because of the Fin bit. >> >> Anwyay, I've improved long messages (although there's a bug for >> multibyte characters I think) >> https://github.com/jbothma/yaws/commit/9d34f72ffde4ea6ec33c399b203c74e986c83c0d >> so all your data might work with my updated frame and unframe. |
From: Jan B. <jb...@gm...> - 2011-10-09 13:37:46
|
On 7 October 2011 01:38, Wes James <com...@gm...> wrote: > ignore the last message..... > > I forgot to go back to the regular functions in yaws_api.erl. Your > work is doing what I was wanting all along. It gets the whole chunk > back to the client properly! > > Awesome!! > > Thanks Jan! Excellent :D I'm now looking at making a test suite to be able to regression test as the rest of the protocol gets implemented. This will also help as the protocol gets finalised. I read that Chromium 16 will bring Websocket version 16 [1]. I think the closing handshake changes a bit between versions 8 and 16, and as you found, I haven't even considered handling closing yet :) Best JD [1] http://groups.google.com/a/chromium.org/group/chromium-extensions/browse_thread/thread/9f8214e3a9ed037f > > -wes > > On Thu, Oct 6, 2011 at 5:24 PM, Wes James <com...@gm...> wrote: >> On Thu, Oct 6, 2011 at 2:35 PM, Jan Bothma <jb...@gm...> wrote: >> <snip> >> >>> Haven't looked in detail, but if you're trying to send two frames with >>> the data fragmented between them, the first bytes should be different >>> because of the Fin bit. >>> >>> Anwyay, I've improved long messages (although there's a bug for >>> multibyte characters I think) >>> https://github.com/jbothma/yaws/commit/9d34f72ffde4ea6ec33c399b203c74e986c83c0d >>> so all your data might work with my updated frame and unframe. > |
From: Jan B. <jb...@gm...> - 2011-10-15 22:28:46
|
On 9 October 2011 15:37, Jan Bothma <jb...@gm...> wrote: > I'm now looking at making a test suite I'm running the Autobahn test suite against my changes. I'll publish useful results, e.g. http://jbothma.co.uk/websocket/autobahn_yaws_c8e27/ for commit https://github.com/jbothma/yaws/commit/c8e27e36a334e588a03e09110546c9f4234138b4 This commit and the one before it fixed a bug that caused incorrect framing of messages of 126 bytes, and adds buffering to allow receiving frames bigger than 8192 bytes (yaws default receive buffer size). The latter is only done for tcp, I'm not paying attention to ssl. I'm using the following config but if you want to run the tests against my branch, note that cases 9.* eat your ram because of my rough buffering. See the commit message for details :) { "servers": [{"agent": "yaws-1.91_jbothma_hy10", "hostname": "my hostname", "port": 8000, "version": 8, "path": "/websockets_autobahn_endpoint.yaws" }], "cases": ["*"] } I changed the api:websocket* functions again so that WebSocket in the api functions means {Socket, ProtocolVersion} as suggested by Ivann Elizaryev. Regards JD |