Re: [Simpleweb-Support] Binary data errors on ResponseStream?

[Niall G. <gal...@ya...>]

Hi Kai,=0A=0AThe issue you describe is quite strange. It could very well be=
 a problem I have not encountered before, perhaps its related to the output=
 buffer in the response stream. I've used Simple in quite a few projects an=
d never seen any data corruption issues. To verify this  it may be worth wh=
ile tweaking the ResponseStream object to collect the data written to it an=
d dump it out on a close, just to see if its being used to write an initial=
ly valid file. Also, setting the response buffer to zero bytes and trying w=
ould clarify if the response buffer is at fault.=0A=0ANectar sounds like a =
very interesting project. Its always interesting deploying large web based =
distributed systems, in fact it was the original reason I wrote the framewo=
rk. I had intended it to cope with very large loads, at the time there was =
no open source alternative. One thing that may help you out in this is usin=
g asynchronous capabilities of Simple. Something I have not documented very=
 much is that the request and response are completely independent of the se=
rvicing thread, unlike Servlets where the Servlet.service method must perfo=
rm the entire transaction. This can be useful in a distributed system as it=
 allows you to perform many back end operations in parallel without seriali=
zing throughput from the server. In this sense the ProtocolHandler.handle i=
s more like an event handler, or a handover, than a conventional method inv=
ocation. You may have read something similar in the Async Servlet JSR, or J=
etty continuations, both of which I feel are
 greatly hindered by legacy compatibility. =0A=0AAlso, I have nearly finish=
ed the implementation of Simple 4.0. Its almost a complete rewrite, and its=
 about 30% faster, and much more responsive. I think it will be a little wh=
ile yet before I release it though, could be a few months yet.=0A=0ANiall=
=0A=0A=0A----- Original Message ----=0AFrom: Kai Schutte <sch...@gm...=
m>=0ATo: sim...@li...=0ASent: Sunday, July 29, 2=
007 4:28:00 AM=0ASubject: [Simpleweb-Support] Binary data errors on Respons=
eStream?=0A=0AHello Niall and all the others,=0A=0AI've come to my first re=
al problem using Simple. I'd like to ask for a little help about this issue=
, then give you a little summary of what I'm doing with Simple: first of al=
l to let you know how much I appreciate Simple, and then maybe see if you h=
ave any advice.=0A=0A=0AMy specific problem is in the context of writing a =
binary file (a JPG image) to the client. I'm setting Content-type, Content-=
length, adding no cache headers. The data of the image comes from a MySQL t=
able. You may argue that storing static images in a database is a silly ide=
a, and I'd agree, this is just test case. My system runs with Tomcat, and j=
ust a few days ago I started to get it to run with Simple as well. Under To=
mcat, the image appears fine, but with Simple, the image is corrupted. =0A=
=0A=0AI've attached the two images from Tomcat and Simple, and a binary dif=
f generated by bsdiff. The two files are the same length (so it's not a cha=
racter escaping issue, and the start and end bytes are the same, so neither=
 file has been truncated) and they are very similar (the diff is very small=
). I'm thinking it might be an encoding setting problem, or maybe I'm using=
 the wrong OutputStream for binary data? I've also attached the implementat=
ion of my ProtocolHandler ,  and the action implementation that retrieves t=
he image from the database and prints it to the OutputStream. The request /=
 response objects are tossed around a few classes in between, but are never=
 really interacted with. The =0Anectar.http.Response and nectar.http.Reques=
t classes are just wrappers to interleave Tomcat's javax.servlet.http.HttpS=
ervlet(Request|Response) and Simple's equivalents. MySQL is in UTF8_bin col=
lation, but this shouldn't affect BLOB fields, right?=0A=0A=0A_____________=
___=0A=0AMy project started in 2003 as an expansion on Apache Struts (a ser=
vlet framework for Tomcat), integrating a modular database adapter (not spe=
cifically relying on SQL), a very strict Model-View-Action design for each =
webpage's functionality. Action can access 'common' functionality through S=
ervice classes, unique in the server instance. I worked 4 months straight o=
n the project non-stop after coming home to my parents from a failed .com e=
xperiment (don't we all have similar stories?). I returned to university, a=
nd only sporadically worked on the project since.=0A=0A=0AThe core design o=
f my system was very good, but Struts very quickly became more of a restric=
tion than a supporting platform. I don't know if you've ever stripped out a=
 core framework out of 50,000+ lines of code, but, it's painful! Right abou=
t the time Java =0A1.5.0 came out, my project was totally free of Struts. I=
 then went to work on implementing generics throughout. The last public rel=
ease of the project was in September 2004 ( =0A=0Ahttp://sourceforge.net/pr=
ojects/nectar/ ); it was called Nectar Web Services and it's a very outdate=
d version. I prefer to call it Nectar Web Platform today. It's very unlikel=
y there will be a new release in the year to come.=0A=0A=0AMy overall goal =
with Nectar is for it to become a very modular system for a cluster of serv=
ers... Load balancers, worker servers, database nodes, monitoring servers, =
a main controller, the works. The application domain would be mainly anythi=
ng a web browser can display, plus eventually streaming content (audio / vi=
deo), content syndication (SOAP style), while keeping the architecture open=
 enough to deal with pretty much anything that comes along.=0A=0A=0AMy curr=
ent main goal in the project are to get rid of more dependencies. The Servl=
et / JSP framework is nice and all, but it often feels like a bad implement=
ation of PHP. MySQL is solid, but for the specific data structures I'm usin=
g, it's just overkill, leads to extra latency, and will be plain counterpro=
ductive in a large cluster environment where a fully distributed database s=
tarts to show it's potential. Tomcat also provides tons of features and gim=
micks, none of which I'm using. Stack traces of exceptions thrown by my cod=
e usually show a list of 15 functions in Tomcat. =0A=0A=0AMy cluster design=
 will hopefully be using one big Layer 4 switch at the very front, handing =
out (round robin style) requests to Simple on several load balancers. The l=
oad balancers would be the only servers running Simple, just to decode the =
HTTP request and pass the information on to Object*putStreams connected to =
the worker servers, which in turn will do all the internal stuff to render =
the response, and pass it back to the load balancers in another IOStream. I=
'm not sure about the details, I might use IPv6 connections internally to a=
llow for multiple channels per connection between load balancer and worker =
node, and then figure out some way to manage multiple requests asynchronous=
ly on the load balancers... =0A=0A=0ALike many Open Source projects, I've g=
ot no funding, no specific requirements or strict goal. It's just my hobby =
which I'm fascinated by, and I've rarely written any code without a beer to=
 keep my company! Of course I'll never have the funds to fully test a clust=
er setup, but maybe I can get something going with enough Virtual PC instan=
ces :)=0A=0A=0AI like being in control of things: SQL on the one hand alrea=
dy pretty much determines how you have to think about queries. Tomcat on th=
e other hand provides dozens of strict and complex tools, which grasp deep =
into your own code. With Simple I basically just have one function to imple=
ment. My code initializes Simple, while Tomcat initializes my code. This al=
one is a huge paradigm shift that makes me feel a lot more comfortable. =0A=
=0A=0AI took one long evening to read through some Simple's code, before I =
chose to integrate it. I found it quite fascinating how... well... simple..=
. it was. It shows you've thought well about what your were going to write =
before you wrote it. Goodness, I wish I had your documentation discipline t=
oo!=0A=0A=0A________________=0A=0AThanks very much for your work, Niall and=
 thanks to all those who helped you. I hope I can be of some use to your pr=
oject once I'm more familiar with it.=0A=0AKind Regards,=0A=0A=0A-Kai=0A=0A=
=0A=0A=0A=0A=0A=0A=0A=0A=0A      __________________________________________=
__________________________________________=0ALuggage? GPS? Comic books? =0A=
Check out fitting gifts for grads at Yahoo! Search=0Ahttp://search.yahoo.co=
m/search?fr=3Doni_on_mail&p=3Dgraduation+gifts&cs=3Dbz