Thread: [Cppcms-users] HTTP response and binary stream (libharu)
Brought to you by:
artyom-beilis
Menu
▾
▴
cppcms-users
|
From: augustin <aug...@ov...> - 2015-09-17 10:09:30
|
Hello,
I am asking here about returning a stream of data representing a PDF document.
I could ask the same question about returning a dynamically generated image,
using a third party library but returning the image with a cppcms application.
I am trying to get my cppcms application to output a PDF file (using libharu).
However, I seem to have a problem returning the binary output.
It seems that the cppcms internals are corrupting the stream.
I have tested separately libharu and it works Ok.
When I integrate the code into my cppcms application, I get a corrupted file.
I am not sure which way exist to return a
// Raw mode, returning simple HTTP headers:
app()->response().io_mode(cppcms::http::response::io_mode_type::raw);
// content type: PDF.
app()->response().out() << "Content-type: application/pdf\r\n";
app()->response().out() << "\r\n";
HPDF_Doc pdf // Create new PDF object.
HPDF_SaveToStream (pdf);
std::ostringstream cppcms_buf(std::ios::binary);
cppcms_buf.imbue(app()->context().locale());
for (;;) {
HPDF_BYTE buf[4096];
HPDF_UINT32 siz = 4096;
HPDF_ReadFromStream (pdf, buf, &siz);
//app()->response().out() << buf << std::flush;
if (siz == 0)
break;
cppcms_buf << buf;
}
// Getting a cppcms::http::response:
app()->response().out() << cppcms_buf.str();
I don't know if Artyom is still maintaining this project. I hope that either
he or someone else can tell me whether it's possible.
thanks,
augustin.
|
|
From: Joerg S. <jo...@br...> - 2015-09-17 11:20:35
|
On Thu, Sep 17, 2015 at 05:52:03PM +0800, augustin wrote:
> I am trying to get my cppcms application to output a PDF file (using libharu).
> However, I seem to have a problem returning the binary output.
> It seems that the cppcms internals are corrupting the stream.
Works fine for me, but I don't set raw mode or anything like that, just:
response().content_type("application/pdf");
response().out() << pdf;
where pdf is a std::string.
Joerg
|
|
From: augustin <aug...@ov...> - 2015-09-17 13:17:02
|
On Thursday, September 17, 2015 07:07:21 PM Joerg Sonnenberger wrote:
> On Thu, Sep 17, 2015 at 05:52:03PM +0800, augustin wrote:
> > I am trying to get my cppcms application to output a PDF file (using
> > libharu). However, I seem to have a problem returning the binary output.
> > It seems that the cppcms internals are corrupting the stream.
>
> Works fine for me, but I don't set raw mode or anything like that, just:
>
> response().content_type("application/pdf");
> response().out() << pdf;
>
> where pdf is a std::string.
Thanks, but pdf is not a std::string.
It is a HPDF_Doc from the libharu library, which is used to create documents
following the PDF standard.
A PDF file may include binary streams.
I have no problem outputting HTML (std::string).
I have no problem outputting simple PDF files which only include a stream of
strings.
But when I try creating more complex PDF files with drawings or pictures (i.e.
PDF files including binary streams), the file gets corrupted and is not
rendered properly.
Right now, I am learning about the complexities of having binary streams in
C++. There are quite a few articles on the topic on the internet.
My main question is:
is it a problem in my own code, or is it a shortcoming of the cppcms library,
that it cannot handle binary streams? Does cppcms need to be patched or should
I handle the stream differently on my end?
thanks,
augustin.
|
|
From: Artyom B. <art...@ya...> - 2015-09-18 19:22:24
|
Several points:
(a) You misunderstand meaning of "raw" output - "raw" means you are sending CGI output including headers etc on your own and CppCMS does not interfere with it. There only very-very specific cases you may actually need this - bottom line you don't need it.(b) std::ostream is just a character or more correctly octet stream, std::string is a container that keeps bytes/"C chars"/octets in it in string friendly way. You can put any binary data to it.
Now in your case the problem is there"
app()->response().out() << buf << std::flush; or
cppcms_buf << buf
Why? std::ostream assumes that buf is NUL terminated string so it would stop on NUL, and this what corrupts the stream.You need to use std::ostream::write, i.e.
out().write(buf,siz)
That's it
Artyom Beilis
From: augustin <aug...@ov...>
To: cpp...@li...
Sent: Thursday, September 17, 2015 4:16 PM
Subject: Re: [Cppcms-users] HTTP response and binary stream (libharu)
On Thursday, September 17, 2015 07:07:21 PM Joerg Sonnenberger wrote:
> On Thu, Sep 17, 2015 at 05:52:03PM +0800, augustin wrote:
> > I am trying to get my cppcms application to output a PDF file (using
> > libharu). However, I seem to have a problem returning the binary output.
> > It seems that the cppcms internals are corrupting the stream.
>
> Works fine for me, but I don't set raw mode or anything like that, just:
>
> response().content_type("application/pdf");
> response().out() << pdf;
>
> where pdf is a std::string.
Thanks, but pdf is not a std::string.
It is a HPDF_Doc from the libharu library, which is used to create documents
following the PDF standard.
A PDF file may include binary streams.
I have no problem outputting HTML (std::string).
I have no problem outputting simple PDF files which only include a stream of
strings.
But when I try creating more complex PDF files with drawings or pictures (i.e.
PDF files including binary streams), the file gets corrupted and is not
rendered properly.
Right now, I am learning about the complexities of having binary streams in
C++. There are quite a few articles on the topic on the internet.
My main question is:
is it a problem in my own code, or is it a shortcoming of the cppcms library,
that it cannot handle binary streams? Does cppcms need to be patched or should
I handle the stream differently on my end?
thanks,
augustin.
------------------------------------------------------------------------------
Monitor Your Dynamic Infrastructure at Any Scale With Datadog!
Get real-time metrics from all of your servers, apps and tools
in one place.
SourceForge users - Click here to start your Free Trial of Datadog now!
http://pubads.g.doubleclick.net/gampad/clk?id=241902991&iu=/4140
_______________________________________________
Cppcms-users mailing list
Cpp...@li...
https://lists.sourceforge.net/lists/listinfo/cppcms-users
|
|
From: Artyom B. <art...@ya...> - 2015-09-18 19:27:26
|
As BTW is shown in this example: http://libharu.sourceforge.net/how_to_use.html#Save_a_document_to_file_ Why? std::ostream assumes that buf is NUL terminated string so it would stop on NUL, and this what corrupts the stream. You need to use std::ostream::write, i.e. out().write(buf,siz) |
|
From: Joerg S. <jo...@br...> - 2015-09-20 17:12:34
|
On Fri, Sep 18, 2015 at 07:22:16PM +0000, Artyom Beilis wrote: > Several points: > (a) You misunderstand meaning of "raw" output - "raw" means you are sending CGI output including headers etc on your own and CppCMS does not interfere with it. There only very-very specific cases you may actually need this - bottom line you don't need it.(b) std::ostream is just a character or more correctly octet stream, std::string is a container that keeps bytes/"C chars"/octets in it in string friendly way. You can put any binary data to it. > Now in your case the problem is there" > app()->response().out() << buf << std::flush; or > cppcms_buf << buf > > Why? std::ostream assumes that buf is NUL terminated string so it would stop on NUL, and this what corrupts the stream.You need to use std::ostream::write, i.e. > out().write(buf,siz) Depends on the type of buf. If it is a std::string, it should work fine? That said, I have no idea what overloading the other library provides. Joerg |
|
From: augustin <aug...@ov...> - 2015-09-22 04:19:24
|
On Saturday, September 19, 2015 03:22:16 AM Artyom Beilis wrote:
> Several points:
> (a) You misunderstand meaning of "raw" output - "raw" means you are sending
> CGI output including headers etc on your own and CppCMS does not interfere
> with it. There only very-very specific cases you may actually need this -
> bottom line you don't need it.
Ok.
I had been struggling with my problem for a few days before I asked, and I was
trying to eliminate all possible sources of corruption. I had come to the
conclusion that this probably was not it.
Thanks for confirming it.
> (b) std::ostream is just a character or more
> correctly octet stream, std::string is a container that keeps bytes/"C
> chars"/octets in it in string friendly way. You can put any binary data to
> it.
> Now in your case the problem is there
app()->response().out() << buf << std::flush;
> or
cppcms_buf << buf
>
> Why? std::ostream assumes that buf is NUL terminated string so it would
> stop on NUL, and this what corrupts the stream.You need to use
> std::ostream::write, i.e. out().write(buf,siz)
> That's it
>
> Artyom Beilis
Thank you Artyom!
You clearly saw the source of the problem. Joerg also gave me a hint earlier
which pointed me in the right direction.
I had searched online and found other newbies like me who made the same wrong
assumption about the << operator, which, I understand now, should not be used
for binary streams.
After reading your mail, I still struggled for a while with type casting.
I had compile errors such as:
error: invalid conversion from 'HPDF_BYTE*' to 'const char*'
or:
error: invalid static_cast from type 'HPDF_BYTE [4096]' to type 'const char*'
Finally, I was forced to use a reinterpret_cast, but apparently it cannot be
avoided:
// #include <hpdf.h>
typedef unsigned char HPDF_BYTE;
// my_pdf.cpp:
// Create PDF.
HPDF_Doc pdf = HPDF_New(pdf::error_handler, NULL);
....
.... // Add content to pdf.
....
HPDF_SaveToStream (pdf);
for (;;) {
HPDF_BYTE buf[4096];
HPDF_UINT32 siz = 4096;
HPDF_ReadFromStream (pdf, buf, &siz);
// cppcms::http::response:
app()->response().out().write(reinterpret_cast<const char*>(buf), siz);
if (siz == 0)
break;
}
app()->response().out() << std::flush;
HPDF_Free(pdf);
I am not sure if the << std::flush at the end is necessary, but it doesn't
seem to hurt.
Artyom, would you like me to create a simple tutorial on using cppcms +
libharu on the cppcms wiki?
Do you think it may be useful to other people?
I can do it if you wish.
On Saturday, September 19, 2015 03:26:34 AM Artyom Beilis wrote:
> As BTW is shown in this
> example:
http://libharu.sourceforge.net/how_to_use.html#Save_a_document_to_file_
Yes, I had already seen that page.
Actually, the project has moved to github, so I was following the
corresponding page there:
https://github.com/libharu/libharu/wiki/Usage-examples
My main problem is lack of experience (I don't code full time) and I am still
baffled by some intricacies of the language and the standard libraries.
I often fail to see the relevance of what I am reading to my problem.
Once I have solved a problem, it is often frustrating to see that I had
actually come across the solution days earlier but failed to understand it!
Believe me, I don't take free online help for granted. I always ask as a last
resort, when I feel really stuck after having spent a few days trying to fully
understand the problem and researching online.
As always, I am grateful for you help, and I am in awe of your mastery of the
language! :)
Blessings,
augustin.
|
|
From: Joerg S. <jo...@br...> - 2015-09-17 13:46:51
|
On Thu, Sep 17, 2015 at 09:16:42PM +0800, augustin wrote: > I have no problem outputting HTML (std::string). > I have no problem outputting simple PDF files which only include a stream of > strings. > But when I try creating more complex PDF files with drawings or pictures (i.e. > PDF files including binary streams), the file gets corrupted and is not > rendered properly. Try creating a std::stringstream with std::ios::binary and write to that from your library. Use the stringstream -> string accessor and write that. That's what I am doing when creating PDFs from PoDoFo. Whether the default streams should be created as binary is a slightly different question. Joerg |
|
From: Marcel H. <ke...@co...> - 2015-09-17 16:15:05
Attachments:
signature.asc
|
On 17.09.2015 15:16, augustin wrote: > Thanks, but pdf is not a std::string. you could and/or should (depending on what are you trying to do) return a base64 encoded string. |
|
From: augustin <aug...@ov...> - 2015-09-18 08:44:05
|
On Thursday, September 17, 2015 11:58:48 PM Marcel Hellwig wrote: > you could and/or should (depending on what are you trying to do) return > a base64 encoded string. Thanks Joerg and Marcel, you open new avenues of investigation for me. I've just been researching the base64 solution, and I fail to see where to employ it and how it would solve the problem. First, the HTTP protocol does not use base64 encoding. It supports binary streams natively (how else would pictures and what not be transferred?). As far as I am aware, Base64 was/is mostly used for mail transport. Secondly, unless I missed it, the PDF protocol does not support base64 encoded PDF files. I cannot find any information about this. What am I trying to do? Create within my cppcms application a PDF document on the fly (dynamically), and return a proper .pdf file to the user via cppcms and the HTTP protocol. As far as I am aware, the problem is either within my cppcms application or within cppcms itself. My experience is limited, and I may be missing something, but I fail to see at what stage to use Base64 strings and how it would help. I'm still looking and testing... Cheers, augustin. . |
|
From: augustin <aug...@ov...> - 2015-09-18 08:53:40
|
On Thursday, September 17, 2015 09:46:41 PM Joerg Sonnenberger wrote: > On Thu, Sep 17, 2015 at 09:16:42PM +0800, augustin wrote: > > I have no problem outputting HTML (std::string). > > I have no problem outputting simple PDF files which only include a stream > > of strings. > > But when I try creating more complex PDF files with drawings or pictures > > (i.e. PDF files including binary streams), the file gets corrupted and > > is not rendered properly. > > Try creating a std::stringstream with std::ios::binary and write to that > from your library. That's what I'm already doing. From the code I posted at the beginning: std::ostringstream cppcms_buf(std::ios::binary); ... // cppcms::http::response: app()->response().out() << cppcms_buf.str(); > Use the stringstream -> string accessor and write > that. That's what I am doing when creating PDFs from PoDoFo. Do your pdf contain only plain text, or do they contain graphics? Can you post a simple hello_world.pdf code, especially the part where the generated PDF is sent back via the cppcms output stream: cppcms::http::response().out() ? I hesitated between using libharu and PoDoFo. Neither library is well documented. I just started with libharu but it's not too late to switch to another library. Are you happy with PoDoFo? Thank you, Joerg. augustin. . |
|
From: Joerg S. <jo...@br...> - 2015-09-18 09:28:52
|
On Fri, Sep 18, 2015 at 04:53:24PM +0800, augustin wrote: > Do your pdf contain only plain text, or do they contain graphics? Embedded fonts, so it would definitely and visibly fail. > Can you post a simple hello_world.pdf code, especially the part where the > generated PDF is sent back via the cppcms output stream: > cppcms::http::response().out() > ? On Sunday, if I remember. > I hesitated between using libharu and PoDoFo. > Neither library is well documented. > I just started with libharu but it's not too late to switch to another > library. > Are you happy with PoDoFo? It could do everything I wanted so far, even though some parts were a bit tricky to get work between pdfJS limitations, the sometimes incomprehensible PDF reference and figuring out the mechanics of PoDoFo. But I don't think a different library would have made things much easier. Joerg |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X