Re: [Cppcms-users] Concurrent download question

[Artyom B. <art...@gm...>]

> The problem is that individual file will have rare downloads(usually
> one file will be downloaded once a month or two)
> and no. of downloads are large. So caching will not help.
> They have to be read asynchronously and concurrently. I am not sure
> that should I create booster::io::service or cppcms::service. Also,
> I find it hard to create booster::io::service as its example is not
> exhaustive. Even if I create booster::io::service the filesystem
> calls may block. I will be using GlusterFS for storage. But that is
> something not within CppCMS's domain so I will take care of
> that.

Got it, sorry I was mislead by previous question regarding the uploads.

Regarding downloads.

1st if you run behind lighttpd or nginx web server (which you probably
should do) I suggest
to use X-Send-File stuff
http://cppcms.com/wikipp/en/page/cppcms_1x_serving_static_files

Also in such case you may need to mount GlusterFS via FUSE or NFS or
use a special plugin
(I've seen one for lighttpd)

They do the best they can in serving static files - but you probably
check the solution explicitly
because maybe lighttpd or nginx are not optimized for GlusterFS as
normal file system.

---------------------------------------------------

If you use CppCMS's internal file server it can work in both sync and async mode
starting from CppCMS 1.2 but you'll need to use stuff like FUSE or NFS
to mount file system.

But in async mode if GlusterFS hangs in opening/reading file you may block
entire event loop.

You can write your own application that does the job.

Also note - if you don't have lots of concurrent downloads it may be
simpler just to use thread per connection - in standard synchronous
application as it would deal with both concurrency and "io optimization"

if you do it asynchronously it can be tricky as if GlusterFS API may block
and it would stall entire event loop. So you need to handle it someway,
i.e. use booster::aio::io_service to handle such and events (I assyme
glusterfs API has some event driven handling)

Also you should note that web server may read the request
faster than it sends it to the client - lighttpd in such a case would
put it into memory and nginx create its own temporary file
so it can be tricky stuff - neither of them would block you from
writing to client (AFAIR) - which can be bad for huge files.

Bottom line several questions you need to answer:

1. Do you work with GlusterFS via FUSE/NFS or its own API?
2. What concurrency level do you expect?
3. What web server does when client does not read in same speed as you send?


Artyom