Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo
How does this work internally? Does scribe append the data into HDFS as it comes in or does add the file to HDFS once the file gets rotated?
Also, do you know if this will work with the S3 backend for HDFS?
Scribe writes to hdfs directly using libhdfs (the c interface to hdfs). It does not wait for the file to be rotated.
This should work for the S3 backend for HDFS, but we haven't yet tried this out.