How does this work internally? Does scribe append the data into HDFS as it comes in or does add the file to HDFS once the file gets rotated?
Also, do you know if this will work with the S3 backend for HDFS?
Scribe writes to hdfs directly using libhdfs (the c interface to hdfs). It does not wait for the file to be rotated.
This should work for the S3 backend for HDFS, but we haven't yet tried this out.
Log in to post a comment.
Sign up for the SourceForge newsletter:
You seem to have CSS turned off.
Please don't fill out this field.