|
From: Rexque F. <rex...@gm...> - 2011-03-03 07:35:47
|
I have a HDFS cluster, I'm pretty sure it works, I have:
app1:
TaskTracker
DataNode
JobTracker
NameNode
app2:
TaskTracker
DataNode
SecondaryNameNode
on both nodes, I get the same result ( emtpy, no errors) if I run, so im
pretty sure HDFS is working:
$hadoop fs -ls /scribed/EVENTSIGNUPSCRIBE
$
here is part of my very plain, vanilla scribe.conf, running on app1:
<primary>
type=file
fs_type=hdfs
file_path=hdfs://app1:8020/scribed
create_symlink=no
use_hostname_sub_directory=no
base_filename=thisisoverwritten
max_size=40000000
rotate_period=daily
rotate_hour=0
rotate_minute=5
add_newlines=1
</primary>
in hadoop in core-site.xml I have this defined:
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://app1:8020</value>
</property>
just for fun, i can telnet to app1 8020 from app1
I made a php script that should write to this, here are the errors I see in
scribe's log:
Wed Mar 2 23:20:59 2011] "STATUS: configuring"
[Wed Mar 2 23:20:59 2011] "got configuration data from file
</home/scribed/conf/scribe.conf>"
[Wed Mar 2 23:20:59 2011] "CATEGORY : default"
[Wed Mar 2 23:20:59 2011] "[default] choosing new retry interval <318>
seconds"
[Wed Mar 2 23:20:59 2011] "Creating default store"
[Wed Mar 2 23:20:59 2011] "configured <1> stores"
[Wed Mar 2 23:20:59 2011] "STATUS: "
[Wed Mar 2 23:20:59 2011] "STATUS: ALIVE"
[Wed Mar 2 23:20:59 2011] "Starting scribe server on port 1463"
Thrift: Wed Mar 2 23:20:59 2011 libevent 1.4.13-stable method epoll
[Wed Mar 2 23:21:34 2011] "[EVENTSIGNUPSCRIBE] Creating new category from
model default"
[Wed Mar 2 23:21:34 2011] "[EVENTSIGNUPSCRIBE] choosing new retry interval
<302> seconds"
[Wed Mar 2 23:21:34 2011] "store thread starting"
[Wed Mar 2 23:21:34 2011] "[EVENTSIGNUPSCRIBE] Failed to create file
<hdfs://app1:8020/scribed/EVENTSIGNUPSCRIBE/EVENTSIGNUPSCRIBE-2011-03-02_00000>
of type <hdfs> for writing"
[Wed Mar 2 23:21:34 2011] "[EVENTSIGNUPSCRIBE] Opened file
</home/scribed/data/scribetest/EVENTSIGNUPSCRIBE/EVENTSIGNUPSCRIBE_00000>
for writing"
[Wed Mar 2 23:21:34 2011] "[EVENTSIGNUPSCRIBE] choosing new retry interval
<29> seconds"
[Wed Mar 2 23:21:34 2011] "[EVENTSIGNUPSCRIBE] Changing state from
<DISCONNECTED> to <DISCONNECTED>"
[Wed Mar 2 23:22:06 2011] "[EVENTSIGNUPSCRIBE] Failed to create file
<hdfs://app1:8020/scribed/EVENTSIGNUPSCRIBE/EVENTSIGNUPSCRIBE-2011-03-02_00000>
of type <hdfs> for writing"
[Wed Mar 2 23:22:06 2011] "[EVENTSIGNUPSCRIBE] choosing new retry interval
<34> seconds"
[Wed Mar 2 23:22:06 2011] "[EVENTSIGNUPSCRIBE] Changing state from
<DISCONNECTED> to <DISCONNECTED>"
[Wed Mar 2 23:22:42 2011] "[EVENTSIGNUPSCRIBE] Failed to create file
<hdfs://app1:8020/scribed/EVENTSIGNUPSCRIBE/EVENTSIGNUPSCRIBE-2011-03-02_00000>
of type <hdfs> for writing"
any idea what I am doing wrong, or what I should be looking for?
thanks
|