#5 PANIC! Task net.javacoding.jspider.core.task.work.FetchRobot

[Anonymous]

INFO [core.throttle.ThrottleFactory] Throttle provider
class is 'class
net.javacoding.jspider.core.throttle.impl.Distribute
dLoadThrottleProvider'
INFO [core.throttle.ThrottleFactory] Throttle provider
class is 'class
net.javacoding.jspider.core.throttle.impl.Distribute
dLoadThrottleProvider'
INFO [core.throttle.ThrottleFactory] Throttle provider
class is 'class
net.javacoding.jspider.core.throttle.impl.Distribute
dLoadThrottleProvider'
INFO [core.throttle.ThrottleFactory] Throttle provider
class is 'class
net.javacoding.jspider.core.throttle.impl.Distribute
dLoadThrottleProvider'
INFO [core.throttle.ThrottleFactory] Throttle provider
class is 'class
net.javacoding.jspider.core.throttle.impl.Distribute
dLoadThrottleProvider'
[Plugin] Job monitor: 81% (8428/10386) [S:37%
(1198/3156) | T:100% (7230/7230)] [blocked:782]
[assigned:6]
[Plugin] ThreadPool Spiders occupation:100% [idle: 0%,
blocked: 0%, busy: 100%], size: 5
[Plugin] ThreadPool Thinkers occupation:0% [idle: 100%,
blocked: 0%, busy: 0%], size: 1
[Plugin] Job monitor: 81% (8428/10386) [S:37%
(1198/3156) | T:100% (7230/7230)] [blocked:782]
[assigned:6]
[Plugin] ThreadPool Thinkers occupation:0% [idle: 100%,
blocked: 0%, busy: 0%], size: 1
[Plugin] ThreadPool Spiders occupation:100% [idle: 0%,
blocked: 0%, busy: 100%], size: 5
[Plugin] Job monitor: 81% (8428/10386) [S:37%
(1198/3156) | T:100% (7230/7230)] [blocked:782]
[assigned:6]
[Plugin] ThreadPool Thinkers occupation:0% [idle: 100%,
blocked: 0%, busy: 0%], size: 1
[Plugin] ThreadPool Spiders occupation:100% [idle: 0%,
blocked: 0%, busy: 100%], size: 5
INFO [core.throttle.ThrottleFactory] Throttle provider
class is 'class
net.javacoding.jspider.core.throttle.impl.Distribute
dLoadThrottleProvider'
INFO [core.throttle.ThrottleFactory] Throttle provider
class is 'class
net.javacoding.jspider.core.throttle.impl.Distribute
dLoadThrottleProvider'
INFO [core.throttle.ThrottleFactory] Throttle provider
class is 'class
net.javacoding.jspider.core.throttle.impl.Distribute
dLoadThrottleProvider'
FATAL [core.threading.WorkerThread] PANIC! Task
net.javacoding.jspider.core.task.work.FetchRobotsTXTTa
skImpl@afd9e3 threw an
excpetion!
java.lang.StringIndexOutOfBoundsException: String index
out of range: -1
at java.lang.String.substring(String.java:1480)
at java.lang.String.substring(String.java:1447)
at
net.javacoding.jspider.core.util.html.RobotsTXTLineSet.fin
dUserAgent(Unknown Source)
at
net.javacoding.jspider.core.util.html.RobotsTXTLineSet.fin
dLineSet(Unknown Source)
at
net.javacoding.jspider.core.util.html.RobotsTXTLineSet.fin
dLineSet(Unknown Source)
at
net.javacoding.jspider.core.rule.impl.RobotsTXTRule.<init
>(Unknown Source)
at
net.javacoding.jspider.core.impl.SpiderContextImpl.registe
rRobotsTXT(Unknown Source)
at net.javacoding.jspider.core.impl.AgentImpl.visit
(Unknown Source)
at
net.javacoding.jspider.core.event.impl.RobotsTXTSpidere
dOkEvent.accept(Unknown Source)
at
net.javacoding.jspider.core.impl.AgentImpl.registerEvent
(Unknown Source)
at
net.javacoding.jspider.core.task.work.BaseWorkerTaskIm
pl.notifyEvent(Unknown Source)
at
net.javacoding.jspider.core.task.work.FetchRobotsTXTTa
skImpl.execute(Unknown Source)
at
net.javacoding.jspider.core.threading.WorkerThread.run
(Unknown Source)

[EJP]

Fixed in upcoming 1.0 release.