[root @ localhost local] # bin/nutch crawl urls - the depth of 3-100
the threadsInjectorJob: Using class org. Apache. Gora. Cassandra. Store. CassandraStore as the gora storage class.
InjectorJob: total number of urls rejected by filters: 1
InjectorJob: total number of urls injected after normalization and filtering: 0
FetcherJob: threads: 100
FetcherJob: parsing: false
FetcherJob: resuming: false
FetcherJob: timelimit set for: 1
Using the queue mode: byHost
The Fetcher: threads: 100
QueueFeeder finished: to total 0 records. Hit by time limit: 0
- finishing thread FetcherThread0, activeThreads=0
.
- finishing thread FetcherThread94, activeThreads=0
The Fetcher: throughput threshold: 1
0/0 spinwaiting/active, 0 pages, 0 errors, 0.0 0 pages/s, 0 0 KB/s, 0 0 URLs in the queues
- activeThreads=0
ParserJob: resuming: false
ParserJob: forced reparse: false
ParserJob: parsing all
FetcherJob: threads: 100
FetcherJob: parsing: false
FetcherJob: resuming: false
FetcherJob: timelimit set for: 1
Using the queue mode: byHost
The Fetcher: threads: 100
.
- activeThreads=0
ParserJob: resuming: false
ParserJob: forced reparse: false
ParserJob: parsing all

CodePudding user response:

I also encountered this problem, and can only catch a page, do you solve

CodePudding user response:

Some of them are can't catch, catch some more about something always, so some can't catch it across the...