-
Notifications
You must be signed in to change notification settings - Fork 547
Open
Labels
t-toolingIssues with this label are in the ownership of the tooling team.Issues with this label are in the ownership of the tooling team.
Description
Logs in crawlee 1.0.0
[BeautifulSoupCrawler] INFO Crawled 0/1 pages, 0 failed requests, desired concurrency 10.
[BeautifulSoupCrawler] INFO Current request statistics:
┌───────────────────────────────┬────────┐
│ requests_finished │ 0 │
│ requests_failed │ 0 │
│ retry_histogram │ [0] │
│ request_avg_failed_duration │ None │
│ request_avg_finished_duration │ None │
│ requests_finished_per_minute │ 0 │
│ requests_failed_per_minute │ 0 │
│ request_total_duration │ 0s │
│ requests_total │ 0 │
│ crawler_runtime │ 42.7ms │
└───────────────────────────────┴────────┘
[crawlee._autoscaling.autoscaled_pool] INFO current_concurrency = 0; desired_concurrency = 10; cpu = 0; mem = 0; event_loop = 0.0; client_info = 0.0
[BeautifulSoupCrawler] INFO Crawled 22/70 pages, 0 failed requests, desired concurrency 10.
[BeautifulSoupCrawler] INFO Crawled 61/70 pages, 0 failed requests, desired concurrency 11.
[crawlee._autoscaling.autoscaled_pool] INFO Waiting for remaining tasks to finish
[BeautifulSoupCrawler] INFO Final request statistics:
┌───────────────────────────────┬────────────┐
│ requests_finished │ 70 │
│ requests_failed │ 0 │
│ retry_histogram │ [70] │
│ request_avg_failed_duration │ None │
│ request_avg_finished_duration │ 2.82s │
│ requests_finished_per_minute │ 196 │
│ requests_failed_per_minute │ 0 │
│ request_total_duration │ 3min 17.4s │
│ requests_total │ 70 │
│ crawler_runtime │ 21.38s │
└───────────────────────────────┴────────────┘Same code in crawlee 1.2.0 and 1.3.0
[BeautifulSoupCrawler] INFO Current request statistics:
┌───────────────────────────────┬──────┐
│ requests_finished │ 0 │
│ requests_failed │ 0 │
│ retry_histogram │ [0] │
│ request_avg_failed_duration │ None │
│ request_avg_finished_duration │ None │
│ requests_finished_per_minute │ 0 │
│ requests_failed_per_minute │ 0 │
│ request_total_duration │ 0s │
│ requests_total │ 0 │
│ crawler_runtime │ 0s │
└───────────────────────────────┴──────┘
[BeautifulSoupCrawler] INFO Crawled 0/318 pages, 0 failed requests, desired concurrency 10.
[crawlee._autoscaling.autoscaled_pool] INFO current_concurrency = 0; desired_concurrency = 10; cpu = 0; mem = 0; event_loop = 0.0; client_info = 0.0
[BeautifulSoupCrawler] INFO Crawled 25/387 pages, 0 failed requests, desired concurrency 10.
[BeautifulSoupCrawler] INFO Crawled 55/387 pages, 0 failed requests, desired concurrency 11.
[crawlee._autoscaling.autoscaled_pool] INFO Waiting for remaining tasks to finish
[BeautifulSoupCrawler] INFO Final request statistics:
┌───────────────────────────────┬────────────┐
│ requests_finished │ 70 │
│ requests_failed │ 0 │
│ retry_histogram │ [70] │
│ request_avg_failed_duration │ None │
│ request_avg_finished_duration │ 3.37s │
│ requests_finished_per_minute │ 178 │
│ requests_failed_per_minute │ 0 │
│ request_total_duration │ 3min 55.7s │
│ requests_total │ 70 │
│ crawler_runtime │ 23.55s │
└───────────────────────────────┴────────────┘Metadata
Metadata
Assignees
Labels
t-toolingIssues with this label are in the ownership of the tooling team.Issues with this label are in the ownership of the tooling team.