2026-09-11 05:43:04 [scrapy.utils.log] INFO: Scrapy 2.17.0 started (bot: priceWatchBot)
2026-09-11 05:43:04 [scrapy.utils.log] INFO: Versions:
{'lxml': '6.1.1',
 'libxml2': '2.14.6',
 'cssselect': '1.4.0',
 'parsel': '1.11.0',
 'w3lib': '2.4.1',
 'Twisted': '26.4.0',
 'Python': '3.12.13 (main, Jul 14 2026, 02:09:00) [GCC 14.2.0]',
 'pyOpenSSL': '26.3.0 (OpenSSL 4.0.1 9 Jun 2026)',
 'cryptography': '49.0.0',
 'Platform': 'Linux-6.1.0-47-cloud-amd64-x86_64-with-glibc2.41'}
2026-09-11 05:43:04 [priceWatchBotSpider] INFO: ID de l'URL : 266
2026-09-11 05:43:04 [scrapy.addons] INFO: Enabled addons:
[]
2026-09-11 05:43:04 [scrapy.extensions.telnet] INFO: Telnet Password: 503b377888752b79
2026-09-11 05:43:04 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
 'scrapy.extensions.logcount.LogCount',
 'scrapy.extensions.telnet.TelnetConsole',
 'scrapy.extensions.memusage.MemoryUsage',
 'scrapy.extensions.closespider.CloseSpider',
 'scrapy.extensions.logstats.LogStats',
 'scrapy.extensions.throttle.AutoThrottle']
2026-09-11 05:43:04 [scrapy.crawler] INFO: Overridden settings:
{'AUTOTHROTTLE_ENABLED': True,
 'AUTOTHROTTLE_MAX_DELAY': 30,
 'AUTOTHROTTLE_START_DELAY': 1,
 'AUTOTHROTTLE_TARGET_CONCURRENCY': 4.0,
 'BOT_NAME': 'priceWatchBot',
 'CLOSESPIDER_PAGECOUNT': 5000,
 'CONCURRENT_REQUESTS': 32,
 'DEPTH_LIMIT': 10,
 'DOWNLOAD_DELAY': 1,
 'DOWNLOAD_TIMEOUT': 30,
 'FEED_EXPORT_ENCODING': 'utf-8',
 'LOG_FILE': '/logs/priceWatchBot/priceWatchBotSpider/a7ac036eada311f18c5a0242ac120004.log',
 'LOG_LEVEL': 'INFO',
 'NEWSPIDER_MODULE': 'priceWatchBot.spiders',
 'RETRY_HTTP_CODES': [403, 429, 500, 502, 503, 504, 408, 522, 524],
 'SPIDER_MODULES': ['priceWatchBot.spiders'],
 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 '
               '(KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36'}
2026-09-11 05:43:05 [priceWatchBot.middlewares] INFO: ImpersonateMiddleware actif (profil=chrome131).
2026-09-11 05:43:05 [scrapy.middleware] WARNING: Disabled priceWatchBot.middlewares.RotatingProxyMiddleware: Aucun proxy configuré (PROXY_LIST vide).
2026-09-11 05:43:05 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.offsite.OffsiteMiddleware',
 'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
 'priceWatchBot.middlewares.ImpersonateMiddleware',
 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
 'scrapy.downloadermiddlewares.stats.DownloaderStats']
2026-09-11 05:43:05 [py.warnings] WARNING: /usr/local/lib/python3.12/site-packages/scrapy/core/downloader/middleware.py:46: ScrapyDeprecationWarning: ImpersonateMiddleware.process_request() requires a spider argument, this is deprecated and the argument will not be passed in future Scrapy versions. If you need to access the spider instance you can save the crawler instance passed to from_crawler() and use its spider attribute.
  self._check_mw_method_spider_arg(mw.process_request)

2026-09-11 05:43:05 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.start.StartSpiderMiddleware',
 'scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
 'scrapy.spidermiddlewares.referer.RefererMiddleware',
 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
 'scrapy.spidermiddlewares.depth.DepthMiddleware']
2026-09-11 05:43:05 [scrapy.middleware] INFO: Enabled item pipelines:
['priceWatchBot.pipelines.PricewatchbotPipeline']
2026-09-11 05:43:05 [scrapy.core.engine] INFO: Spider opened
2026-09-11 05:43:05 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2026-09-11 05:43:05 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6025
2026-09-11 05:43:07 [scrapy.downloadermiddlewares.retry] ERROR: Gave up retrying <GET https://www.piscineshop.com/504-aquabike-water-rider-5.html> (failed 3 times): 403 Forbidden
2026-09-11 05:43:07 [scrapy.spidermiddlewares.httperror] INFO: Ignoring response <403 https://www.piscineshop.com/504-aquabike-water-rider-5.html>: HTTP status code is not handled or not allowed
2026-09-11 05:43:07 [scrapy.core.engine] INFO: Closing spider (finished)
2026-09-11 05:43:07 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 1875,
 'downloader/request_count': 3,
 'downloader/request_method_count/GET': 3,
 'downloader/response_bytes': 1206,
 'downloader/response_count': 3,
 'downloader/response_status_count/403': 3,
 'elapsed_time_seconds': 2.319343652576208,
 'finish_reason': 'finished',
 'finish_time': datetime.datetime(2026, 9, 11, 5, 43, 7, 592048, tzinfo=datetime.timezone.utc),
 'httperror/response_ignored_count': 1,
 'httperror/response_ignored_status_count/403': 1,
 'items_per_minute': 0.0,
 'log_count/ERROR': 1,
 'log_count/INFO': 4,
 'memusage/max': 85176320,
 'memusage/startup': 85176320,
 'response_received_count': 1,
 'responses_per_minute': 30.0,
 'retry/count': 2,
 'retry/max_reached': 1,
 'retry/reason_count/403 Forbidden': 2,
 'scheduler/dequeued': 3,
 'scheduler/dequeued/memory': 3,
 'scheduler/enqueued': 3,
 'scheduler/enqueued/memory': 3,
 'start_time': datetime.datetime(2026, 9, 11, 5, 43, 5, 272695, tzinfo=datetime.timezone.utc)}
2026-09-11 05:43:07 [scrapy.core.engine] INFO: Spider closed (finished)