Cloudfare middleware recursively bypassess protection












1















i have installed cfscrape,and use it in my scrapy project to bypass cloudfare protection.It seems not working.I can add some more information about my code if necessary.I provide some code from settings.py:`



DOWNLOAD_DELAY = 0.25



DEFAULT_REQUEST_HEADERS = {

  'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',

  'Accept-Language': 'en',

}

USER_AGENT_LIST = [

    'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.7 (KHTML, like asdlkjqwdj) Chrome/16.0.912.36 Safari/535.7',

    'Mozilla/5.0 (Windows NT 2.1; WOW64) AppleWebKit/535.7 (KHTML, like kqjwdqwd) Chrome/16.0.912.36 Safari/123123.123',

    'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.7 (KHTML, like asdqwdqw) Chrome/16.0.912.36 Safari/535.7'

]



DOWNLOADER_MIDDLEWARES = {

    # The priority of 560 is important, because we want this middleware to kick in just before the scrapy built-in `RetryMiddleware`.

     'scrapy_cloudflare_middleware.middlewares.CloudFlareMiddleware': 560,

     'binaaz.middlewares.RandomUserAgentMiddleware': 400,

     'binaaz.middlewares.ProxyMiddleware': 410,

     'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware': None

    # Disable compression middleware, so the actual HTML pages are cached

}


and here is output:



2019-01-18 15:02:59 [scrapy.core.engine] INFO: Spider opened

2019-01-18 15:02:59 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)

2019-01-18 15:02:59 [scrapy.extensions.telnet] DEBUG: Telnet console listening on 127.0.0.1:6023

2019-01-18 15:03:00 [cloudflaremiddleware] DEBUG: Cloudflare protection detected on https://bina.az/items/all, trying to bypass...

2019-01-18 15:03:00 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): bina.az

2019-01-18 15:03:00 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 503 None

2019-01-18 15:03:08 [urllib3.connectionpool] DEBUG: Resetting dropped connection: bina.az

2019-01-18 15:03:09 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /cdn-cgi/l/chk_jschl?jschl_vc=6d688f88cb5562fd00a5ee16689e26c9&pass=1547809384.962-1OiFgM1jrn&jschl_answer=18.089701993600002 HTTP/1.1" 302 159

2019-01-18 15:03:09 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 503 None

2019-01-18 15:03:17 [urllib3.connectionpool] DEBUG: Resetting dropped connection: bina.az

2019-01-18 15:03:17 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /cdn-cgi/l/chk_jschl?jschl_vc=e719be7793db058c78b89dc24f6e39b7&pass=1547809393.639-gfrUEjO5oa&jschl_answer=26.1093884035 HTTP/1.1" 302 159

2019-01-18 15:03:17 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 204 0

2019-01-18 15:03:17 [cloudflaremiddleware] DEBUG: Successfully bypassed the protection for https://bina.az/items/all, re-scheduling the request

2019-01-18 15:03:18 [cloudflaremiddleware] DEBUG: Cloudflare protection detected on https://bina.az/items/all, trying to bypass...

2019-01-18 15:03:18 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): bina.az

2019-01-18 15:03:18 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 503 None

2019-01-18 15:03:26 [urllib3.connectionpool] DEBUG: Resetting dropped connection: bina.az

2019-01-18 15:03:27 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /cdn-cgi/l/chk_jschl?jschl_vc=faf6647605a0b45ed9958c055de11334&pass=1547809403.2-RdtkjUMCBv&jschl_answer=35.723157815600004 HTTP/1.1" 302 159

2019-01-18 15:03:27 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 503 None

2019-01-18 15:03:35 [urllib3.connectionpool] DEBUG: Resetting dropped connection: bina.az

2019-01-18 15:03:35 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /cdn-cgi/l/chk_jschl?jschl_vc=8a0665b0b9df3707f15de0299d196b33&pass=1547809411.873-xrZecnfmeE&jschl_answer=-3.7878104051 HTTP/1.1" 302 159

2019-01-18 15:03:36 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 204 0

2019-01-18 15:03:36 [cloudflaremiddleware] DEBUG: Successfully bypassed the protection for https://bina.az/items/all, re-scheduling the request

2019-01-18 15:03:36 [cloudflaremiddleware] DEBUG: Cloudflare protection detected on https://bina.az/items/all, trying to bypass...

2019-01-18 15:03:36 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): bina.az

2019-01-18 15:03:36 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 503 None

2019-01-18 15:03:45 [urllib3.connectionpool] DEBUG: Resetting dropped connection: bina.az

2019-01-18 15:03:45 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /cdn-cgi/l/chk_jschl?jschl_vc=671d8f3bfcb227960607a55f33183748&pass=1547809421.422-h5xJraFbRf&jschl_answer=20.2767620475 HTTP/1.1" 302 159

2019-01-18 15:03:45 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 204 0

2019-01-18 15:03:45 [cloudflaremiddleware] DEBUG: Successfully bypassed the protection for https://bina.az/items/all, re-scheduling the request


P.S: I have updated cfscrape to latest version.






python-3.x scrapy







share|improve this question























  • Have you considered opening a bug report at github.com/Anorov/cloudflare-scrape/issues ?

    – Gallaecio
    2 days ago











  • Okay man, i will

    – Adilzada
    2 days ago
















1















i have installed cfscrape,and use it in my scrapy project to bypass cloudfare protection.It seems not working.I can add some more information about my code if necessary.I provide some code from settings.py:`



DOWNLOAD_DELAY = 0.25



DEFAULT_REQUEST_HEADERS = {

  'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',

  'Accept-Language': 'en',

}

USER_AGENT_LIST = [

    'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.7 (KHTML, like asdlkjqwdj) Chrome/16.0.912.36 Safari/535.7',

    'Mozilla/5.0 (Windows NT 2.1; WOW64) AppleWebKit/535.7 (KHTML, like kqjwdqwd) Chrome/16.0.912.36 Safari/123123.123',

    'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.7 (KHTML, like asdqwdqw) Chrome/16.0.912.36 Safari/535.7'

]



DOWNLOADER_MIDDLEWARES = {

    # The priority of 560 is important, because we want this middleware to kick in just before the scrapy built-in `RetryMiddleware`.

     'scrapy_cloudflare_middleware.middlewares.CloudFlareMiddleware': 560,

     'binaaz.middlewares.RandomUserAgentMiddleware': 400,

     'binaaz.middlewares.ProxyMiddleware': 410,

     'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware': None

    # Disable compression middleware, so the actual HTML pages are cached

}


and here is output:



2019-01-18 15:02:59 [scrapy.core.engine] INFO: Spider opened

2019-01-18 15:02:59 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)

2019-01-18 15:02:59 [scrapy.extensions.telnet] DEBUG: Telnet console listening on 127.0.0.1:6023

2019-01-18 15:03:00 [cloudflaremiddleware] DEBUG: Cloudflare protection detected on https://bina.az/items/all, trying to bypass...

2019-01-18 15:03:00 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): bina.az

2019-01-18 15:03:00 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 503 None

2019-01-18 15:03:08 [urllib3.connectionpool] DEBUG: Resetting dropped connection: bina.az

2019-01-18 15:03:09 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /cdn-cgi/l/chk_jschl?jschl_vc=6d688f88cb5562fd00a5ee16689e26c9&pass=1547809384.962-1OiFgM1jrn&jschl_answer=18.089701993600002 HTTP/1.1" 302 159

2019-01-18 15:03:09 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 503 None

2019-01-18 15:03:17 [urllib3.connectionpool] DEBUG: Resetting dropped connection: bina.az

2019-01-18 15:03:17 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /cdn-cgi/l/chk_jschl?jschl_vc=e719be7793db058c78b89dc24f6e39b7&pass=1547809393.639-gfrUEjO5oa&jschl_answer=26.1093884035 HTTP/1.1" 302 159

2019-01-18 15:03:17 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 204 0

2019-01-18 15:03:17 [cloudflaremiddleware] DEBUG: Successfully bypassed the protection for https://bina.az/items/all, re-scheduling the request

2019-01-18 15:03:18 [cloudflaremiddleware] DEBUG: Cloudflare protection detected on https://bina.az/items/all, trying to bypass...

2019-01-18 15:03:18 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): bina.az

2019-01-18 15:03:18 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 503 None

2019-01-18 15:03:26 [urllib3.connectionpool] DEBUG: Resetting dropped connection: bina.az

2019-01-18 15:03:27 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /cdn-cgi/l/chk_jschl?jschl_vc=faf6647605a0b45ed9958c055de11334&pass=1547809403.2-RdtkjUMCBv&jschl_answer=35.723157815600004 HTTP/1.1" 302 159

2019-01-18 15:03:27 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 503 None

2019-01-18 15:03:35 [urllib3.connectionpool] DEBUG: Resetting dropped connection: bina.az

2019-01-18 15:03:35 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /cdn-cgi/l/chk_jschl?jschl_vc=8a0665b0b9df3707f15de0299d196b33&pass=1547809411.873-xrZecnfmeE&jschl_answer=-3.7878104051 HTTP/1.1" 302 159

2019-01-18 15:03:36 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 204 0

2019-01-18 15:03:36 [cloudflaremiddleware] DEBUG: Successfully bypassed the protection for https://bina.az/items/all, re-scheduling the request

2019-01-18 15:03:36 [cloudflaremiddleware] DEBUG: Cloudflare protection detected on https://bina.az/items/all, trying to bypass...

2019-01-18 15:03:36 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): bina.az

2019-01-18 15:03:36 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 503 None

2019-01-18 15:03:45 [urllib3.connectionpool] DEBUG: Resetting dropped connection: bina.az

2019-01-18 15:03:45 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /cdn-cgi/l/chk_jschl?jschl_vc=671d8f3bfcb227960607a55f33183748&pass=1547809421.422-h5xJraFbRf&jschl_answer=20.2767620475 HTTP/1.1" 302 159

2019-01-18 15:03:45 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 204 0

2019-01-18 15:03:45 [cloudflaremiddleware] DEBUG: Successfully bypassed the protection for https://bina.az/items/all, re-scheduling the request


P.S: I have updated cfscrape to latest version.






python-3.x scrapy







share|improve this question























  • Have you considered opening a bug report at github.com/Anorov/cloudflare-scrape/issues ?

    – Gallaecio
    2 days ago











  • Okay man, i will

    – Adilzada
    2 days ago














1












1








1








i have installed cfscrape,and use it in my scrapy project to bypass cloudfare protection.It seems not working.I can add some more information about my code if necessary.I provide some code from settings.py:`



DOWNLOAD_DELAY = 0.25



DEFAULT_REQUEST_HEADERS = {

  'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',

  'Accept-Language': 'en',

}

USER_AGENT_LIST = [

    'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.7 (KHTML, like asdlkjqwdj) Chrome/16.0.912.36 Safari/535.7',

    'Mozilla/5.0 (Windows NT 2.1; WOW64) AppleWebKit/535.7 (KHTML, like kqjwdqwd) Chrome/16.0.912.36 Safari/123123.123',

    'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.7 (KHTML, like asdqwdqw) Chrome/16.0.912.36 Safari/535.7'

]



DOWNLOADER_MIDDLEWARES = {

    # The priority of 560 is important, because we want this middleware to kick in just before the scrapy built-in `RetryMiddleware`.

     'scrapy_cloudflare_middleware.middlewares.CloudFlareMiddleware': 560,

     'binaaz.middlewares.RandomUserAgentMiddleware': 400,

     'binaaz.middlewares.ProxyMiddleware': 410,

     'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware': None

    # Disable compression middleware, so the actual HTML pages are cached

}


and here is output:



2019-01-18 15:02:59 [scrapy.core.engine] INFO: Spider opened

2019-01-18 15:02:59 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)

2019-01-18 15:02:59 [scrapy.extensions.telnet] DEBUG: Telnet console listening on 127.0.0.1:6023

2019-01-18 15:03:00 [cloudflaremiddleware] DEBUG: Cloudflare protection detected on https://bina.az/items/all, trying to bypass...

2019-01-18 15:03:00 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): bina.az

2019-01-18 15:03:00 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 503 None

2019-01-18 15:03:08 [urllib3.connectionpool] DEBUG: Resetting dropped connection: bina.az

2019-01-18 15:03:09 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /cdn-cgi/l/chk_jschl?jschl_vc=6d688f88cb5562fd00a5ee16689e26c9&pass=1547809384.962-1OiFgM1jrn&jschl_answer=18.089701993600002 HTTP/1.1" 302 159

2019-01-18 15:03:09 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 503 None

2019-01-18 15:03:17 [urllib3.connectionpool] DEBUG: Resetting dropped connection: bina.az

2019-01-18 15:03:17 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /cdn-cgi/l/chk_jschl?jschl_vc=e719be7793db058c78b89dc24f6e39b7&pass=1547809393.639-gfrUEjO5oa&jschl_answer=26.1093884035 HTTP/1.1" 302 159

2019-01-18 15:03:17 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 204 0

2019-01-18 15:03:17 [cloudflaremiddleware] DEBUG: Successfully bypassed the protection for https://bina.az/items/all, re-scheduling the request

2019-01-18 15:03:18 [cloudflaremiddleware] DEBUG: Cloudflare protection detected on https://bina.az/items/all, trying to bypass...

2019-01-18 15:03:18 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): bina.az

2019-01-18 15:03:18 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 503 None

2019-01-18 15:03:26 [urllib3.connectionpool] DEBUG: Resetting dropped connection: bina.az

2019-01-18 15:03:27 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /cdn-cgi/l/chk_jschl?jschl_vc=faf6647605a0b45ed9958c055de11334&pass=1547809403.2-RdtkjUMCBv&jschl_answer=35.723157815600004 HTTP/1.1" 302 159

2019-01-18 15:03:27 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 503 None

2019-01-18 15:03:35 [urllib3.connectionpool] DEBUG: Resetting dropped connection: bina.az

2019-01-18 15:03:35 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /cdn-cgi/l/chk_jschl?jschl_vc=8a0665b0b9df3707f15de0299d196b33&pass=1547809411.873-xrZecnfmeE&jschl_answer=-3.7878104051 HTTP/1.1" 302 159

2019-01-18 15:03:36 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 204 0

2019-01-18 15:03:36 [cloudflaremiddleware] DEBUG: Successfully bypassed the protection for https://bina.az/items/all, re-scheduling the request

2019-01-18 15:03:36 [cloudflaremiddleware] DEBUG: Cloudflare protection detected on https://bina.az/items/all, trying to bypass...

2019-01-18 15:03:36 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): bina.az

2019-01-18 15:03:36 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 503 None

2019-01-18 15:03:45 [urllib3.connectionpool] DEBUG: Resetting dropped connection: bina.az

2019-01-18 15:03:45 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /cdn-cgi/l/chk_jschl?jschl_vc=671d8f3bfcb227960607a55f33183748&pass=1547809421.422-h5xJraFbRf&jschl_answer=20.2767620475 HTTP/1.1" 302 159

2019-01-18 15:03:45 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 204 0

2019-01-18 15:03:45 [cloudflaremiddleware] DEBUG: Successfully bypassed the protection for https://bina.az/items/all, re-scheduling the request


P.S: I have updated cfscrape to latest version.






python-3.x scrapy







share|improve this question














i have installed cfscrape,and use it in my scrapy project to bypass cloudfare protection.It seems not working.I can add some more information about my code if necessary.I provide some code from settings.py:`



DOWNLOAD_DELAY = 0.25



DEFAULT_REQUEST_HEADERS = {

  'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',

  'Accept-Language': 'en',

}

USER_AGENT_LIST = [

    'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.7 (KHTML, like asdlkjqwdj) Chrome/16.0.912.36 Safari/535.7',

    'Mozilla/5.0 (Windows NT 2.1; WOW64) AppleWebKit/535.7 (KHTML, like kqjwdqwd) Chrome/16.0.912.36 Safari/123123.123',

    'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.7 (KHTML, like asdqwdqw) Chrome/16.0.912.36 Safari/535.7'

]



DOWNLOADER_MIDDLEWARES = {

    # The priority of 560 is important, because we want this middleware to kick in just before the scrapy built-in `RetryMiddleware`.

     'scrapy_cloudflare_middleware.middlewares.CloudFlareMiddleware': 560,

     'binaaz.middlewares.RandomUserAgentMiddleware': 400,

     'binaaz.middlewares.ProxyMiddleware': 410,

     'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware': None

    # Disable compression middleware, so the actual HTML pages are cached

}


and here is output:



2019-01-18 15:02:59 [scrapy.core.engine] INFO: Spider opened

2019-01-18 15:02:59 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)

2019-01-18 15:02:59 [scrapy.extensions.telnet] DEBUG: Telnet console listening on 127.0.0.1:6023

2019-01-18 15:03:00 [cloudflaremiddleware] DEBUG: Cloudflare protection detected on https://bina.az/items/all, trying to bypass...

2019-01-18 15:03:00 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): bina.az

2019-01-18 15:03:00 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 503 None

2019-01-18 15:03:08 [urllib3.connectionpool] DEBUG: Resetting dropped connection: bina.az

2019-01-18 15:03:09 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /cdn-cgi/l/chk_jschl?jschl_vc=6d688f88cb5562fd00a5ee16689e26c9&pass=1547809384.962-1OiFgM1jrn&jschl_answer=18.089701993600002 HTTP/1.1" 302 159

2019-01-18 15:03:09 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 503 None

2019-01-18 15:03:17 [urllib3.connectionpool] DEBUG: Resetting dropped connection: bina.az

2019-01-18 15:03:17 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /cdn-cgi/l/chk_jschl?jschl_vc=e719be7793db058c78b89dc24f6e39b7&pass=1547809393.639-gfrUEjO5oa&jschl_answer=26.1093884035 HTTP/1.1" 302 159

2019-01-18 15:03:17 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 204 0

2019-01-18 15:03:17 [cloudflaremiddleware] DEBUG: Successfully bypassed the protection for https://bina.az/items/all, re-scheduling the request

2019-01-18 15:03:18 [cloudflaremiddleware] DEBUG: Cloudflare protection detected on https://bina.az/items/all, trying to bypass...

2019-01-18 15:03:18 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): bina.az

2019-01-18 15:03:18 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 503 None

2019-01-18 15:03:26 [urllib3.connectionpool] DEBUG: Resetting dropped connection: bina.az

2019-01-18 15:03:27 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /cdn-cgi/l/chk_jschl?jschl_vc=faf6647605a0b45ed9958c055de11334&pass=1547809403.2-RdtkjUMCBv&jschl_answer=35.723157815600004 HTTP/1.1" 302 159

2019-01-18 15:03:27 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 503 None

2019-01-18 15:03:35 [urllib3.connectionpool] DEBUG: Resetting dropped connection: bina.az

2019-01-18 15:03:35 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /cdn-cgi/l/chk_jschl?jschl_vc=8a0665b0b9df3707f15de0299d196b33&pass=1547809411.873-xrZecnfmeE&jschl_answer=-3.7878104051 HTTP/1.1" 302 159

2019-01-18 15:03:36 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 204 0

2019-01-18 15:03:36 [cloudflaremiddleware] DEBUG: Successfully bypassed the protection for https://bina.az/items/all, re-scheduling the request

2019-01-18 15:03:36 [cloudflaremiddleware] DEBUG: Cloudflare protection detected on https://bina.az/items/all, trying to bypass...

2019-01-18 15:03:36 [urllib3.connectionpool] DEBUG: Starting new HTTPS connection (1): bina.az

2019-01-18 15:03:36 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 503 None

2019-01-18 15:03:45 [urllib3.connectionpool] DEBUG: Resetting dropped connection: bina.az

2019-01-18 15:03:45 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /cdn-cgi/l/chk_jschl?jschl_vc=671d8f3bfcb227960607a55f33183748&pass=1547809421.422-h5xJraFbRf&jschl_answer=20.2767620475 HTTP/1.1" 302 159

2019-01-18 15:03:45 [urllib3.connectionpool] DEBUG: https://bina.az:443 "GET /items/all HTTP/1.1" 204 0

2019-01-18 15:03:45 [cloudflaremiddleware] DEBUG: Successfully bypassed the protection for https://bina.az/items/all, re-scheduling the request


P.S: I have updated cfscrape to latest version.






python-3.x scrapy




python-3.x scrapy






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked 2 days ago









AdilzadaAdilzada

236




236













  • Have you considered opening a bug report at github.com/Anorov/cloudflare-scrape/issues ?

    – Gallaecio
    2 days ago











  • Okay man, i will

    – Adilzada
    2 days ago



















  • Have you considered opening a bug report at github.com/Anorov/cloudflare-scrape/issues ?

    – Gallaecio
    2 days ago











  • Okay man, i will

    – Adilzada
    2 days ago

















Have you considered opening a bug report at github.com/Anorov/cloudflare-scrape/issues ?

– Gallaecio
2 days ago





Have you considered opening a bug report at github.com/Anorov/cloudflare-scrape/issues ?

– Gallaecio
2 days ago













Okay man, i will

– Adilzada
2 days ago





Okay man, i will

– Adilzada
2 days ago












0






active

oldest

votes











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54252837%2fcloudfare-middleware-recursively-bypassess-protection%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























0






active

oldest

votes








0






active

oldest

votes









active

oldest

votes






active

oldest

votes
















draft saved

draft discarded




















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid …



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54252837%2fcloudfare-middleware-recursively-bypassess-protection%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







-

Popular posts from this blog

Homophylophilia

Image
Haec pagina ad tempus protecta est ob maleficia perpetua. Quaesemus, emendationes suggere in pagina disputationis vel magistratum ad hanc deprotegendam pete. Homophylophilia Hyacinthus complexus ab amante suo Apolline, in pictura Mors Hyacinthi quam Jean Broc pinxit. Homophylophilia est sui sexus vel generis amor vel mores romantici vel sexuales, res venerea masculina in masculos, aut mulier in mulieres appetentes. Hoc ingenium sexuale est "exemplar permanens vel inclinatio ad experiendam sexualem, adfectionalem, vel romanticam vim attrahendi," praecipue vel solum ex hominibus eiusdem sexús; notio "etiam ad hominis iudicium identitatis privatae socialisque in illis attractionibus conditum refert, in moribus quoque qui illos exprimant, et in societate communitatis aliorum qui illos communicent." [1] [2] Per aevos alii aliter rem definiverunt, sed Occidentali nostrae aetatis cultu civili non tantum ad ipsa acta erotica spectat, sed etiam igitur
Read more

Updating UILabel text programmatically using a function

Image
0 so I am trying to update the UIlabel text using a func. I have seen few other questions like mine and I tried their method but it doesn't work. fun returns no value when used in the UIlabel. I have shown my code below.It has to be noted that when I use the function in viewdidload I can retrieve the data. lazy var name: UILabel = { let lb = UILabel() lb.text? = getthename() lb.lineBreakMode = NSLineBreakMode.byWordWrapping lb.textColor = .mainGreen lb.textAlignment = .left lb.font = UIFont.boldSystemFont(ofSize:12.0) return lb }() func getthename() -> String { var wins : String = "" let ud = (Auth.auth().currentUser?.uid) self.ref = Database.database().reference() self.ref.child("Users").child(ud!).observeSingleEvent(of: .value, with: {(snapshot) in
Read more

Cloud Functions - OpenCV Videocapture Read method fails for larger files from cloud storage

Image
0 I created a cloud function to read a video file from cloud storage and read frames and create an image for each frame. This function works for small files which is 1-2 MB, However, this method fails while reading larger file 130 MB. import cv2 from google.cloud import storage vidcap = cv2.VideoCapture("https://storage.googleapis.com/[bucker_name]/[filename.mp4]") if vidcap.isOpened(): vidcap.open("https://storage.googleapis.com/[bucker_name]/[filename.mp4]") success,image = vidcap.read() print(success) Expected the print will return true and an image will hold an image of a first frame. python-3.x function opencv cloud share | improve this question
Read more