Sj[K0ddlZddlZddlZddlZddlmZddlmZddlm Z m Z ddl m Z m Z ddl mZddlmZddlmZdd lmZdd lmZmZmZdd lmZdd lmZddlZdd lmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&ddl'm(Z(ddgZ)ddgZ*ddgZ+gdZ,dgZ-dZ.e(Z/dZ0de$fdZ1de#fdZ2de%de&fdZ3de%de&fdZ4de%de&fdZ5de%de&fd Z6de%de&fd!Z7de%de&fd"Z8de%de&fd#Z9de%d$e:de"fd%Z;d.d&ed'ede%d&efd+Z?de%d&ed$e:de"fd,Z@de%d&efd-ZAdS)/N) timedelta)escape)unquotequote)FunctionTimedOut func_timeout)TimeoutException) WebDriver)By)Keys)presence_of_element_located staleness_oftitle_is) ActionChains) WebDriverWait) STATUS_ERROR STATUS_OKChallengeResolutionResultTChallengeResolutionTHealthResponse IndexResponse V1RequestBaseV1ResponseBase)SessionsStoragez Access deniedz Attention Required! | Cloudflarez*div.cf-error-title span.cf-code-label spanz*#cf-error-details div.cf-error-overview h1zJust a moment...z DDoS-Guard) z#cf-challenge-runningz.ray_idz .attack-boxz#cf-please-waitz#challenge-spinnerz#trk_jschal_jsz#turnstile-wrapperz .lds-ringztd.info #js_infozdiv.vc div.text-box h2#input[name='cf-turnstile-response']c~tjdtjdtjztj}|)tjdt jdntjd|ztj}|dkr)tjdt jdntjd|ztjd tj }tjd |ztjd dS) Nz#Testing web browser installation...z Platform: z,Chrome / Chromium web browser not installed!rzChrome / Chromium path: z'Chrome / Chromium version not detected!z!Chrome / Chromium major version: zLaunching web browser...zFlareSolverr User-Agent: zTest successful!) logginginfoplatformutilsget_chrome_exe_patherrorsysexitget_chrome_major_versionget_user_agent)chrome_exe_pathchrome_major_version user_agents /app/flaresolverr_service.pytest_browser_installationr-<s L6777 L 1 3 33444/11O DEEE   //ABBB 9;;r!! ?@@@   8;OOPPP L+,,,%''J L,z9::: L#$$$$$returncti}d|_tj|_tj|_|S)NzFlareSolverr is ready!)rmsgr"get_flaresolverr_versionversionr( userAgentress r,index_endpointr7Ts;   C&CG022CK(**CM Jr.c<ti}t|_|S)N)rrstatusr5s r,health_endpointr:\s   CCJ Jr.reqcttjdz}tjdt j| t |}ni#t$r\}ti}d|_ t|_ dt|z|_ tj|j Yd}~nd}~wwxYw||_ttjdz|_t j|_tjdt j|tjd|j|jz dz d|S)Nz#Incoming request => POST /v1 body: TzError: zResponse => POST /v1 body: z Response in z s)inttimerr r"object_to_dict_controller_v1_handler Exceptionr __error_500__rr9strmessager$startTimestamp endTimestampr2r3debug)r;start_tsr6es r,controller_v1_endpointrKbsJ49;;%&&H LRu7KC7P7PRRSSS#$S)) ###R   ! #a&&(  ck"""""""" #"C49;;-..C022CK MK0DS0I0IKKLLL LR!1C4F!F$ NRRRSSS JsA C(AB??Cc(|jtd|jtjd|jtjd|jt|jdkrd|_|jdkrt|}n|jdkrt|}ni|jdkrt|}nN|jd krt|}n3|jd krt|}ntd |jd |S) Nz%Request parameter 'cmd' is mandatory.z;Request parameter 'headers' was removed in FlareSolverr v2.z=Request parameter 'userAgent' was removed in FlareSolverr v2.ri`zsessions.createz sessions.listzsessions.destroyz request.getz request.postzRequest parameter 'cmd' = 'z ' is invalid.) cmdrBheadersrwarningr4 maxTimeoutr>_cmd_sessions_create_cmd_sessions_list_cmd_sessions_destroy_cmd_request_get_cmd_request_post)r;r6s r,rArAws  w?@@@ {UVVV } WXXX ~S^!4!4q!8!8 w###"3'' O # # %% & & &#C(( M ! !s## N " "$$LcgLLLMMM Jr.cP|jtd|jtd|jt jd|jt jdt|d}ti}|j |_ |j |_ |j |_ |S)Nz>Request parameter 'url' is mandatory in 'request.get' command.z1Cannot use 'postBody' when sending a GET request.ARequest parameter 'returnRawHtml' was removed in FlareSolverr v2.rPrhsession_ttl_minutesrrigetrrHrDlifetimedriverr" get_webdriverrgr _evil_logicPLATFORM_VERSIONclosequitrrBreplace) r;rrtimeoutr{rfttlrhrkrJs r,r^r^sr#.!!D(G FM ; _J@C@Wa)C$;<<<<]aC-11*cBBNGU V eXbeeeffff U]gUU*-g.>.>.@.@*A*AUUILSUUUVVV^FF(33F M] ^ ^ ^G[32GHH { Mv1%--  KKMMM MK L L L L  M1 ZZZXgXXXYYY WWW7#a&&..u:U:UUVVVW{ Mv1%--  KKMMM MK L L L L  M1s%C;D=C>>DD6:BGG65G6tabscx|tjd}|d} t |||d}|r||krt jd||St jd|dtj d) NrrT)rzTurnstile token: z-Failed to extract token possibly click failedz let el = document.createElement('button'); el.style.position='fixed'; el.style.top='0'; el.style.left='0'; document.body.prepend(el); el.focus(); r) rr CSS_SELECTOR get_attributerrr rHexecute_scriptr?r)r{r token_input current_valueturnstile_tokens r,_get_turnstile_tokenr&s%%bo7\]]K--g66MVd++++%33G<<  '-// BBBCCC&& FGGG      1 %r.cd}|jtjd|jd||jd}t D]P}|tj|}t|dkrd}tj d|znQ|rt||j}ntjd|S) NNavigating to... z) in order to pass the turnstile challengeFrTz.Turnstile challenge detected. Selector found: )r{rzTurnstile challenge not found) tabs_till_verifyrrHrZryTURNSTILE_SELECTORS find_elementsr rlenr r)r;r{rturnstile_challenge_foundselectorfound_elementss r,_resolve_turnstile_captchar=sO ' \#'\\\]]] 37$)!+  H#11"/8LLN>""Q&&,0) MPXXYYY' % <2&sG[\\\OO M: ; ; ; r.c ti}t|_d|_t j}|j|j}|rmgd} tjd|| di| dd|in$#t$rtjdYnwxYwtjd|j d}|d krt||n2|j ||j nt||}|jt#|jd krtjd |jD]2}||d ||3|d krt||n||j t jrtjd |j|t.jd}|j} t4D]&} | | rtd't8D]D} |t.j| } t#| d krtdEd} t>D]G} | | krd} tj!d| znH| sXtDD]P} |t.j| } t#| d krd} tj!d| znQd }| r |dz}t>D]a} tjdtG|zdz| ztI|tJ&tO| btDD]m} tjdtG|zdz| ztI|tJ&tQt.j| fnnU#tR$rFtjdtU||t.jd}YnwxYw:tjd tI|tJ+tY|n$#t$rtjdYnwxYwtj!dd|_ntj!dd|_t[i}|j.|_ d|_|/|_t j0||_1||_2|j3sji|_4|j5rP|j5d krEtj!dtG|j5zdztmj7|j5|j|_8|j9r|:|_;||_<|S) Nr)*z*.pngz*.jpgz*.jpegz*.gifz*.webpz*.bmpz*.svgz*.icoz*.PNGz*.JPGz*.JPEGz*.GIFz*.WEBPz*.BMPz*.SVGz*.ICOz*.tiffz*.tifz*.jpez*.apngz*.avifz*.heicz*.heifz*.TIFFz*.TIFz*.JPEz*.APNGz*.AVIFz*.HEICz*.HEIFz*.cssz*.CSSz*.woffz*.woff2z*.ttfz*.otfz*.eotz*.WOFFz*.WOFF2z*.TTFz*.OTFz*.EOTzNetwork.setBlockedURLs: %szNetwork.enablezNetwork.setBlockedURLsurlsz>Network.setBlockedURLs failed or unsupported on this webdriverrrdrzSetting cookies...namezResponse HTML: htmlziCloudflare has blocked this request. Probably your IP is banned for this site, check in your web browser.FTz!Challenge detected. Title found: z$Challenge detected. Selector found: rzWaiting for title (attempt z): zWaiting for selector (attempt zTimeout waiting for selectorzWaiting for redirectzTimeout waiting for redirectzChallenge solved!zChallenge not detected!zWaiting z) seconds before returning the response...)=rrr9rEr"get_config_disable_media disableMediarrHexecute_cdp_cmdrBrZ _post_requestrryrcookiesr delete_cookie add_cookieget_config_log_html page_sourcerr TAG_NAMEtitleACCESS_DENIED_TITLES startswithACCESS_DENIED_SELECTORSrrCHALLENGE_TITLESlowerr CHALLENGE_SELECTORSrDr SHORT_TIMEOUT until_notrr r runtilrr current_url get_cookiesr(r4rreturnOnlyCookiesrN waitInSecondsr?rresponsereturnScreenshotget_screenshot_as_base64 screenshotr_)r;r{rrr6 disable_media block_urlsrcookie html_element page_titlerrrchallenge_foundattemptrbs r,r}r}Ps r " "CCJCK244M #( \     \ M6 C C C  " "#3R 8 8 8  " "#;fj=Q R R R R \ \ \ MZ [ [ [ [ [ \  M/cg//000O c6""""   ' JJsw    8fEEO {3s{#3#3a#7#7 +,,,k & &F   0 0 0   f % % % % V   #v & & & & JJsw     ""? =);==>>>&&r{F;;LJ&dd   ' ' dcdd d d,dd--boxHH ~   " "cdd d # O! ;;==J,,.. . ."O L""Q&&"& ChNOOO' G&0 H H!A+-TTEM"?#g,,"NQV"VY^"^___!&-88BB8E??SSSS!4RRHM"BS\\"QTY"Y\d"deee!&-88BB3R_h4OPPRRRR# H H H <===V$$$ &222;GG  H# H4  ,--- : &- 0 0 6 6|L7Q7Q R R R R : : : M8 9 9 9 9 9 :  ())))  .//// .r22M*MM"..00M#26::M$3M!  4 "    *!2Q!6!6 Lc#*;&<&<>SSc d|jd}|jr|jddkr|jn|jr|jddnd}|d}|D]}|dd} t|d}n#t$r |d}YnwxYw|d krN t |dkrt|dnd}n-#t$r t |dkr|dnd}YnwxYw|d d }|d tt|d tt|dz }|dz }d|d} | d | dS)Nz
r?rr&=submit"z"z
z
zC zk z+data:text/html;charset=utf-8,{html_content}) html_content) rZr[splitrrBrrrrryformat) r;r{ post_form query_stringpairspairpartsrrrs r,rrsHswHHHI#&<xCLOs4J4J3<=C>)r)Brr!r%r?datetimerrr urllib.parserrrrselenium.commonr #selenium.webdriver.chrome.webdriverr selenium.webdriver.common.byr selenium.webdriver.common.keysr .selenium.webdriver.support.expected_conditionsr rr'selenium.webdriver.common.action_chainsrselenium.webdriver.support.waitrr"dtosrrrrrrrrrmrrrrrrrrir-r7r:rKrArTrUrQrRrSrDr^r>rrrr}rr.r,rs? ''''''''77777777,,,,,,999999++++++//////9999999999@@@@@@999999 11111111111111111111%$$$$$& 10   * "?$$%%%0   .* .>-N&=^"m(Mn }     MMM3M;OMMMM@cB#.M9&V]VIVsVG[VVVVr`}`i``````r.