Home > Back-end >  I want the scripts to run before I parse the html. I using dart(flutter)
I want the scripts to run before I parse the html. I using dart(flutter)

Time:11-08

When I make request I get this.

I want the html to load and the run the scripts to create the full body before I parse it.

I working with flutter(dart).

I tried using the webview but I'm not getting the html from after the page is loaded.

When I make the request to the site, this is what the response looks like.

I need some help guys.

How should I go about it?

<!DOCTYPE HTML>
<html lang="en-US">

<head>
    <meta charset="UTF-8" />
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=Edge,chrome=1" />
    <meta name="robots" content="noindex, nofollow" />
    <meta name="viewport" content="width=device-width,initial-scale=1" />
    <title>Just a moment...</title>
    <style type="text/css">
        html,
        body {
            width: 100%;
            height: 100%;
            margin: 0;
            padding: 0;
        }

        body {
            background-color: #ffffff;
            color: #000000;
            font-family: -apple-system, system-ui, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen, Ubuntu, "Helvetica Neue", Arial, sans-serif;
            font-size: 16px;
            line-height: 1.7em;
            -webkit-font-smoothing: antialiased;
        }

        h1 {
            text-align: center;
            font-weight: 700;
            margin: 16px 0;
            font-size: 32px;
            color: #000000;
            line-height: 1.25;
        }

        p {
            font-size: 20px;
            font-weight: 400;
            margin: 8px 0;
        }

        p,
        .attribution,
            {
            text-align: center;
        }

        #spinner {
            margin: 0 auto 30px auto;
            display: block;
        }

        .attribution {
            margin-top: 32px;
        }

        @keyframes fader {
            0% {
                opacity: 0.2;
            }

            50% {
                opacity: 1.0;
            }

            100% {
                opacity: 0.2;
            }
        }

        @-webkit-keyframes fader {
            0% {
                opacity: 0.2;
            }

            50% {
                opacity: 1.0;
            }

            100% {
                opacity: 0.2;
            }
        }

        #cf-bubbles>.bubbles {
            animation: fader 1.6s infinite;
        }

        #cf-bubbles>.bubbles:nth-child(2) {
            animation-delay: .2s;
        }

        #cf-bubbles>.bubbles:nth-child(3) {
            animation-delay: .4s;
        }

        .bubbles {
            background-color: #f58220;
            width: 20px;
            height: 20px;
            margin: 2px;
            border-radius: 100%;
            display: inline-block;
        }

        a {
            color: #2c7cb0;
            text-decoration: none;
            -moz-transition: color 0.15s ease;
            -o-transition: color 0.15s ease;
            -webkit-transition: color 0.15s ease;
            transition: color 0.15s ease;
        }

        a:hover {
            color: #f4a15d
        }

        .attribution {
            font-size: 16px;
            line-height: 1.5;
        }

        .ray_id {
            display: block;
            margin-top: 8px;
        }

        #cf-wrapper #challenge-form {
            padding-top: 25px;
            padding-bottom: 25px;
        }

        #cf-hcaptcha-container {
            text-align: center;
        }

        #cf-hcaptcha-container iframe {
            display: inline-block;
        }
    </style>

    <meta http-equiv="refresh" content="35">
    <script type="text/javascript">
        //<![CDATA[
    (function(){
      
      window._cf_chl_opt={
        cvId: "2",
        cType: "non-interactive",
        cNounce: "86849",
        cRay: "6a97b753b854da86",
        cHash: "31c7b92e6efc7fe",
        cPMDTk: "",
        cFPWv: "b",
        cTTimeMs: "1000",
        cRq: {
          ru: "aHR0cHM6Ly9nZy5jby51ay9yYWNpbmcvMjctb2N0LTIwMjEvZmFrZW5oYW0tMTI1NQ==",
          ra: "UG9zdG1hblJ1bnRpbWUvNy4yOC40",
          rm: "R0VU",
          d: "kPxE7HAIMaSY0 Q26IXxpuTBsg9MYQ7J/ICHGaynkKTojWIXlwrmVPwT F9kXDNE8Jc60RxAMi3ET5lG3UfaNS8h7kz8aJbykZ4EwheY/kJcnVBvvndt9WGhXuDvU0FOh9cZ9SAgRuYgtmkn hRxxi5owt/uhRmgg5MfZWETvbbMY/jCMrFpPhenjJFusWE1eM0Uz1E7p5hOK/W9P9wDRmC4A8f4z9PTWHMJY6EjsHR5GNEOZzHvWUd5uKRmIbQxylvGKD8smzOR1mUYrI hlAUB8U6wptxbbpDZDcxkEYk7M/bp4oIkQrJNBkPvVGmnVcBvZN2QrSJdUIVFNA9kjKh4AfzpxPMkfqhoJIXOhMQK lr1R7T3wHf/qRMpzvgN5HD8wSySaGq ZMUraHmms1bvI3p25Ril83l3CvfGA573nzB0Pl9wBt1Ilj et9TonO/yCPUT9ncqu/HsuK6zoTsYs1XO9emsj8iGHyH20tEDL/QIQu0Xm Na2nREmz4LLYNg/y/S/IlGc8LjDv548dVWGJxGuPSzeDWod49nzqXZR6u4otbfcHIIMdEgPjTa",
          t: "MTYzNjEzMjQ0Mi4yMDcwMDA=",
          m: "UHh8TWbrY8rIVckQYr2DEHf3xtqOy3IWbHpurhKo0Dg=",
          i1: "bPb8laPvucq92fuTb8UHJQ==",
          i2: "uyNfsEipqDYhS6 qFdYUkA==",
          zh: "C039uzYEN6IwJDV7FwZVYsBSveHeK850qjEkkfVJ6vw=",
          uh: "q2fUGDQwB25BKEmSOHJOj1NUZxjS6q3TVwtPqObOVZM=",
          hh: "nfF1CEMggKTkMnWiuwyIjIi54NW2HaYcu4gV2CdK4ks=",
        }
      }
      window._cf_chl_enter = function(){window._cf_chl_opt.p=1};
      
    })();
    //]]>
    </script>


</head>

<body>
    <table width="100%" height="100%" cellpadding="20">
        <tr>
            <td align="center" valign="middle">
                <div class="cf-browser-verification cf-im-under-attack">
                    <noscript>
                        <h1 data-translate="turn_on_js" style="color:#bd2426;">Please turn JavaScript on and reload the
                            page.</h1>
                    </noscript>
                    <div id="cf-content" style="display:none">

                        <div id="cf-bubbles">
                            <div class="bubbles"></div>
                            <div class="bubbles"></div>
                            <div class="bubbles"></div>
                        </div>
                        <h1><span data-translate="checking_browser">Checking your browser before accessing</span>
                            gg.co.uk.</h1>

                        <div id="no-cookie-warning" class="cookie-warning" data-translate="turn_on_cookies"
                            style="display:none">
                            <p data-translate="turn_on_cookies" style="color:#bd2426;">Please enable Cookies and reload
                                the page.</p>
                        </div>
                        <p data-translate="process_is_automatic">This process is automatic. Your browser will redirect
                            to your requested content shortly.</p>
                        <p data-translate="allow_5_secs" id="cf-spinner-allow-5-secs">Please allow up to 5
                            seconds&hellip;</p>
                        <p data-translate="redirecting" id="cf-spinner-redirecting" style="display:none">
                            Redirecting&hellip;</p>
                    </div>

                    <form class="challenge-form" id="challenge-form"
                        action="/racing/27-oct-2021/fakenham-1255?__cf_chl_jschl_tk__=I4acmyvWUrWbLXAGjYC5HXUh31EzMntIEES9DIs8Yg8-1636132442-0-gaNycGzNBmU"
                        method="POST" enctype="application/x-www-form-urlencoded">
                        <input type="hidden" name="md" value="WYgGQTwD0.PlAYN7zN9M7XpYgnUJxpt6W_rYCboneoQ-1636132442-0-AfDISHvT2GBIpWp1EPYu9cdxCuFQpLT_u7UKTH9TL-mj0X8Xxw9raGV2KUlDtB4DX7Z3h-LYRsfmLlNmrV-WstInbyHg2H3gWXOuMU2mnguq0TdSnDmB0MWffohG7RydsDYePGxxBYBAXhJMAYyRLWlbcprMPKCaHVm9zAxWf5tZFgYfCLP-CLdoXXS_HuKNhA0iXqanK-LYFcSJXKuvTr0M7oAx-vhljyX8JW3sJ2sgJlV7vLLB9MPl3l7WgWBZ4q0bW8KKVhNi3WXReGdIZUpIirNA6D2xHAe5oyfDMqUfEyopDE2EFGeNxKb2yI0qPvmmaup4WjHIG0DangljT0VcnK8oR2-lz7tVreiQQS50DcnX_EiDswWlbfXDvz83RI3IDOmSjBJXQZBSClMjhkCPs9yf5wOGXtq2jjd2sIFFWgw3zS5Q0n9dsTfFCNSFRGtOtHmCM2ztI8ivbpfvf27CNK55Xb7b2BPCOICarJUqM16ELz7AAQ6lS0OJymzCU0G1eC58DDvCiMCx6AC7nAI2ce38OjfjYHO5Wq_lI8u8VFrGaS4nZyySvn6-0gGqRA" />
                        <input type="hidden" name="r" value="i7Mahy4asxjhgyvRAVdQk8RCAO5shxagJMSNb2vkdXM-1636132442-0-Acl9elCj95M0xGc183cGdekXyIhJ RXABhl9YAfeIxJZd5a0u4EX8e1wb7wJdeYsBwO/e42SiuMP3O5t3PoeEibaVnEIPnLgipqmvVXoSiOuK302AlzY5JmI0Xy80sRBsoyWLglYAEPZft7cFoS XK3ayn3GMqgoQpnf8nhjeVmOx6g1v25gqwGYoz6MdvE/7Z2GAwvCnqP2Bk7fjp7gBBQ1pWSgzCEP2jXsbYuqcBE7HHlNbMyJF1tqOIAd1Hm7S8KtAQUllGHNfJy7s/L6Ibvg55SCP8XHUlAqASeMvqbJors5HimAx iZgpjfYhetytGpQTqh8m7BlVyZBUkvVl6HA3r0eR0OSVOOUf/8BmuYTMrZKqkElg BY4sNTy0UqLeuyLH5RS3zMLdNDx75IkGexDVPx2KXBxPwdmPhVO2jmoJdSl ocBTzfPqjsXw XAyo9Sm5id7TZ/Z xxXYA2RMCpBiGBXPdfu2U CJI4/S/6TWfmunwpMwRIfEby1fQNmvNCJWJPIybXpZR0tOH/eln7CoeR8qaSrCUjP7H5fNSgiIi/gtH2oxm3D20y5T4Hd1EGEtKf0E1pLUGSeZY8LBsIMtvPlnXr5srsz4965 WZu87RK9w8filse 9rtIT6KM55QcgIE7Cvb4yfI74x7y uJ0bjUoc8D3ke5GjK4isNsMlh0kxgY9gVJmzxReZ9n9ahT77FOcOJh8eQEQOCPZQ8J9DqUduOkH7775AvPseKj1CPb3   EJvoaZGdNIfYvUV6PjMAsnthNOaCKDQ8oHiYZLW8A8YxF2AUrQxMBGjeIjvv8OCAkFwk4RhX/JSqNXgWE mj6x ySSNFtBJp7cOEdB27Teyx11ImRvTXBZdv09XFIgYRO7WYmI/1sB/qfn65uL1Pr0chh5mCX8FgX1r/CRw/IkvJhLpDPNxq0MiXJK/gUuZK7URNCQI3ep Dv 7lyuvi9sQgkXl2ne8fSp8Nbh3MogLWFAYx0o7Jz1iZY4Vtryw87I e6Ditw9eklqURwEiSE7yIJ5foZGLZwClJs9W1pfpZ2G2OIU8HBvN/O4ERCqT2nLRVJH513AUzPv9DpowCMUxANLCAIvJL9XmSbVeUBvUQmdpOK grZlgFRx6d8/kKepJStd8iuN/P9SGFPmOGKQHmECFiu70JPk4QCS3J08BjhuIpC4uvfLNmAzQi1yYEOpwSfv1XPf4DLS/GO59qg/jbgu0UZyRBQDL/KE4wOiE8Dtw7ACjkUtaON1NDgOS5nKkUu0diqIIRvreQdLCy4ehjuvG7LuWPPdrEmU4ZYS3R IPHNBEBbak2pszPGm8Xc0GtVWldmtAeRr sN0K8JeRPIbORtPuhwMPpGGosCiPmqxHshlyS0qPB0kYsykHvjDXzI63E3EccRyo353ByRSLe0wcz0k32Ku4I7uDGN/YmQEcggLEVsShXXUFA FOgQzuM6PE9o NPgke/M1mJBIuyyOrBtQFNskIgXQA0jPVVpJz84OeR3ylkTsVDGOvnhj5nYPEk27gDxHVoYc2C6Omi/FwRaZcGQrBrTxCTejlnmEnGTUd M"/>
                        <input type="hidden" value="3734809bfba8e7119ff4a9e747f0a631" id="jschl-vc" name="jschl_vc"/>
                        <!-- <input type="hidden" value="" id="jschl-vc" name="jschl_vc"/> -->
                        <input type="hidden" name="pass" value="1636132443.207-qh8v/uVVqU"/>
                        <input type="hidden" id="jschl-answer" name="jschl_answer"/>
  </form>

                        <script type="text/javascript">
                            //<![CDATA[
      (function(){
          var a = document.getElementById('cf-content');
          a.style.display = 'block';
          var isIE = /(MSIE|Trident\/|Edge\/)/i.test(window.navigator.userAgent);
          var trkjs = isIE ? new Image() : document.createElement('img');
          trkjs.setAttribute("src", "/cdn-cgi/images/trace/jschal/js/transparent.gif?ray=6a97b753b854da86");
          trkjs.id = "trk_jschal_js";
          trkjs.setAttribute("alt", "");
          document.body.appendChild(trkjs);
          
          var cpo=document.createElement('script');
          cpo.type='text/javascript';
          cpo.src="/cdn-cgi/challenge-platform/h/b/orchestrate/jsch/v1?ray=6a97b753b854da86";
          document.getElementsByTagName('head')[0].appendChild(cpo);
        }());
      //]]>
                        </script>



                        <div id="trk_jschal_nojs"
                            style="background-image:url('/cdn-cgi/images/trace/jschal/nojs/transparent.gif?ray=6a97b753b854da86')">
                        </div>
                </div>


                <div class="attribution">
                    DDoS protection by <a rel="noopener noreferrer" href="https://www.cloudflare.com/5xx-error-landing/"
                        target="_blank">Cloudflare</a>
                    <br />
                    <span class="ray_id">Ray ID: <code>6a97b753b854da86</code></span>
                </div>
            </td>

        </tr>
    </table>
</body>

</html>
<iframe name="sif1" sandbox="allow-forms allow-modals allow-scripts" frameborder="0"></iframe>

CodePudding user response:

try to move <script> at the top of your app

CodePudding user response:

I used the flutter WebView and it worked.

I wrote the full js script in a file.

The script does the parsing and returns the required data as json.

So I pass the script file content to my WebView controller's evaluateJavaScript and the result will be json. So I can just take that data and work with it.

// webview's controller
WebViewController? _controller;

// setup your webview and get the controller 
WebView(
  initialUrl: widget.url ?? 'https://www.google.com.tr',
  onWebViewCreated: (controller) => _controller = controller,
  javascriptMode: JavascriptMode.unrestricted,
  gestureNavigationEnabled: true,
  onPageFinished: (_) => readJS(),
);

// 
void readJS() async {
  String js = await fileReader.readFile("lib/js/main.js"); // read script file
  String data = await _controller!.evaluateJavascript(js);
  print("data\n$data");
}

check out the full code here

  • Related