Selenium chromedriver for retrieving tables from a Webpage and export in csv file-CodePudding

table structure for web Url is below.

<table data-v-58f8de9f="" >
        <span data-v-58f8de9f="" >
        <thead data-v-58f8de9f="">
            <tr data-v-58f8de9f="" >
                <th data-v-58f8de9f="" colspan="1" rowspan="1" ><span data-v-58f8de9f="" >Exchange</span><!----></th>
                <th data-v-58f8de9f="" colspan="1" rowspan="1" ><span data-v-58f8de9f="" >Calls</span><!----></th>
                <th data-v-58f8de9f="" colspan="1" rowspan="1" ><span data-v-58f8de9f="" >Puts</span><!----></th>
                <th data-v-58f8de9f="" colspan="1" rowspan="1" ><span data-v-58f8de9f="" >P/C Ratio</span><!----></th>
                <th data-v-58f8de9f="" colspan="1" rowspan="1" ><span data-v-58f8de9f="" >Volume</span><!----></th>
                <th data-v-58f8de9f="" colspan="1" rowspan="1" ><span data-v-58f8de9f="" >Market Share</span><!----></th>
            </tr>
        </thead></span>
        <tbody data-v-c962f2de="" data-v-58f8de9f="">
            <tr data-v-c962f2de="" >
                <td data-v-c962f2de="" >AMEX</td>
                <td data-v-c962f2de="" >3,497,381</td>
                <td data-v-c962f2de="" >4,251,149</td>
                <td data-v-c962f2de="" >0.1</td>
                <td data-v-c962f2de="" >1,678,530</td>
                <td data-v-c962f2de="" >5.70%</td>
            </tr>
        </tbody><!----><!----><!---->

Code which i m trying somehow below if condition is not satisfied can you please help me to extract above table record from Web page. Thanks in advance!

    class ExtractDataFromATable
    {
        static void Main(string[] args)
        {
            IWebDriver driver = new ChromeDriver();
            string Total=string.Empty;
            driver.Navigate().GoToUrl("some URL");
// All Html source code are getting in WebSource 
            string WebSource = driver.PageSource;
            string matchvalue = "<table 'marketData - table marketData - table--active marketData-table--border marketData-table--padded marketData-table--auto table-responsive'"   ">";
            if(WebSource.Contains(matchvalue))
            {
                int start=WebSource.IndexOf(matchvalue);
                string starttoend=WebSource.Substring(start);
                int end=starttoend.IndexOf("</table");
                Total=WebSource.Substring(start, end 1)  "/table>";
            }
        }
    }

CodePudding user response：

You should look into HTMLAglityPack. With HTMLAglityPack I would load the driver.Pagesouce into a new HTML Document and then do something like: var marketData = html.DocumentNode.SelectNodes("//td[@marketData-tableItem']");. Then you can run a for loop on the market data: foreach(var data in marketData){string x = data.innerText;}

CodePudding user response：

The below code is in Java, modify this code to C#, but you can use these same locators:

driver.get("<URL>");

// to accept cookies        
driver.findElement(By.xpath(".//*[text()='Accept analytic cookies']")).click();
        
// to click on the view button      
driver.findElement(By.cssSelector(".marketData-inputBtn")).click();
        
// to print the headings        
List<WebElement> headings = driver.findElements(By.xpath("(.//*[@class='marketData-resultsSection'])[2]//th/span"));
for (WebElement ele :  headings) {
    System.out.print(ele.getText()   " | ");
}
System.out.println();
        
// to print the data        
List<WebElement> data = driver.findElements(By.xpath("(.//*[@class='marketData-resultsSection'])[2]//td"));
int i = 0;
for (WebElement ele : data) {
    i  ;
    System.out.print(ele.getText()   " | ");
    if (i == 6) {
        i = 0;
        System.out.println();
    }
}

Output:

Exchange | Calls | Puts | P/C Ratio | Volume | Market Share | 
AMEX | 1,829,955 | 1,645,962 | 0.9 | 3,475,917 | 7.23% | 
...
...