Problem:-

I've two nested loop inside a main for loop fetching different data from a range of 5 pages. Every page has 5 rows, But after writing data to csv, I only get 5 cells in which each cell has 5 values.

Solution I want :-

I want all the dates as well as time to be in different rows, so total there should be at least 25 rows.

My Minimal Code :-

all = []

for i in range(d, 0, -1):
    driver.find_element_by_link_text(f'{i}').click()
    time.sleep(5)
    c12 = []
    d12 = []

    date1 = driver.find_elements_by_class_name('GridBiMonthlyBonusEligibleListDatetd.NoSortColumn')
    for i in reversed(date1):
        print(i.text)
        c12.append(i.text)
        if i is None:
            break

    date_time = driver.find_elements_by_class_name('GridBiMonthlyBonusEligibleListLoginDatetd.NoSortColumn')
    for i1 in reversed(date_time):
        print(i1.text)
        d12.append(i1.text)
        if i1 is None:
            break

    z = [c12, d12]
    all.append(z)


 df = pd.DataFrame(all)
 path = 'C:\\Application Data\\pyt_project\\Google_sign_in_automation'
 a1 = ['Date', 'Date_time']
 filename = 
 dt.datetime.now().strftime("Login_logout_time_%d_%b_%y_%I_%M_%p.csv")
 p1 = os.path.join(path, filename)
 df.to_csv(p1, headers = a1, index=True)
 print('printed output')

CSV Output I got from above code

                              Date                                        Date_time
 03 Nov 2021 08 Nov 2021 09 Nov 2021 10 Nov 2021 11 Nov 2021 | 18:39 12:59 13:05 12:57 12:57

CSV Output I want

   Date     Date_time
03 Nov 2021  18:39
08 Nov 2021  12:59
09 Nov 2021  13:05
10 Nov 2021  12:57
11 Nov 2021  12:57

Things I tried but didn't worked

df = pd.DataFrame({'Date' : [c12],
                'Date_time' : [d12]
                },
                columns=['Date', 'Date_time'])

CodePudding user response：

Note Because the question only considers part of the code, I'll assume that everything that happens before works and only go into the obvious parts

I want all the dates as well as time to be in different rows

You can simply zip() your two lists if they have the same length:

pd.DataFrame(zip(c12,d12),columns=['Date', 'Date_time'])

Example

import pandas as pd

c12 = ['03 Nov 2021', '08 Nov 2021','09 Nov 2021','10 Nov 2021','11 Nov 2021']
d12 = ['18:39','12:59','13:05','12:57','12:57']

pd.DataFrame(zip(c12,d12),columns=['Date', 'Date_time'])

Output

But after writing data to csv, I only get 5 cells in which each cell has 5 values.

Put the lists to keep your data outside the loop and check your indentation.

c12 = []
d12 = []

for i in range(d, 0, -1):
    driver.find_element_by_link_text(f'{i}').click()
    time.sleep(5)

    date1 = driver.find_elements_by_class_name('GridBiMonthlyBonusEligibleListDatetd.NoSortColumn')
    for i in reversed(date1):
        print(i.text)
        c12.append(i.text)
        if i is None:
            break

    date_time = driver.find_elements_by_class_name('GridBiMonthlyBonusEligibleListLoginDatetd.NoSortColumn')
    for i1 in reversed(date_time):
        print(i1.text)
        d12.append(i1.text)
        if i1 is None:
            break

pd.DataFrame(zip(c12,d12),columns=['Date', 'Date_time'])#.to_csv(...)