Home > Mobile >  How to iterate over the rows from 2 files, compare the values and update the value in a file when th
How to iterate over the rows from 2 files, compare the values and update the value in a file when th

Time:10-10

Here I have 2 files, first tryout file has lots of data as follows:

BEG_COMP_DATA
  17
   1 "                     stator_yoke"
   2 "              stator_tooth_shaft"
   3 "               stator_tooth_head"
   4 "                          coil_1"
   5 "                          coil_2"
   6 "                          coil_3"
   7 "                      stator_air"
   8 "                        rotor_IL"
   9 "                      rotor_core"
  10 "                        rotor_da"
  11 "                       rotor_nda"
  12 "               rotor_air_speiche"
  13 "                        magnet_1"
  14 "                        magnet_2"
  15 "                        magnet_3"
  16 "                       stator_il"
  17 "                      sldm_rotor"
  18 "                      magnet"
  19 "                      iron"
  20 "                      air"
END_COMP_DATA
..
..
..
..
BEG_ELEM_DATA
 -------        0        0        0     3156        0    15208        0        0        0       
 0        0        0
             1    1 106     1177     1200     1184     1201     1183     1202
             2    1 106        3     1203     1188     1204      121      123
          ...
          2270    6 106      404     2599     2559     2648      405      415
          2271    6 106     2533     2646     2557     2639     2538     2611
          ...
          3749   10 106    10690    10692    10693    10695    10658    10689
          3949   10 106    11201    11223    11224    11226    11171    11202

Its a huge data but for me only data which has value 10 in BEG_ELEM_DATA for different elements is imporatant(in last 2 rows, value 10 in column 2 comes after element 3749, 3949 and so on) and second file has densities, based on these densities, the value 10 changes to 18 or 19 or 20. The rule is if density_1 = 1 & density_2 = 0, then its a magnet, if density_1 = 1 & density_2 = 1, then its a iron otherwise its air for density_1 = 0 & density_2 = 0. Density file is as follows:

  $ON
$DVAR
3749   2   1.0   0
3949   2   1.0   0
4154   2   1.0   1.0
5335   2   0     0
..
..

for example, for element 3749 (who has value 10 in tryout file), from density file in column 3 and 4, its density_1 = 1 and density_2 = 0, then its a magnet and as for magnet belongs to 18 component from tryout file shown above. This 18 value needs to be assigned in place of 10 for 3749 element in tryout file.

  18 "                      magnet"
  19 "                      iron"
  20 "                      air"

So it will look like in Tryout_New file:

   3749   18 106    10690    10692    10693    10695    10658    10689

So far I tried to do this python, so first I added 18, 19, 20 in BEG_COMP_DATA and save to Tryout_New file and then tried to use for and if loop but I am stuck, anyone can help. My code so far is:

oldFileName = 'Tryout.hmo'
newFileName = 'Tryout_NEW.hmo'
topoFileName = 'DesnityFile.topo'
my_file = open(oldFileName)
string_list = my_file.readlines()
my_file.close()
string_list[21] = "      18 "'"                      magnet"'"\n"
string_list[22] = "      19 "'"                      iron"'"\n"
string_list[23] = "      20 "'"                      air"'"\n"

my_file_2 = open(newFileName, "w")
new_file_contents = "".join(string_list)
my_file_2.write(new_file_contents)
my_file_2.close()
readable_file = open(newFileName)
read_file = readable_file.read()
for line in string_list:
    my_file_2.write(line)
if line.find("BEG_ELEM_DATA") > -1:
    newFile.write("")
    topo_read=  open(topoFileName, "r")
    topolines=  topo_read.readlines()
    topo_read.close()
        for line in topolines:
            if desity_1= 1.0 and density_2= 0 :
                str1 = "magnet"
            elif desity_1= 1.0 and density_2= 1.0 :
                str2 = "iron"
            else:
                str3 = "air"
                

I really need help in this and I am new to programming. Thank you.

CodePudding user response:

I'm also still a beginner in Python, but I tried to solve your problem and here is my solution:

I guess there are way better ways to do it because here you have to import all data to a dataframe before comparing it.

Also I don't know if you can read your data with pd.read_csv to a dataframe because I don't know *.hmo and *.topo

import pandas as pd

df = pd.read_csv('tryout.csv', delimiter=';')
df2 = pd.read_csv('density.csv', delimiter=';')

for idx, row in df.iterrows():
    for idx2, row2 in df2.iterrows():
        if row[0] == row2[0]:
            if row2[2] == 1 and row2[3] == 0 :
                # it is magnet, value 18
                row[1] = 18
            elif row2[2] == 1 and row2[3] == 1 :
                # it is iron, value 19
                row[1] = 19
            else:
                # it is air, value 20
                row[1] = 20
df.to_csv('new_tryout.csv')

What my code is doing here, it loads both files to dataframes. Then iterate over every line to compare where the ID in both files is the same (e.g 3749). If true there are the 3 if statements whether it is magnet/iron/air and change the value in df to the right number.

At the end save the new df to a new file 'new_tryout.csv'

I created 2 testfiles for it and it worked the way it should.

CodePudding user response:

ok, here is another solution. For anybody else who reads this: this is still only a temporary solution but @Sagar and me both don't know to do it better.

import pandas as pd
df = pd.read_csv('tryout.hmo', header = 0, names = list('ABCDEFGHIJKLM'), delimiter=r'\s ', skiprows=[i for i in range(52362)])
df2 = pd.read_csv('Density.topo', header = 0, names = list('ANOP'), delimiter=r'\s ', skiprows=1)
df2 = df2.iloc[:-3, :]

df3 = df.merge(df2, how='outer', on='A')
df3[['O','P']] = df3[['O','P']].fillna(-1).astype(int).replace(-1, np.nan)

df3['B']= df3.apply(lambda x: 18 if x['B']==10 and x['O']==1 and x['P']==0 else (
    19 if x['B']==10 and x['O']==1 and x['P']==1 else (
    20 if x['B']==10 and x['O']==0 and x['P']==0 else x['B'])), axis=1)

df3.to_csv('new_tryout.csv')

It finished the code in less than a second, so it is far better than iterrows or itertuples. The new csv file includes both the tryout file and the density file. They are merged together by the first column of tryout file (ID i guess)

I didn't check all of this very big file but from the few random points I checked, it seems as this way works.

  • Related