Unit test in python is not working despite jupyter notebook working fine-CodePudding

I am having trouble running a test on my .py script which basically just creates a mapping file (takes 3 datasets and merges them). Every time I run the test in the terminal, it fails and I don't know what I am doing wrong because when I run it in jupyter notebook, it works.

Here is what I have (not working):

import unittest
from pandas.util.testing import assert_frame_equal
from create_mapping_file import *


def main():
        PATH_ARCHIVED_ENSEMBLS = 'test-data/original_files/archived_emsembls.txt'
        PATH_ARCHIVED_ACCESSIONS = 'test-data/original_files/archived_accessions.txt'
        PATH_UNIPROT_MAPPING = 'test-data/original_files/uniprot_name_mapping.dat'
        
        actual = create_mapping_df(PATH1, PATH2, PATH3)
        actual = actual.replace("", np.nan, regex = True)
        
      #  actual.to_csv('expected.csv', index = False) 
      # when I include the above line, the test runs fine, but when I run it the second 
      # time around, it stops working

        expected=pd.read_csv("expected.csv")
        assert_frame_equal(expected.reset_index(drop = True), actual.reset_index(drop = True))
    
if __name__ == '__main__':
        main()

Here is the error I am getting. I'm confused for two main reasons: most of the IDs look the exact same--and it claims they are different) There are some IDs that aren't the same and I don't know why they aren't the exact same considering all I did was export the 'actual' and name it 'expected':

DataFrame.iloc[:, 3] (column name="Protein ID") values are different (31.16418 %)
[index]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, ...]
[left]:  [ENSP00000361930, ENSP00000300161, ENSP00000264335, ENSP00000461762, ENSP00000248975, ENSP00000306330, ENSP00000340989, ENSP00000371267, ENSP00000238081, ENSP00000379287, ENSP00000379288, ENSP00000430072, ENSP00000309503, ENSP00000395114, ENSP00000398599, ENSP00000379281, ENSP00000379283, ENSP00000379286, ENSP00000263776, ENSP00000391775, ENSP00000368109, ENSP00000261461, ENSP00000442866, ENSP00000164133, ENSP00000417963, ENSP00000420674, ENSP00000377669, ENSP00000337641, ENSP00000452396, ENSP00000404632, ENSP00000412324, ENSP00000329009, ENSP00000408389, ENSP00000262239, ENSP00000333905, ENSP00000324804, ENSP00000311344, ENSP00000376775, ENSP00000410671, ENSP00000459838, ENSP00000461254, ENSP00000459827, ENSP00000459644, ENSP00000459456, ENSP00000343317, ENSP00000437193, ENSP00000325074, ENSP00000370113, ENSP00000377936, ENSP00000421396, ENSP00000398779, ENSP00000336591, ENSP00000377933, ENSP00000349283, ENSP00000377932, ENSP00000431320, ENSP00000377931, ENSP00000377935, ENSP00000399970, ENSP00000469896, ENSP00000335083, ENSP00000372042, ENSP00000422374, ENSP00000423649, ENSP00000425247, ENSP00000358421, ENSP00000432268, ENSP00000445122, ENSP00000358424, ENSP00000297679, ENSP00000262520, ENSP00000370662, ENSP00000350018, ENSP00000411979, ENSP00000337213, ENSP00000422168, ENSP00000403231, ENSP00000388152, ENSP00000348685, ENSP00000424846, ENSP00000409746, ENSP00000422605, ENSP00000374488, ENSP00000424072, ENSP00000355428, ENSP00000412203, ENSP00000373301, ENSP00000388553, ENSP00000386231, ENSP00000253688, ENSP00000294973, ENSP00000265395, ENSP00000219431, ENSP00000380918, ENSP00000348809, ENSP00000340691, ENSP00000362314, ENSP00000308472, ENSP00000380659, ENSP00000328103, ...]
[right]: [ENSP00000361930, ENSP00000300161, ENSP00000264335, ENSP00000461762, ENSP00000248975, ENSP00000306330, ENSP00000340989, ENSP00000371267, ENSP00000238081, ENSP00000379287, ENSP00000395114, ENSP00000430072, ENSP00000379286, ENSP00000379281, ENSP00000309503, ENSP00000379283, ENSP00000379288, ENSP00000398599, ENSP00000263776, ENSP00000391775, ENSP00000368109, ENSP00000261461, ENSP00000442866, ENSP00000164133, ENSP00000417963, ENSP00000420674, ENSP00000377669, ENSP00000337641, ENSP00000452396, ENSP00000404632, ENSP00000412324, ENSP00000262239, ENSP00000333905, ENSP00000329009, ENSP00000408389, ENSP00000324804, ENSP00000311344, ENSP00000459456, ENSP00000459838, ENSP00000459827, ENSP00000437193, ENSP00000376775, ENSP00000459644, ENSP00000343317, ENSP00000461254, ENSP00000410671, ENSP00000325074, ENSP00000370113, ENSP00000377936, ENSP00000349283, ENSP00000377933, ENSP00000377932, ENSP00000421396, ENSP00000377931, ENSP00000377935, ENSP00000336591, ENSP00000431320, ENSP00000398779, ENSP00000399970, ENSP00000469896, ENSP00000335083, ENSP00000425247, ENSP00000372042, ENSP00000422374, ENSP00000423649, ENSP00000358421, ENSP00000432268, ENSP00000445122, ENSP00000358424, ENSP00000297679, ENSP00000370662, ENSP00000262520, ENSP00000350018, ENSP00000411979, ENSP00000337213, ENSP00000422168, ENSP00000374488, ENSP00000422605, ENSP00000348685, ENSP00000388152, ENSP00000409746, ENSP00000424072, ENSP00000424846, ENSP00000403231, ENSP00000355428, ENSP00000412203, ENSP00000373301, ENSP00000388553, ENSP00000386231, ENSP00000253688, ENSP00000294973, ENSP00000265395, ENSP00000219431, ENSP00000380918, ENSP00000348809, ENSP00000340691, ENSP00000362314, ENSP00000308472, ENSP00000380659, ENSP00000328103, ...]

CodePudding user response：

Solution suggested first was to add a sort step to the test. As they just looked sorted differently, as I pointed out in my comment from Sept 3.

Thinking about it more, you could probably also rely on Python's set math / comparison abilities to check the items included are the same in the test. Consider:

a = ["a","b","c"]
b = ["b","a","c"]

set(a) == set(b)

That results in returning True.

Of course, if the number of repeated items in the list mattered, use of typecasting to a set for the check wouldn't work as it only takes one representative of each item that is duplicated or occurs more than once. Make a list a set and then casting back to a list (if that is indeed the type needed) is a very common idiom in Python to reduce a list to the unique elements.