pytest fixtures: testing pandas dataframe-CodePudding

I have some scripts in package directory and some tests in tests directory, along with a CSV file containing a dataframe that i want to use for testing purposes.

main_directory/
  |
  |- package/
  |   |- foo.py
  |   |- bar.py
  | 
  |- tests/
      |- conftest.py
      |- test1.py
      |- test.csv

I am using pytest and i have defined a conftest.py that contains a fixture that i want to use for the whole test session, that should return a pandas test dataframe imported from a csv file, as in the following:

#conftest.py

import pytest
from pandas import read_csv

path="test.csv"

@pytest.fixture(scope="session")
def test_data():
    return read_csv(path)

I have been trying to use the fixture to return the test dataframe for the test_functions.

The original test functions were a bit more complex, calling pandas groupby on the object returned by the fixture. I kept on getting the error 'TestStrataFrame' object has no attribute 'groupby' so i simplified the test to the test below and, as I was still getting errors, I realized that i am probably missing something.

My test is the following:

#test1.py
import unittest
import pytest

class TestStrataFrame(unittest.TestCase):

    def test_fixture(test_data):
        assert isinstance(test_data,pd.DataFrame) is True

The above test_fixture returns:

=============================================== FAILURES ================================================
_____________________________________ TestStrataFrame.test_fixture ______________________________________

test_data = <tests.test_data.TestStrataFrame testMethod=test_fixture>

    def test_fixture(test_data):
        ciao=test_data
>       assert isinstance(ciao,pd.DataFrame) is True
E       AssertionError: assert False is True
E           where False = isinstance(<tests.test_data.TestStrataFrame testMethod=test_fixture>, <class 'pandas.core.frame.DataFrame'>)
E             where <class 'pandas.core.frame.DataFrame'> = pd.DataFrame

tests/test_data.py:23: AssertionError
=========================================== warnings summary ============================================
../../../../../opt/miniconda3/envs/geo/lib/python3.7/importlib/_bootstrap.py:219
  /opt/miniconda3/envs/geo/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject
    return f(*args, **kwds)

-- Docs: https://docs.pytest.org/en/stable/warnings.html
======================================== short test summary info ========================================
FAILED tests/test_data.py::TestStrataFrame::test_fixture - AssertionError: assert False is True
================================ 1 failed, 4 passed, 1 warning in 12.82s ================================

How can i do this correctly?

PS : At the moment i would not focus on the RuntimeWarning. I am getting since after I have started trying to solve this issue, but i am quite sure the tests were failing even before I got that warning - so they are probably unrelated. I reinstalled the environment and the warning persists, hopefully might go away with solving the issue...

CodePudding user response：

The error is coming as test_data is not been passed to test_fixture method. for example below are two ways you can tweak your Class and its method.

import unittest
import pytest
import pandas as pd

class TestStrataFrame(unittest.TestCase):
    test_data=pd.DataFrame()
    def test_fixture(self):
        test_data=pd.DataFrame()
        assert isinstance(test_data,pd.DataFrame) is True
    def test_fixture_1(self):
        assert isinstance(TestStrataFrame.test_data,pd.DataFrame) is True

and run from terminal : pytest test_sample.py

CodePudding user response：

This is the expected behavior if you take note of this page here. That page clearly states:

The following pytest features do not work, and probably never will due to different design philosophies:

 1. Fixtures (except for autouse fixtures, see below);
 2. Parametrization;
 3. Custom hooks;

You can modify your code to the following to work.

# conftest.py
from pathlib import Path
import pytest
from pandas import read_csv

CWD = Path(__file__).resolve()
FIN = CWD.parent / "test.csv"

@pytest.fixture(scope="class")
def test_data(request):
    request.cls.test_data = read_csv(FIN)


# test_file.py
import unittest
import pytest
import pandas as pd

@pytest.mark.usefixtures("test_data")
class TestStrataFrame(unittest.TestCase):
    def test_fixture(self):
        assert hasattr(self, "test_data")
        assert isinstance(self.test_data, pd.DataFrame)

==>pytest tests/
============================= test session starts ==============================
platform darwin -- Python 3.9.1, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /Users/***/Desktop/scripts/stackoverflow
collected 1 item                                                               

tests/test_file.py .                                                     [100%]

============================== 1 passed in 0.03s ===============================

You can see more about mixing fixtures with the unittest framework here.