Method reads csv input stream differently than csv file-CodePudding

I was tasked with writing a method that would take in a csv file and persist its data into the appropriate space in the database. The function I wrote does so successfully when the csv data is input directly. However, when using cURL and inputting a whole csv file, it does not read the new line delimiters. In effect, the csv then becomes one row with x number of columns, where x is the number of cells in the file. I have tried changing the csv format (e.g. using carriage return vs line feed) but nothing seems to work. Attached is the code that runs through csv, it takes in an InputStream csvData:

CSVReader reader = new CSVReader(new InputStreamReader(csvData));
String[] line;
int bookNum = 1, lineNum = 2; // skip headers
while ((line = reader.readNext()) != null) {
    // map line
    String productCode = line[0]; 
    String author = line[1];
    String description = line[2];
    Integer edition;
    try {
        edition = Integer.parseInt(line[3].replaceAll("[^\\d.]", ""));
    } catch (NumberFormatException nfe) {
        edition = null;
    }
    String copyright = line[4];
    String publisher = line[5];
    BigDecimal listPrice = !line[6].equals("") ? new BigDecimal(line[6]) : null;

    // do stuff with data...

    if (bookNum == 1) System.out.println("1 book has been processed");
    else System.out.println(bookNum   " books have been processed");
      bookNum;
      lineNum;
}

CodePudding user response：

I think there is no way to achieve what you want as the reader.readNext() itself return array represents the columns of your CSV file, and the columns in the array indexed by the order of the columns in your CSV file and you can manipulate the array by previously knowing the columns header so you can get your data correct. Based on this:

line[0] will be your first column / line[1] will be second column and So on.

update adding some code, please try this code instead of yours and tell us what is the result

InputStream inputStream = Files.newInputStream(Paths.get(ClassLoader.getSystemResource("csv/test.csv").toURI()));
    List<String[]> list = new ArrayList<>();
    CSVReader csvReader = new CSVReader(new 
                                     InputStreamReader(inputStream));
    String[] line;
    while ((line = csvReader.readNext()) != null) {
        System.out.println(line[0] " --- " line[1]);
        list.add(line);
    }
    inputStream.close();
    csvReader.close();
    return list;

my file in the class path with the same project and I assumed the data will be as folllow:

colA | colB
 A   | B
 C   | D
 D   | E

the output as follow:

colA --- colB
 A --- B
 C --- D
 E --- F

CodePudding user response：

This works for me. I think the issue is how your input csvData is being formed. Does this work with when you try?

import java.io.ByteArrayInputStream;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.UnsupportedEncodingException;

import com.opencsv.CSVReader;
import com.opencsv.exceptions.CsvValidationException;

public class TestCsv {

    private static final String TEST_CSV = "ISBN,Author,Title,Edition,Copyright,Publisher,Value,Grade\n"
              "1781435460553,1ALTEN,1WORKING WITH AUDIO (PB),12,,1CENGAGE L,0.01,1\n"
              "2781435460553,2ALTEN,1WORKING WITH AUDIO (PB),22,,2CENGAGE L,0.02,2\r\n"
              "3781435460553,3ALTEN,1WORKING WITH AUDIO (PB),32,,3CENGAGE L,0.03,3\r\n";

    public static void main(String[] args) throws CsvValidationException, UnsupportedEncodingException, IOException {
        System.out.println("String Test");
        parseCsv(new ByteArrayInputStream(TEST_CSV.getBytes("UTF-8"))); // from String
        
        System.out.println("\n\n---------------------\nFrom File Test");
         String fileName = "/tmp/test.csv";

            try (FileInputStream fis = new FileInputStream(fileName)) {
                parseCsv(fis);
            }
        
        
        
    }

    public final static void parseCsv(InputStream csvData) throws CsvValidationException, IOException {
        CSVReader reader = new CSVReader(new InputStreamReader(csvData));
        String[] line;
        int bookNum = 1, lineNum = 2; // skip headers
        while ((line = reader.readNext()) != null) {
            // map line
            String productCode = line[0];
            String author = line[1];
            String description = line[2];
            Integer edition;
            try {
                edition = Integer.parseInt(line[3].replaceAll("[^\\d.]", ""));
            } catch (NumberFormatException nfe) {
                edition = null;
            }
            String copyright = line[4];
            String publisher = line[5];
            String listPrice = line[6];

            // do stuff with data...

            if (bookNum == 1) {
                System.out.println("1 book has been processed");
            } else {
                System.out.println(bookNum   " books have been processed");
            }
            
              bookNum;
              lineNum;
        }

    }

}

This also works for me so not quite sure what the issue you are having:

    System.out.println("standard input");
    try {
        parseCsv(System.in);
    } catch (IOException e) {
        // If nothing is passed in
        e.printStackTrace();
    }