I am trying to read data from multiple csv file present in resources/input/user_*.csv and writing in one csv file present in resources/output/user.csv

Below is my Config File

@Configuration
@EnableBatchProcessing
public class BatchConfig {

    private static final Logger log = LoggerFactory.getLogger(BatchConfig.class);

    @Autowired
    private StepBuilderFactory stepBuilderFactory;

    @Autowired
    private JobBuilderFactory jobBuilderFactory;

    @Value(value = "classpath:input/user_*.csv")
    private Resource[] resources;

    @Bean
    public FlatFileItemReader<User> flatFileItemReader(){
        log.info("flatFileItemReader execution started");
        FlatFileItemReader<User> reader = new FlatFileItemReader<>();
        reader.setName("ReadMultipleCsv");
        reader.setLineMapper(lineMapper());
        log.info("flatFileItemReader execution completed");
        return reader;
    }

    private LineMapper<User> lineMapper(){
        DefaultLineMapper<User> lineMapper = new DefaultLineMapper<>();

        DelimitedLineTokenizer lineTokenizer = new DelimitedLineTokenizer();
        lineTokenizer.setDelimiter(",");
        lineTokenizer.setStrict(false);
        lineTokenizer.setNames(new String[] {"id","name","department","salary"});

        BeanWrapperFieldSetMapper<User> fieldSetMapper = new BeanWrapperFieldSetMapper<>();
        fieldSetMapper.setTargetType(User.class);

        lineMapper.setLineTokenizer(lineTokenizer);
        lineMapper.setFieldSetMapper(fieldSetMapper);
        return lineMapper;
    }

    @Bean
    public MultiResourceItemReader<User> multiResourceItemReader(){
        log.info("multiResourceItemReader() execution started");
        MultiResourceItemReader<User> reader = new MultiResourceItemReader<>();
        reader.setResources(resources);
        reader.setDelegate(flatFileItemReader());
        log.info("multiResourceItemReader() execution completed");
        return reader;
    }

    @Bean
    public FlatFileItemWriter<User> writer(){
        log.info("writer() execution started");
        FlatFileItemWriter<User> writer = new FlatFileItemWriter<>();
        writer.setResource(new ClassPathResource("output/user.csv"));
        writer.setAppendAllowed(true);

        DelimitedLineAggregator<User> delimitedLineAggregator = new DelimitedLineAggregator<>();
        delimitedLineAggregator.setDelimiter(",");
        BeanWrapperFieldExtractor<User> beanWrapperFieldExtractor = new BeanWrapperFieldExtractor<>();
        beanWrapperFieldExtractor.setNames(new String[] {"id","name","department","salary"});
        delimitedLineAggregator.setFieldExtractor(beanWrapperFieldExtractor);
        writer.setLineAggregator(delimitedLineAggregator);
        log.info("writer() execution completed");
        return writer;
    }

    @Bean
    public Step step(){
        return stepBuilderFactory.get("get-student").<User, User>chunk(5)
                .reader(multiResourceItemReader())
                .writer(writer()).build();
    }

    @Bean
    public Job job(){
        return jobBuilderFactory.get("process-student").incrementer(new RunIdIncrementer())
                .flow(step()).end().build();
    }
}

And this is my user entity

@Getter
@Setter
@AllArgsConstructor
@NoArgsConstructor
public class User {

    private String id;
    private String name;
    private String department;
    private String salary;
}

When I run this code, I don't get error, but I don't see the data in output/user.csv file I don't what's wrong, can anyone help me ? Thanks

CodePudding user response：

The problem is in the writer you have to use new FileSystemResource("src/main/resources/output/user.csv") instead of new ClassPathResource("output/user.csv"). With this you also don't have to worry to create the file first, it will create it if not exists.

 @Bean
 public FlatFileItemWriter<User> writer(){
    log.info("writer() execution started");
    FlatFileItemWriter<User> writer = new FlatFileItemWriter<>();
    writer.setResource(new FileSystemResource("src/main/resources/output/user.csv"));
    writer.setAppendAllowed(true);
    DelimitedLineAggregator<User> delimitedLineAggregator = new DelimitedLineAggregator<>();
    delimitedLineAggregator.setDelimiter(",");
    BeanWrapperFieldExtractor<User> beanWrapperFieldExtractor = new BeanWrapperFieldExtractor<>();
    beanWrapperFieldExtractor.setNames(new String[] {"id","name","department","salary"});
    delimitedLineAggregator.setFieldExtractor(beanWrapperFieldExtractor);
    writer.setLineAggregator(delimitedLineAggregator);
    log.info("writer() execution completed");
    return writer;
}