Spring Batch: fixed length FlatFileItemReader expect a footer line or fail-CodePudding

Having a simple text file with fixed length records and a fixed footer, I want to validate that the footer in fact exists as the last line (in that specific format) and also keep processing in chunks, since the file is very large. If the footer line should not exist, it means the file was incomplete and the job should fail.

I am unable to find a way to do this, any help would be appreciated.

Java Configuration

@Configuration
public class JobConfig {

    @Bean
    public Step exampleLoad(
            StepBuilderFactory stepBuilderFactory) {
        return stepBuilderFactory.get("exampleLoad")
                .<ExampleRecord, ExampleEntity>chunk(5000)
                .reader(ads10Reader())
                .processor(exampleProcessor())
                .writer(exampleWriter())
                .build();
    }

    @Bean
    @StepScope
    public FlatFileItemReader<ExampleRecord> exampleReader() {
        return new FlatFileItemReaderBuilder<ExampleRecord>()
                .name("exampleReader")
                .resource(...)
                .fixedLength()
                .strict(false)
                .columns(new Range(1, 2), new Range(3, 4), new Range(5, 6))
                .names("a", "b", "c")
                .targetType(ExampleRecord.class)
                .build();
    }

    // processor/writers omitted for brevity
}

Example simple text file:

AABBCC
DDEEFF
XXYYZZ
Total number of records: 3

CodePudding user response：

You can't do that in a single step because the footer is at the end of the file. I would do that validation in a prior step with a simple tasklet that counts the number of lines and checks if the count matches with the last line (if that line exists at all). The code should obviously try to get that information without loading the entire file in memory.

To get the line count efficiently, java.nio.file.Files.lines(yourPath).count() should work. To get the last line, check Quickly read the last line of a text file?. With that, you should be able to implement the file validation tasklet. You can use a conditional flow or a decider to decide if the next (pre-validated chunk-oriented) step should run or not.