I am trying to populate an H2 database with values from a CSV file, like this:
@Component
public class DBWriterOrder implements ItemWriter<OrderEntity> {
private OrderRepository orderRepository;
@Autowired
public DBWriterOrder(OrderRepository orderRepository) {
this.orderRepository = orderRepository;
}
@Override
public void write(List<? extends OrderEntity> orders) throws Exception {
System.out.println("Data Saved for Orders: " orders);
orderRepository.saveAll(orders);
}
}
@Component
public class ProcessorOrder implements ItemProcessor<OrderEntity, OrderEntity> {
public SimpleDateFormat sdf = new SimpleDateFormat("dd-MM-yyyy");
@Override
public OrderEntity process(OrderEntity orderEntity) throws Exception {
Date deliveryDate = sdf.parse(orderEntity.getDeliveryDate().toString());
long deliveryDateInMillis = deliveryDate.getTime();
orderEntity.setDeliveryDate(deliveryDateInMillis);
Date lastUpdated = sdf.parse(orderEntity.getLastUpdated().toString());
long lastUpdatedInMillis = lastUpdated.getTime();
orderEntity.setLastUpdated(lastUpdatedInMillis);
return orderEntity;
}
}
@Configuration
@EnableBatchProcessing
public class SpringBatchConfigOrder {
@Bean
public Job jobOrder(JobBuilderFactory jobBuilderFactory,
StepBuilderFactory stepBuilderFactory,
ItemReader<OrderEntity> itemReader,
ItemProcessor<OrderEntity, OrderEntity> itemProcessor,
ItemWriter<OrderEntity> itemWriter
) {
Step step = stepBuilderFactory.get("ETL-file-load")
.<OrderEntity, OrderEntity>chunk(100)
.reader(itemReader)
.processor(itemProcessor)
.writer(itemWriter)
.build();
return jobBuilderFactory.get("ETL-Load")
.incrementer(new RunIdIncrementer())
.start(step)
.build();
}
@Bean
public FlatFileItemReader<OrderEntity> itemReaderOrder() {
FlatFileItemReader<OrderEntity> flatFileItemReader = new FlatFileItemReader<>();
flatFileItemReader.setResource(new FileSystemResource("src/main/resources/orders.csv"));
flatFileItemReader.setName("CSV-Reader");
flatFileItemReader.setLinesToSkip(1);
flatFileItemReader.setLineMapper(lineMapperOrder());
return flatFileItemReader;
}
@Bean
public LineMapper<OrderEntity> lineMapperOrder() {
DefaultLineMapper<OrderEntity> defaultLineMapper = new DefaultLineMapper<>();
DelimitedLineTokenizer lineTokenizer = new DelimitedLineTokenizer();
lineTokenizer.setDelimiter(",");
lineTokenizer.setStrict(false);
lineTokenizer.setNames("id","destination","deliveryDate","statusOrder","lastUpdated");
BeanWrapperFieldSetMapper<OrderEntity> fieldSetMapper = new BeanWrapperFieldSetMapper<>();
fieldSetMapper.setTargetType(OrderEntity.class);
defaultLineMapper.setLineTokenizer(lineTokenizer);
defaultLineMapper.setFieldSetMapper(fieldSetMapper);
return defaultLineMapper;
}
}
@RestController
@RequestMapping("/loadOrder")
public class OrderLoadController {
@Autowired
JobLauncher jobLauncherOrder;
@Autowired
Job jobOrder;
@GetMapping
public BatchStatus load() throws JobParametersInvalidException, JobExecutionAlreadyRunningException, JobRestartException, JobInstanceAlreadyCompleteException {
Map<String, JobParameter> maps = new HashMap<>();
maps.put("time", new JobParameter(System.currentTimeMillis()));
JobParameters parameters = new JobParameters(maps);
JobExecution jobExecution = jobLauncherOrder.run(jobOrder, parameters);
System.out.println("JobExecution: " jobExecution.getStatus());
System.out.println("Batch is Running...");
while (jobExecution.isRunning()) {
System.out.println("...");
}
return jobExecution.getStatus();
}
}
Also this is my entity class:
@Entity(name = "orders")
@Data
public class OrderEntity {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
@ManyToOne(fetch = FetchType.EAGER)
private DestinationEntity destination;
private Long deliveryDate;
@Enumerated(value = EnumType.STRING)
private OrderStatus statusOrder;
private Long lastUpdated;
}
And this is my CSV file:
id,destination,deliveryDate,statusOrder,lastUpdated
1,Ploiesti,15-12-2021,NEW,15-12-2021
2,Ploiesti,15-12-2021,NEW,15-12-2021
3,Pitesti,15-12-2021,NEW,15-12-2021
4,Pitesti,15-12-2021,NEW,15-12-2021
5,Pitesti,15-12-2021,NEW,15-12-2021
When I call the endpoint localhost:8082/loadController, my DB instead of being populated it remains empty and all I got is this error in the console:
org.springframework.batch.item.file.FlatFileParseException: Parsing error at line: 2 in resource=[file [C:\Users\ALEX\Desktop\FinalProject\demo\src\main\resources\orders.csv]], input=[1,Ploiesti,15-12-2021,NEW,15-12-2021]
at org.springframework.batch.item.file.FlatFileItemReader.doRead(FlatFileItemReader.java:189) ~[spring-batch-infrastructure-4.3.4.jar:4.3.4]
Caused by: org.springframework.validation.BindException:
org.springframework.validation.BeanPropertyBindingResult: 3 errors
Field error in object 'target' on field 'lastUpdated': rejected value [15-12-2021]; codes [typeMismatch.target.lastUpdated,typeMismatch.lastUpdated,typeMismatch.java.lang.Long,typeMismatch]; arguments [org.springframework.context.support.DefaultMessageSourceResolvable: codes [target.lastUpdated,lastUpdated]; arguments []; default message [lastUpdated]]; default message [Failed to convert property value of type 'java.lang.String' to required type 'java.lang.Long' for property 'lastUpdated'; nested exception is java.lang.NumberFormatException: For input string: "15-12-2021"]
Field error in object 'target' on field 'destination': rejected value [Ploiesti]; codes [typeMismatch.target.destination,typeMismatch.destination,typeMismatch.com.example.demo.destination.DestinationEntity,typeMismatch]; arguments [org.springframework.context.support.DefaultMessageSourceResolvable: codes [target.destination,destination]; arguments []; default message [destination]]; default message [Failed to convert property value of type 'java.lang.String' to required type 'com.example.demo.destination.DestinationEntity' for property 'destination'; nested exception is java.lang.IllegalStateException: Cannot convert value of type 'java.lang.String' to required type 'com.example.demo.destination.DestinationEntity' for property 'destination': no matching editors or conversion strategy found]
Field error in object 'target' on field 'deliveryDate': rejected value [15-12-2021]; codes [typeMismatch.target.deliveryDate,typeMismatch.deliveryDate,typeMismatch.java.lang.Long,typeMismatch]; arguments [org.springframework.context.support.DefaultMessageSourceResolvable: codes [target.deliveryDate,deliveryDate]; arguments []; default message [deliveryDate]]; default message [Failed to convert property value of type 'java.lang.String' to required type 'java.lang.Long' for property 'deliveryDate'; nested exception is java.lang.NumberFormatException: For input string: "15-12-2021"]
In the end, my question is what should I do and how should I do to get things done?
CodePudding user response：
Looks like you need a custom FieldSetMapper because you need not only convert String into Date and then into Long but also lookup DestinationEntity by its name. Here is an example that converts String into Date
public class PersonFieldSetMapper implements FieldSetMapper<Person> {
@Override
public Person mapFieldSet(FieldSet fieldSet) throws BindException {
return new Person(fieldSet.readLong("id"),
fieldSet.readString("firstName"),
fieldSet.readString("lastName"),
fieldSet.readDate("birthdate", "yyyy-MM-dd HH:mm:ss"));
}
}
Example taken from here https://www.dineshonjava.com/spring-batch-read-from-csv-and-write-to-relational-db/
You also need to somehow add a Map<String, DestinationEntity> property to this FieldSetMapper to lookup DestinationEntity
CodePudding user response：
You can use FieldSetMapper like below and replace Mapper from your code. Use index based reads for each field. Like id will be 0 index
public class OrderEntityMapper implements FieldSetMapper<OrderEntity> {
@Override
public OrderEntity mapFieldSet(FieldSet fieldSet) throws BindException {
OrderEntity order=new OrderEntity();
order.setId(fieldSet.readLong(0));
// other fields
return order;
}
Then replace BeanWrapperFieldSetMapper<OrderEntity> fieldSetMapper = new BeanWrapperFieldSetMapper<>() like
@Bean
public LineMapper<OrderEntity> lineMapperOrder() {
DefaultLineMapper<OrderEntity> defaultLineMapper = new DefaultLineMapper<>();
DelimitedLineTokenizer lineTokenizer = new DelimitedLineTokenizer();
lineTokenizer.setDelimiter(",");
lineTokenizer.setStrict(false);
lineTokenizer.setNames("id","destination","deliveryDate","statusOrder","lastUpdated");
defaultLineMapper.setLineTokenizer(lineTokenizer);
defaultLineMapper.setFieldSetMapper(new OrderEntityMapper());
return defaultLineMapper;
}
}