Home > database >  delayed while creating data because it fetched a lot of data beforehand and had to be saved
delayed while creating data because it fetched a lot of data beforehand and had to be saved

Time:07-19

I want to create data from a copy of the previous data and save it with the new data.

@Entity
@Table(name = "tableA")
@Getter
@Setter
public class TableA {

    @Id
    @GeneratedValue(strategy = javax.persistence.GenerationType.IDENTITY)
    private Long id; 

    private String status;
    
    private int revision;

    @OneToOne(fetch = FetchType.LAZY, mappedBy = "tableA")
    @PrimaryKeyJoinColumn
    @JsonIgnoreProperties("phMt")
    private Field1 field1;

    @OneToOne(fetch = FetchType.LAZY, cascade = CascadeType.ALL, mappedBy = "tableA")
    @PrimaryKeyJoinColumn
    @JsonIgnore
    private Field2 field2;

    @OneToOne(fetch = FetchType.LAZY, cascade = CascadeType.ALL, mappedBy = "tableA")
    @PrimaryKeyJoinColumn
    @JsonIgnore
    private Field3 field3;

    @OneToMany(fetch = FetchType.LAZY, mappedBy = "tableA", orphanRemoval = true)
    @JsonIgnore
    private List<Field4> field4;

    @OneToOne(fetch = FetchType.LAZY, cascade = CascadeType.ALL, mappedBy = "tableA")
    @PrimaryKeyJoinColumn
    @JsonIgnore
    private Field5 field5;

    @OneToMany(fetch = FetchType.LAZY, mappedBy = "tableA", orphanRemoval = true)
    @JsonIgnore
    private List<Field6> field6;

    @OneToOne(fetch = FetchType.LAZY, cascade = CascadeType.ALL, mappedBy = "tableA")
    @PrimaryKeyJoinColumn
    @JsonIgnore
    private Field7 field7;

    @OneToOne(fetch = FetchType.LAZY, mappedBy = "tableA")
    @JsonIgnore
    private Field8 field8;

    @OneToOne(fetch = FetchType.LAZY, cascade = CascadeType.ALL, mappedBy = "tableA")
    @PrimaryKeyJoinColumn
    @JsonIgnore
    private Field9 field9;

    @OneToOne(fetch = FetchType.LAZY, cascade = CascadeType.ALL, mappedBy = "tableA")
    @PrimaryKeyJoinColumn
    @JsonIgnore
    private Field10 field10;

    @OneToMany(fetch = FetchType.LAZY, mappedBy = "tableA", orphanRemoval = true)
    @JsonIgnore
    private List<Field11> field11;

    @OneToMany(fetch = FetchType.LAZY, mappedBy = "tableA", orphanRemoval = true)
    @JsonIgnore
    private List<Field12> field12;

    @OneToMany(fetch = FetchType.LAZY, mappedBy = "tableA", orphanRemoval = true)
    @JsonIgnore
    private List<Field13> field13;

}

For example I want to create data in table A with revision 1, what I want is when the data is created automatically copy data from revision 0 that was previously in tableA. Here's a function that I created to create new data that is the same as the data in revision 0.

@Transactional
@Override
public void create(String status, String revision) {
    TableA tableA = new TableA();
    tableA.setStatus(status);

    if (status.equals("Revision")) {
        tableA.setRevision(revision);

        //Clone Field 2
        Field2 field2Prev = tableAPrev.getField2();
        Field2 field2Next = new Field2();
        BeanUtils.copyProperties(field2Prev, field2Next, "id");
        field2Next.setTableA(tableANext);
        tableANext.setField2(field2Next);
        save(tableANext);

        //Clone Field 3
        Field3 field3Prev = tableAPrev.getField3();
        Field3 field3Next = new Field3();
        BeanUtils.copyProperties(field3Prev, field3Next, "id");
        field3Next.setTableA(tableANext);
        tableANext.setField3(field3Next);
        save(tableANext);

        //Clone Field 4
        List<Field4> field4Prev = tableAPrev.getField4();
        List<Field4> field4Next = new ArrayList<>();
        if (!field4Prev.isEmpty()) {
            for (int i = 0; i < field4Prev.size(); i  ) {
                Field4 field4 = new Field4();
                BeanUtils.copyProperties(field4Prev.get(i), field4, "id", "tableA");
                field4.setTableA(tableANext);
                field4Next.add(field4);
            }
            field4Repository.saveAll(field4Next);
        }

        //Clone Field 5
        Field5 field5Prev = tableAPrev.getField5();
        Field5 field5Next = new Field5();
        BeanUtils.copyProperties(field5Prev, field5Next, "id");
        field5Next.setTableA(tableANext);
        tableANext.setField5(field5Next);
        save(tableANext);

        //Clone Field 6
        List<Field6> field6Prev = tableAPrev.getField6();
        List<Field6> field6Next = new ArrayList<>();
        if (!field6Prev.isEmpty()) {
            for (int i = 0; i < field6Prev.size(); i  ) {
                Field6 field6 = new Field6();
                BeanUtils.copyProperties(field6Prev.get(i), field6, "id", "tableA");
                field6.setTableA(tableANext);
                field6Next.add(field6);
            }
            field6Repository.saveAll(field6Next);
        }

        //Clone Field 7
        Field7 field7Prev = tableAPrev.getField7();
        Field7 field7Next = new Field7();
        BeanUtils.copyProperties(field7Prev, field7Next, "id");
        field7Next.setTableA(tableANext);
        tableANext.setField7(field7Next);
        save(tableANext);

        //Clone Field 8
        Field8 field8Prev = tableAPrev.getField8();
        if (field8Prev != null) {
            Field8 field8Next = new Field8();
            List<Field8A> field8ANext = new ArrayList<>();
            BeanUtils.copyProperties(field8Prev, field8Next, "id", "tableA", "dataChild8A");
            field8Next.setTableA(tableANext);
            field8Next.setDataChild8A(field8ANext);
            field8Repository.save(field8Next);

            List<Field8A> field8APrev = field8Prev.getDataChild8A();
            if (!field8APrev.isEmpty()) {
                for (int i = 0; i < field8APrev.size(); i  ) {
                    Field8A field8A = new Field8A();
                    BeanUtils.copyProperties(field8APrev, field8A, "id", "field8");
                    field8A.setField8(field8Next);
                    field8ANext.add(field8A);
                }
                field8ARepository.saveAll(field8ANext);
            }
        }

        // etc until Field 13

    } 
    save(tableA);
}

But when the API is hit it doesn't show a response and is delayed. And the process takes up a lot of memory. What I want to ask is how efficient it is to be able to copy a lot of data and save it perfectly?

CodePudding user response:

What I want to ask is how efficient it is to be able to copy a lot of data and save it perfectly?

As you noted this is not very efficient for large amounts of data. The first to improve performance could be to break it into multiple transactions. Due to JPAs caching behavior it will keep all your changes in the EntityManager which might cause high memory consumption and bad performance.

But really JPA is the wrong tool for this kind of work in the first place.

I recommend using straight SQL through a JDBC template. Using INSERT INTO SELECT statements you should be able to perform the same actions in a few percent of the time you need using JPA, because you are not loading all the data into memory, but keeping it in the database all the time.

CodePudding user response:

  • You should remove all the intermediate saves and replace them by only one in the end of the method

    some thing like this :

    save(tableANext); save(tableA);

  • Related