Converting list of model entities with relationships to DTO takes time-CodePudding

I'm using Spring boot JPA to get list of objects (Using Java 8 now). Each object has relationships and I use the related objects also to transform to a dto list. Let's say I have below model classes.

public class Product {

    @EmbeddedId
    private ProductId id;

    @ManyToOne(fetch = FetchType.LAZY)
    @JoinColumn(name = "userid", referencedColumnName = "USER_ID")
    @MapsId("userId")
    private User owner;
}

public class User {

    @Id
    @Column(name = "USER_ID")
    private Long userId;
    
    @OneToOne(fetch = FetchType.LAZY)
    @JoinColumn(name = "gp_code", referencedColumnName = "GP_CODE")
    @JoinColumn(name = "userid", referencedColumnName = "USER_ID")
    private UserGroup userGroup;
    
    @OneToOne(fetch = FetchType.LAZY)
    @JoinColumnsOrFormulas(value = {
            @JoinColumnOrFormula(formula = @JoinFormula(value = "country_id", referencedColumnName = "COUNTRY_ID")),
            @JoinColumnOrFormula(column = @JoinColumn(name = "region_code", referencedColumnName = "REGION_CODE")) })
    private Country country;
}

I do query for List<Product> and using stream I'm converting it into a dto object. During which I call the related entity to get the data. I have the below code and works fine unless the list is too much. If I have 1000 items in the list it takes around 30 seconds.

I believe because of lazy loading this is happening. What is the best way to optimize this? One option is to do pagination, but I cannot do it. I need all results together.

Is there any way to parallelly execute this? I tried to call parellelStream() instead of stream(), but it's same result.

public List<ProductDTO> getProducts(String countryId) {
    //List<Product> products = Query Result
    List<ProductDTO> productsList = products.stream().filter(isOwnerFromCountryAndInAnyGroup(countryId))
                    .map(product -> getProductDTO(product)).collect(Collectors.toList());
}

private Predicate<? super Product> isOwnerFromCountryAndInAnyGroup(String countryId) {
        return product -> {
            User user = product.getOwner();
            return null != user && null != user.getCountry()
                    && user.getCountry().getCountryId().equals(countryId) && (null != user.getUserGroup());
        };
    }
    
private ProductDTO getProductDTO(Product product) {
        ProductDTO productDTO = new ProductDTO();
        productDTO.setProductNbr(product.getId().getProductNbr());
        productDTO.setPrice(product.getPrice());
        productDTO.setOwnerName(product.getOwner().getName());
        return productDTO;
    }

Edit

I missed to add the line productDTO.setOwnerName(product.getOwner().getName()); for the purpose of asking question here. With query or using filter I'm getting the correct number of results. And with lazy loading, query returns faster and then while calling getOwner() for each row, the process takes time (30 seconds). And with FethType.EAGER, the query takes similar time(30 seconds) and then processes faster. Either way it is similar time.

To fasten the process, is there any way to execute the stream code block in parallel and collect all results together in list?

CodePudding user response：

public List<ProductDTO> getProducts(String countryId) {
    //List<Product> products = Query Result
    List<ProductDTO> productsList = products.stream().filter(isOwnerFromCountryAndInAnyGroup(countryId))
                    .map(product -> getProductDTO(product)).collect(Collectors.toList());
}

From your use case here I am pretty confident that it is not the creation of DTO that takes time. It is that you retrieve a huge set from database (even the complete table of Products) and then you filter for a relation with a specific country just from java.

So Step1 optimization:

If you want to filter for products that are associated with a user from a specific country then this can go on JPA level and translated in optimal way in database. Then the allocation of resources (memory, cpu) would be much more optimal, instead of your java application trying to load a huge data set and filter it there.

@Query("SELECT p FROM Product p where p.owner IS NOT NULL AND p.owner.userGroup IS NOT NULL AND p.owner.country IS NOT NULL AND p.owner.country.id = :countryId")
List<Product> findProductRelatedWithUserFromCountry(@Param String countryId);

and remove the filtering from your method getProducts.

Step2 optimization:

In addition to the above, not only you can pass the java filtering in the database query by moving it to JPA layer but you can also optimize the query a bit more by defining in JPA that you want to load the associated Owner as well so that it doesn't hit later the database to retrieve it when you create the DTO. You can achieve this with join fetch, so your query should now become:

@Query("SELECT p FROM Product p JOIN FETCH p.owner own where p.owner IS NOT NULL AND own.userGroup IS NOT NULL AND own.country IS NOT NULL AND own.country.id = :countryId")
List<Product> findProductRelatedWithUserFromCountry(@Param String countryId);

Step3 optimization:

If we want to take it an extra step further it seems that most times using DTO projections would speed up the execution. This can happen as the query would define only specific information it needs to retrieve and convert into DTO instead of the complete entities.

So your query now would be:

@Query("SELECT new org.your.package.where.dto.is.ProductDTO(p.id.productNbr, p.price, own.name) FROM Product p JOIN FETCH p.owner own where p.owner IS NOT NULL AND own.userGroup IS NOT NULL AND own.country IS NOT NULL AND own.country.id = :countryId")
List<ProductDTO> findProductRelatedWithUserFromCountry(@Param String countryId);

Also remember to have the DTO constructor used in the JPA query available in your ProductDTO.class.