I have existing Spring Boot application that uses Camel framework for getting data from different folders. All routes poll data in similar way:

from(fileUriWithCurrentDay(path, config.getParams())).routeId(ROUTE_ID)

where method defined as follows:

public static String fileUriWithCurrentDay(String path, Map<String, String> params) {
path = path.endsWith(SEPARATOR) ? path : path   SEPARATOR;
return "file:"   path   currentDayPath()   fileParams(params);
}

and returns something like depending on the day:

c:/test/data/2021/242?noop=true&idempotent=false&autoCreate=false&includeExt=xml&scheduler=spring&scheduler.cron = "0 0 13 ? * *"

The cron is set to start one time per day. When cron time is reached the route starts to perform its job and finishes it correctly. But on the next day it starts to poll the same folder, e.g. 242 but it should be 243.

So the problem is: from endpoint should be calculated dynamically at its start.

I've read about dynamic from: in Camel, but didn't find any information that such option is supported out of the box. I've tried to use pollEnrich() option using header that is calculated, but it seem doesn't support dynamic endpoints as returns next error:

org.apache.camel.ResolveEndpointFailedException Dynamic expressions with ${ } placeholders is not allowed. Use the fileName option to set the dynamic expression.

How from endpoint can be refreshed after route is started?

I also see 2 possible solutions but didn't know will it help:

1. Restart routes for recalculating from endpoint uri.
2. Restart whole Camel context. Currently all routes are marked as Spring @Component.

CodePudding user response：

Unlike other EIP component, string in routeId and from endpoint are fixed once the route is created (unless you recreate the route with another from endpoint url). Thus, 2 possible direction to move on. One way is to recreate the route with new from endpoint url and another is to lookup parameter in the component to support your action.

1. Recreate consumer route with same routeId in same CamelContext

   • Trigger below action daily (e.g. use timer route)
   • Use RouteBuilder to build route config (with same consumer route id and new from endpoint url)
   • recreate route using addRoutes method of CamelContext class with new route config

2. Leverage filterDirectory in file component

   • Set file path to common root folder (i.e. c:/test/data)
   • Enable recursive to lookup sub-directories
   • Use filterDirectory to filter directory base on simple language
     • From simple language documentation, date command of simple language is using java.text.SimpleDateFormat
     • From Oracle doc, SimpleDateFormat do support both year and Day in year (assume 242 and 243 in your example are day in year)

Disclaimer

I did not use filterDirectory before, way 2 is deduce from Camel documentation.

CodePudding user response：

Looking at the documentation for the File component what you are trying to do is not possible purely with the from-endpoint URI.

I would suggest using a common base directory in the URI and creating an implementation of GenericFileFilter that you'd use with the from-endpoint. It allows for a lot of freedom in implementing the logic for which files are and which aren't processed.

To read files from base/path/2021 create something like this:

@Component
public class FileFilter<T> implements GenericFileFilter<T> {
    @Override
    public boolean accept(GenericFile<T> file) {
        if (file.isDirectory()) {
            return "2021".equals(file.getFileName());
        }
        return file.getFileName().startsWith("2021/");
    }
}

with endpoint URI like this:

file://base/path?recursive=true&filter=#fileFilter

When implementing the GenericFileFilter you should be mindful that accept() will be called for the directories as well.