This is a code snippet.

FtpDownloader.java

ExecutorService pool = Executors.newFixedThreadPool(5);

 for (FTPFile file : files) {
     if (!file.isFile()) continue;
     pool.submit(new FtpFileDownloader(file));
}

FTPFileDownloader.java

public class FtpFileDownloader implements Runnable{
    static Logger logger = LoggerFactory.getLogger(FtpFileDownloader.class);

    private FTPFile file;

    public FtpFileDownloader(FTPFile file) {
        this.file = file;
    }

    private OutputStream outputStream;

    @Override
    public void run() {
        try{
            long start = System.currentTimeMillis();
            String fileName = file.getName();
            logger.info("File is {}", fileName);

            outputStream = new BufferedOutputStream(new FileOutputStream("/home/user/Downloads/"   "FtpDownloads"   "/"   fileName));

            //get the file from the remote system
            ftpClient.retrieveFile(fileName, outputStream);

            showServerReply(ftpClient);
            logger.info("[{}ms, {} processing finished.]",System.currentTimeMillis()-start,fileName);


        }catch (Exception e){
            logger.info("FtpFileDownloader expection");
            e.printStackTrace();
        }finally {
            try {
                //close output stream
                outputStream.close();
            } catch (IOException e) {
                logger.info("Io exception happened");
                e.printStackTrace();
            }
        }

I've created a fixed thread pool of size 5. So after downloading 5 files from the server by each individual server

Thread is not disconnecting from the server even after its file is downloaded and waiting there for FTP server to disconnect

2022-03-01 15:49:33.584  INFO 10931 --- [pool-1-thread-5] t.a.f.listener.ftp.FtpFileDownloader     : File is mail-send-winforms.png
2022-03-01 15:49:33.587  INFO 10931 --- [pool-1-thread-4] t.a.f.listener.ftp.FtpFileDownloader     : File is mail-editor.png
2022-03-01 15:50:33.769  INFO 10931 --- [pool-1-thread-1] t.a.f.listener.ftp.FtpFileDownloader     : FtpFileDownloader expection
2022-03-01 15:50:33.771  INFO 10931 --- [pool-1-thread-1] t.a.f.listener.ftp.FtpFileDownloader     : File is mime-explorer.png
java.net.SocketTimeoutException: Connect timed out
    at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:546)
    at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:597)
    at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327)
    at java.base/java.net.Socket.connect(Socket.java:633)
    at org.apache.commons.net.ftp.FTPClient._openDataConnection_(FTPClient.java:866)
    at org.apache.commons.net.ftp.FTPClient._retrieveFile(FTPClient.java:971)
    at org.apache.commons.net.ftp.FTPClient.retrieveFile(FTPClient.java:3308)
    at tech.adoptnet.ftppractice.listener.ftp.FtpFileDownloader.run(FtpFileDownloader.java:35)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:833)

after downloading the files every thread is waiting until connection time out and then it goes to download other files.

How to make thread resuse itself without connection time out?

CodePudding user response：

I think I got what you're trying to do. First of all, FTP doesn't itself let you use multiple connections simultaneously (For ref Uploading files in parallel with Java FTP client).

What you're doing is not thread-safe as well. So I advise you to make parallel connections and download from there.