I recently met a problem, I wrote a selenium crawler in python, automatic processing tasks, now on the four Windows server,
I update the crawler file is to use remote desktop connection, copy and paste in the new code, and then close the crawlers are running, then open a new replacement,
Ok now, after much server, this manual maintenance always don't feel quite right,
I ask what's the good way? How can automatically update the code and automatic restart after the update crawler python program? What tools can use? Thank you very much!

CodePudding user response:

Git automatic deployment
A key to the script start

CodePudding user response:

Because it is a way

CodePudding user response:

Write a management system, docking a ready-made message platform, connect the crawler, the management system management system, including creeper code management system and task management system,
Message platform can use XMPP server, crawler read code assigned to it, tasks, after completion of the task the results back to the management platform

Then search, found to have a ready-made system
https://juejin.im/post/5c7f9014f265da2d9d1cd34b