Used in sparkstreaming kafka directstream interface to get the data, will not be offset to update to the zookeeper, this will lead to the job after the restart can only be read from the latest offset, resulting in the loss of data, in order to avoid this situation, the website suggests that can be manually update their implementation will be offset to the zookeeper, I use a python, but no spark of python interface in Java and scala kafkacluster this class, don't know oneself how to manually, there are many online scala and Java implementation of the operation code, asked master to a python version, online, etc.,! Genuflect is begged!!!!!!!!! The younger brother programming ability is limited, expert patient explanation, genuflect is begged fuels,

CodePudding user response:

Oneself the top, the great god