how can I make a kafka rest api call when the key is unquoted?-CodePudding

I'm trying to get kafka messages to an android app.

I realized a little late in the game that the only way to do that is with Kafka Rest Proxy.

I've set it all up but my key is unquoted leading to an error in deserialization.

with /bin/kafka-console-consumer --bootstrap-server localhost:9092 --topic my_topic --property print.key=true --from-beginning

I receive truck/6084/position {"timestamp":1636987919544,"truckId":6084,"driverId":36,"routeId":1177752383,"eventType":"Normal","correlationId":"597220341814695046","latitude":24.7700961485576,"longitude":46.6207663588065}

I have these messages already running to a PC application without any issues via a typical consumer almost line for line from confluent.

my exact error is

{"error_code":40801,"message":"com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'truck': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')\n at [Source: (byte[])\"truck/7103/position\"; line: 1, column: 7]"}

when trying to get with "jsonschema"

{"error_code":40801,"message":"Error deserializing key/value for partition vehicle_tracking_sysA-5 at offset 0. If needed, please seek past the record to continue consumption."}

Any insight would be greatly appreciated.

Chris

EDIT:

Autoconnected Player Kafka - Received Expection: Sequence contains no matching element trace:   at System.Linq.Enumerable.Single[TSource] (System.Collections.Generic.IEnumerable`1[T] source, System.Func`2[T,TResult] predicate) [0x00000] in <00000000000000000000000000000000>:0 
  at Confluent.Kafka.Impl.Librdkafka.SetDelegates (System.Type nativeMethodsClass) [0x00000] in <00000000000000000000000000000000>:0 
  at Confluent.Kafka.Impl.Librdkafka.TrySetDelegates (System.Collections.Generic.List`1[T] nativeMethodCandidateTypes) [0x00000] in <00000000000000000000000000000000>:0 
  at Confluent.Kafka.Impl.Librdkafka.LoadLinuxDelegates (System.String userSpecifiedPath) [0x00000] in <00000000000000000000000000000000>:0 
  at Confluent.Kafka.Impl.Librdkafka.Initialize (System.String userSpecifiedPath) [0x00000] in <00000000000000000000000000000000>:0 
  at Confluent.Kafka.Consumer`2[TKey,TValue]..ctor (Confluent.Kafka.ConsumerBuilder`2[TKey,TValue] builder) [0x00000] in <00000000000000000000000000000000>:0 
  at Confluent.Kafka.ConsumerBuilder`2[TKey,TValue].Build () [0x00000] in <00000000000000000000000000000000>:0 
  at KafkaConsumer threadHandle.StartKafkaListener () [0x00000] in <00000000000000000000000000000000>:0 
  at System.Threading.ThreadHelper.ThreadStart_Context (System.Object state) [0x00000] in <00000000000000000000000000000000>:0 
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <00000000000000000000000000000000>:0 
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <00000000000000000000000000000000>:0 
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state) [0x00000] in <00000000000000000000000000000000>:0 
  at System.Threading.ThreadHelper.ThreadStart () [0x00000] in <00000000000000000000000000000000>:0

CodePudding user response：

It appears that you are trying to use Jackson to parse the entire response which includes non json content. With the info you've given it looks like you have 2 options

Find a way to discard anything in the file that's not part of the json response and feed that in to Jackson
Manually parse the response pulling out the parts you need