I want to serialize and deserialize scala case class using avro ReflectDatumReader/ReflectDatumWriter.

Below is my complete code in scala -

import org.apache.avro.io.{DecoderFactory, EncoderFactory}
import org.apache.avro.reflect.{ReflectDatumReader, ReflectDatumWriter}
import java.io.ByteArrayOutputStream
import java.util.UUID

case class MyRecord(
        string: String ="",
        bool: Boolean = false,
        bigInt: BigInt = 0,
        bigDecimal: BigDecimal = 0,
      )


object AvroEncodingDemoApp extends App {
  val parser = new org.apache.avro.Schema.Parser()
  val a = new MyRecord(string = "???", bool = false, bigInt = BigInt.long2bigInt(1), bigDecimal = BigDecimal.decimal(5))
  val avroSchema = parser.parse(
    """
      |{
      | "type": "record",
      | "name": "MyRecord",
      | "fields": [{
      |     "name": "string",
      |     "type": "string"
      | }, {
      |     "name": "bool",
      |     "type": "boolean"
      | }, {
      |     "name": "bigInt",
      |     "type": {
      |         "type": "long",
      |         "precision": 24,
      |         "scale": 24
      |     }
      | }, {
      |     "name": "bigDecimal",
      |     "type": {
      |         "type": "double",
      |         "logicalType": "decimal",
      |         "precision": 48,
      |         "scale": 24
      |     }
      | }]
      |}
      |""".stripMargin)
  val writer = new ReflectDatumWriter[MyRecord](avroSchema)
  val boaStream = new ByteArrayOutputStream()
  val jsonEncoder = EncoderFactory.get.jsonEncoder(avroSchema, boaStream)
  writer.write(a, jsonEncoder)
  jsonEncoder.flush()

  val reader = new ReflectDatumReader[MyRecord](avroSchema)
  val jsonDecoder = DecoderFactory.get().jsonDecoder(avroSchema, new String(boaStream.toByteArray))
  val output = reader.read(null, jsonDecoder)
  println(output)

}

when I run the code I get below error -

Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: java.lang.NoSuchMethodException: MyRecord.() at org.apache.avro.specific.SpecificData.newInstance(SpecificData.java:473) at org.apache.avro.specific.SpecificData.newRecord(SpecificData.java:491) at org.apache.avro.reflect.ReflectData.newRecord(ReflectData.java:1057) at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:237) at org.apache.avro.specific.SpecificDatumReader.readRecord(SpecificDatumReader.java:123) at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:180) at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:161) at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:154) at AvroEncodingDemoApp$.delayedEndpoint$AvroEncodingDemoApp$1(AvroEncodingDemoApp.scala:54) at AvroEncodingDemoApp$delayedInit$body.apply(AvroEncodingDemoApp.scala:14) at scala.Function0.apply$mcV$sp(Function0.scala:42) at scala.Function0.apply$mcV$sp$(Function0.scala:42) at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17) at scala.App.$anonfun$main$1(App.scala:98) at scala.App.$anonfun$main$1$adapted(App.scala:98) at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575) at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573) at scala.collection.AbstractIterable.foreach(Iterable.scala:933) at scala.App.main(App.scala:98) at scala.App.main$(App.scala:96) at AvroEncodingDemoApp$.main(AvroEncodingDemoApp.scala:14) at AvroEncodingDemoApp.main(AvroEncodingDemoApp.scala) Caused by: java.lang.RuntimeException: java.lang.NoSuchMethodException: MyRecord.() at org.apache.avro.specific.SpecificData.lambda$static$0(SpecificData.java:71) at org.apache.avro.util.internal.ClassValueCache$1.computeValue(ClassValueCache.java:35) at java.base/java.lang.ClassValue.getFromHashMap(ClassValue.java:228) at java.base/java.lang.ClassValue.getFromBackup(ClassValue.java:210) at java.base/java.lang.ClassValue.get(ClassValue.java:116) at org.apache.avro.util.internal.ClassValueCache.apply(ClassValueCache.java:45) at org.apache.avro.util.internal.ClassValueCache.apply(ClassValueCache.java:28) at org.apache.avro.specific.SpecificData.newInstance(SpecificData.java:470) ... 21 more Caused by: java.lang.NoSuchMethodException: MyRecord.() at java.base/java.lang.Class.getConstructor0(Class.java:3349) at java.base/java.lang.Class.getDeclaredConstructor(Class.java:2553) at org.apache.avro.specific.SpecificData.lambda$static$0(SpecificData.java:67) ... 28 more

How can I make serializing/deserializing work for scala case class? Is there any working example available? I tried to add default params to fix this issue but couldn't.

CodePudding user response：

You should add not default values but empty constructor

case class MyRecord(
    string: String,
    bool: Boolean,
    bigInt: BigInt,
    bigDecimal: BigDecimal,
) {
  def this() = this("", false, 0, 0)
}

https://scastie.scala-lang.org/DmytroMitin/ZjNsyGwNT5eoaPblJEV2Mw/1