Databricks SCALA UDF cannot load class when registering function-CodePudding

I have followed this guide and this question trying to implement a decryption function to use in a SQL view.

I have compiled this scala code in the example to a jar file and uploaded to the Databricks File System (DBFS):

import com.macasaet.fernet.{Key, StringValidator, Token};
import org.apache.hadoop.hive.ql.exec.UDF;
import java.time.{Duration, Instant};

class Validator extends StringValidator {
    override def getTimeToLive() : java.time.temporal.TemporalAmount = {
      Duration.ofSeconds(Instant.MAX.getEpochSecond());
    }
  }

class udfDecrypt extends UDF {

    def evaluate(inputVal: String, sparkKey : String): String = {

      if( inputVal != null && inputVal!="" ) {
        val keys: Key = new Key(sparkKey)
        val token = Token.fromString(inputVal)
        val validator = new Validator() {}
        val payload = token.validateAndDecrypt(keys, validator)
        payload
      } else return inputVal
    }
  }

I can declare the function as demonstrated:

%sql
CREATE OR REPLACE FUNCTION default.udfDecrypt AS 'com.nm.udf.udfDecrypt'
USING jar 'dbfs:/FileStore/jars/decryptUDF.jar';

But if I try to call it an error is thrown:

%sql
SELECT default.udfDecrypt(field, '{key}') FROM default.encrypted_test;

Error in SQL statement: AnalysisException: Can not load class 'com.nm.udf.udfDecrypt' when registering the function 'default.udfDecrypt', please make sure it is on the classpath; line 1 pos 7

I have noticed that the function can be declared using any jar file path (even one that doesn't exist) and it will still return 'OK'.

I am using Databricks for Azure.

CodePudding user response：

It seems like your UDF code is missing:

package com.nm.udf;

at the top.