Which version of geomesa-accumulo to install on Spark 3.3.0-CodePudding

I need to install geomesa-accumulo on Spark 3.3.0 using a Docker container. In particular, I'm using the jupyter/pyspark-notebook image and I have tried to install geomesa-accumulo_2.11-2.0.0, but there seems to be an incompatibility problem. I think I should use another version of geoserver-accumulo, but I can't find references to figure out which version to install.

From this code snippet, I get the following error.

val dataStoreParams = Map( 
    "geotools" -> "true", 
    "dbtype" -> "postgis",
    "host" -> "backend-postgres", 
    "user" -> "postgres", 
    "passwd" -> "mypassword", 
    "port" -> "5432", 
    "database" -> "dev")

val dataStore = DataStoreFinder.getDataStore(dataStoreParams);

warning: one deprecation (since 2.12.0); for details, enable :setting -deprecation' or :replay -deprecation' java.lang.IllegalArgumentException: org.geotools.data.DataStoreFactorySpi is not an ImageIO SPI class at java.desktop/javax.imageio.spi.ServiceRegistry.checkClassAllowed(ServiceRegistry.java:716) at java.desktop/javax.imageio.spi.ServiceRegistry.(ServiceRegistry.java:117) at org.geotools.factory.FactoryRegistry.(FactoryRegistry.java:155) at org.geotools.factory.FactoryCreator.(FactoryCreator.java:91) at org.geotools.data.DataStoreFinder.getServiceRegistry(DataStoreFinder.java:128) at org.geotools.data.DataStoreFinder.getAvailableDataStores(DataStoreFinder.java:115) at org.geotools.data.DataStoreFinder.getDataStore(DataStoreFinder.java:87) ... 42 elided

CodePudding user response：

You can use the following maven dependency:

<dependency>
    <groupId>org.locationtech.geomesa</groupId>
    <artifactId>geomesa-accumulo-spark_2.11</artifactId>
    <version>3.3.0</version>
</dependency>

References:

https://mvnrepository.com/artifact/org.locationtech.geomesa/geomesa-accumulo-spark_2.11

CodePudding user response：

There are instructions for installing into Jupyter Notebook in the GeoMesa documentation, including the jars to use. Please note that at the time of this post, the latest version of GeoMesa is 3.4.1, and it officially supports up to Spark 3.1, so you may hit incompatibilities.

When you install, make sure that you use the appropriate Scala version for your version of Spark - Spark 3.3.0 by default uses Scala 2.12, so you would need to use GeoMesa artifacts ending with _2.12.