spark-streaming-CodePudding

Tags > spark-streaming

09-09databaseWhen using a broadcasted spark variable in a udf, how do I ensure the variable is not used before it
09-09EnterpriseWhy is this JSON giving null values when I deserialize and try to extract values
08-30OSParsing json multiline schema in spark streaming
08-21OSAccess In-Memory Spark Dataframe from different nodes
07-02Software engineeringPython Proxy Error when streaming XML files from Azure Event Hub using Databricks
06-30NetHow can I close db connection from spark.read?
05-31Software engineeringRead Spark streaming source once when applying and writing multiple transformations
05-19OSoffset management in spark streaming
05-19Back-endScala Kafka Spark: Get all kafkaConsumer endOffSets and assign it to a val
05-11Netrequirement failed: Provided Maven Coordinates must be in the form 'groupId:artifactId:version&
05-05Software designSpark structured streaming- checkpoint metadata growing indefinitely
04-22Back-endPython kafka module when used with pyspark causes 'ModuleNotFound' error?
04-13NetIngesting data into spark using socketTextStream
04-12databaseClass org.apache.hadoop.fs.s3a.auth.IAMInstanceCredentialsProvider not found when trying to write da
03-25MobilePySpark dataframe not getting saved in Hive due to file format mismatch
03-24NetPyspark code error: Invalid argument, not a string or column
03-16OSInterpreting dataframe out of Array JSON in Spark Structured Streaming
03-10MobileWhat's the right way to "log and skip" validated transformations in spark-streaming
03-02front endpyspark inserInto in overwrite mode is appending and not overwriting partitions
01-04BlockchainAppend values from one column to another JSON column in the same dataframe
12-29NetScala Dataframe column split URL parameters to new columns
12-28BlockchainSpark Scala split column values in a dataframe to appended lists
12-24Software designAzure Databricks EventHub connection error
12-23EnterpriseWhere can I find the JavaTestUtils and BatchCounter dependencies referenced in JavaMapWithStateSuite
12-11EnterpriseHow to add a column to complex Spark structures with python?
12-09MobileSpark twitter streaming application Error on windows 10 when submitting python file to local server
11-30otherorg.apache.spark.sql.AnalysisException: Non-time-based windows are not supported on streaming DataFr
11-18MobileHow can I add several columns (still not populated) to the DataFrame in Spark Structured Streaming
11-12NetHow to arrange dataframe according to the given json format
11-11Software engineeringSpark and Kafka: how to increase parallelism for producer sending large batch of records improving n
10-22Software designCan we use Kstream with Spark Streaming?
10-20MobileAmbiguous reference to array fields in pyspark
10-20MobileCan we use Kstream with Spark Streaming in 2021?
10-18OSjava.lang.ClassNotFoundException: Failed to find data source: org.apache.bahir.sql.streaming.mqtt.MQ
10-18OSHow to output to console the format of the dataFrames in format("console")?
09-29Back-endHow to set yarn application id in a PySpark app
09-24OSIs it possible to explode a column without including the values from other columns?
09-24OSHow to pass ojai configuration from driver to executors in spark?