Category : apache-kafka-streams

I am new to Structured Streaming with Kafka. Trying to convert the , delimited data from Kafka to Dataframe in PySpark using schema and from_csv. kafkaDataSchema = StructType([ StructField("sid", StringType()), StructField("timestamp", LongType()), StructField("sensor", StringType()), StructField("value", StringType()), ]) kafkaStream = spark.readStream .format("kafka") .option("kafka.bootstrap.servers", self.config.get(‘kafka-config’, ‘bootstrap-servers’)) .option("subscribe", self.config.get(‘kafka-config’, ‘topic-list-input’)) .option("startingOffsets", self.config.get(‘kafka-config’, ‘startingOffsets’)) .load() .selectExpr("CAST(value AS STRING)") formattedStream ..

Read more

I’m new to Apache Kafka. I’m trying to run their DemoApp in StreamAPI section and I get the following error in cmd: The syntax of the command is incorrect. Here’re the variants I’ve used: Variant 1: binwindowskafka-console-consumer.bat –bootstrap-server localhost:9092 –topic streams-wordcount-output –from-beginning –formatter kafka.tools.DefaultMessageFormatter –property print.key=true –property print.value=true Variant 2: binwindowskafka-console-consumer.bat –bootstrap-server localhost:9092 –topic streams-wordcount-output ..

Read more