Bigdata Applications with Microservices & Airflow
KAFKA cluster start:
create some topic in kafka which can be used to pass json message.
SpringBoot Kafka Microservice :
Spark Jobs as Service :
Job reads the kafka topic and prints the message also writes it to parameter file in Airflow execution directory.
Airflow setup and Dag creation through parameter file :