• Skills: SQL, Kafka, Docker, Kubernetes, Python, Java, Scala, Apache Kafka, Message queues, Airflow
• Experiences: Senior
• Employments: Full-time
• Locations: Egypt (Remote)

Description:

• Builds and runs real-time streaming pipelines with low latency and high recoverability.
• Strong hands-on experience with Apache Flink, including DataStream API and Flink SQL.
• Involves event time processing, checkpointing, savepoints, and state management in production.
• Works with high-volume Kafka streaming and data modeling for analytics into a lakehouse.
• Monitors pipeline health and manages recovery when jobs fail.

Requirements:

• Strong hands-on experience with Apache Flink (DataStream, Flink SQL).
• Experience with high-volume Kafka streaming and real-time data processing.
• Knowledge of event time processing, checkpointing, savepoints, and state management.
• Experience in data modeling for analytics and streaming into a lakehouse.
• Familiarity with monitoring pipeline health and handling job recovery.
• Knowledge of Apache Iceberg or other open table formats.
• Experience with Airflow.
• On-premises or self-managed cluster experience.

Benefits:

• Opportunity to work with cutting-edge technologies in real-time data processing.
• Engage in a dynamic and collaborative work environment.