Cloud Streaming Engineer - Azure Cloud Services (5-6 yrs)

Job Title : Cloud Streaming Engineer.

Job type : Onsite ( Mumbai ).

Experience : 5+ years.

Work Mode :

Work from Office (WFO) - 5 days a week (Flexible Timings).

Mandatory Requirement :

- Must attend daily client stand-up at 9 AM PST without fail.

Experience Requirement :

Minimum : 5+ years of relevant hands-on experience.

Mandatory Technical Skills (Hands-on Experience Required) :

The ideal candidate must have worked on multiple projects involving the following technologies :

Programming & Database Expertise :

- Python : Strong expertise in Python, including scripting and data manipulation.

- SQL : Ability to write complex SQL queries, including JOINs, Aggregations, and Window Functions.

- Data Quality Management : Ensuring data accuracy and consistency.

Streaming & Big Data Processing :

- Apache Flink (Hands-on experience required) - Streaming Data Processing.

- Apache Kafka (Hands-on expertise required) - Event Streaming Platform.

- Spark Streaming or Flink Streaming with Kafka (Hands-on experience required).

Spark & Hive :

- Strong understanding of Spark concepts, Spark Sessions, and Streaming Processing.

- Experience in real-time ETL pipelines using Spark Streaming/Flink Streaming.

- Optimization techniques for Spark/PySpark and Hive applications.

- Snowflake, Hive, Trino (AWS Athena) for Data Storage & Processing.

Cloud & Orchestration :

- Azure Cloud & Services (Preferred).

- Orchestration : Experience with Apache Airflow or similar tools.

Additional Key Skills :

- Data Lake Development Experience.

- Batch Processing and File Storage (S3 or equivalent).

- Spark Optimization & Spark Submit Command.

- Troubleshooting & Debugging Flink Pipelines.

Soft Skills & Evaluation Criteria :

- Strong problem-solving skills - Ability to write and optimize efficient streaming queries.

- Excellent communication skills - Must effectively communicate complex technical solutions.

- Ability to write production-ready code for real-world streaming scenarios.

Sample Evaluation Topics :

Candidates should be prepared to answer technical questions and demonstrate hands-on expertise on :

- Spark Streaming vs. Batch Processing.

- Spark Structured Streaming Concepts.

- Use of spark. readStream() method in Spark Streaming Applications.

- Usage of writeStream() method with different supported arguments.

- Action required to read data from Kafka queue (start()).

- Printing streaming operation output on Terminal using format("console").

Required Hands-on Experience with the Following Technologies :

- Python .

- SQL (Advanced Queries).

- Apache Flink.

- Apache Kafka.

- Spark (including Streaming & Batch Processing).

- Hive or Snowflake.