Data Engineer - SQL/PySpark/Hadoop (4-7 yrs)

Job Description :

We are looking for candidates who can join immediately.

InnotatzIT Solutions is a global Software Development, IT Product Design and IT Consulting Company predominantly focused on Web Application Development, Digital Innovations & IT Recruitment Solutions.

We also provide global digital consultancy that helps brands leverage design, analytics, and engineering to create meaningful, personalized customer experiences.

We are looking for Data Engineer with 4+ years of experience for one of our client based in Hyderabad.

This is a permanent remote role.

Responsibilities :

- Build and maintain robust, scalable data pipeline architectures to collect, connect, and curate data from various sources.

- Assemble large, complex data sets that meet functional and non-functional business requirements.

- Identify, design, and implement internal process improvements, including automating manual processes, optimizing data delivery, and re-designing infrastructure for greater scalability.

- Work closely with stakeholders to address data-related technical issues and support their data infrastructure needs.

- Develop analytics tools that utilize the data pipeline, providing actionable insights into customer acquisition, operational efficiency, and key business performance metrics.

- Ensure data quality and integrity by applying best practices in data management and governance.

- Collaborate with data scientists, business analysts, and other technological stakeholders to understand and meet data needs.

Qualifications :

- Bachelor's or master's degree in computer science, Statistics, Informatics, Information Systems, or a related quantitative field.

- Proven experience as a Data Engineer or in a similar role.

- Advanced knowledge of SQL and experience with both relational SQL and NoSQL databases.

- Demonstrated ability building and optimizing 'big data' data pipelines, architectures, and datasets.

- Experience conducting root cause analysis on internal and external data and processes to answer specific business questions and identify improvement opportunities.

- Strong analytical skills with experience working with unstructured data sets.

- Experience with big data tools (Hadoop, Spark, Kafka, etc.) and data pipeline/workflow management tools (Azkaban, Luigi, Airflow, etc.

- Knowledge of AWS cloud services (EC2, EMR, RDS, Redshift) and stream-processing systems (Storm, Spark-Streaming, etc.

Preferred Skills :

- Familiarity with machine learning algorithms and data modeling.

- Proficiency in Python, Java, or Scala.

- Excellent project management and organizational abilities.