GrowExx - Senior Data Engineer - Spark/Hadoop (5-6 yrs)

Job Description :

Key Responsibilities :

- Design architecture for data activation inclusive of an API gateway that can be consumed by client applications.

- Develop and maintain scalable pipelines that deploy data out to product applications, website data Layers, and digital advertising platforms (Salesforce, Google, Facebook, etc.)

- Develop and deploy scalable new features.

- Identify and fix bugs to resolve data quality issues in a timely manner.

- Develop, maintain, update Tableau reporting views and data sources.

- Implement a data monitoring framework that ensures production tables are always accurate.

- Write clean, maintainable, and efficient Java code for data transformation and integration tasks.

- Take data science/machine learning model prototypes and prepare them for production deployments.

- Improve real-time data availability on an as-needed basis for activation use-cases.

- Build pipelines that ingest digital advertising data on an as-needed basis.

- Document all pipelines and maintain a data catalogue.

Key Skills :

- Experience in data engineering or related fields.

- Proficiency in Java for developing scalable data solutions.

- Experience with big data technologies such as Hadoop, Apache Spark, Kafka, or similar frameworks.

- Understanding of relational and NoSQL databases

- Hands-on experience with AWS

- Knowledge of data modelling, data warehousing, and schema design.

- Proficiency in SQL and query optimization techniques.

- Familiarity with CI/CD pipelines and version control systems (e., Git)

- Experience with other programming languages such as Python or Scala.

- Knowledge of containerization and orchestration tools like Docker and Kubernetes.

- Familiarity with machine learning pipelines and tools.

Education and Experience : 5+ years as a Data Engineer or similar roles or similar roles.

Analytical and Personal skills :

- Must have good logical reasoning and analytical skills

- Good Communication skills in English - both written and verbal

- Demonstrate Ownership and Accountability of their work

- Attention to details