Databricks Data Engineer

NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.

We are currently seeking a Databricks Data Engineer to join our team in Bangalore, Karn taka (IN-KA), India (IN).

Job Duties: Key Responsibilities:

Data Pipeline Development: Design, build, and maintain scalable data pipelines using Databricks on AWS.
Data Integration: Ingest data from multiple sources (structured and unstructured) using AWS services such as S3, Lambda, Glue, and Kinesis.
Big Data Processing: Develop and optimize Spark-based ETL jobs to process large datasets on Databricks.
Data Architecture: Collaborate with the architecture team to implement best practices for cloud-based data lakes and data warehouses.
Optimization: Tune Spark jobs for performance improvements and efficient resource usage.
Strong experience with Databricks, including Apache Spark development.
Proficiency in Python or Scala for Spark job development.
Experience with SQL for querying and managing data.
Familiarity with CI/CD tools (eg, Jenkins, GitLab) for automating deployment and testing of data pipelines.
Knowledge of big data technologies (Hadoop, Hive) and ETL frameworks.
Familiarity with AWS security best practices (IAM, VPC, security groups).

Minimum Skills Required: Data Engineer, Databricks, Cloud migration 3+ year (Multiple projects)