Softility - Senior/Lead Big Data Engineer (10-15 yrs)

Big Data Developer

Job Description :

- Participate in the design and implementation phases for data projects to help translate high level business requirements into technical solutions

- Provide technically sound solutions for the ingestion, storage, transformation, and presentation of enterprise data, including ETL design, data storage strategies, data access and security

- Driving the design of scalable solutions while considering recoverable and resiliency requirements

- Participate in DevOps related activities, including involvement with continuous integration, automated deployment, automated testing, and continuous monitoring to enhance performance and product quality

- Provide timely support for troubleshooting and resolving production issues

Required Technical and Professional Expertise :

- Experience with any of the following : Big Data technologies, Analytics, Hadoop ecosystems, DataLakes, or Data Migration

- Experience with cloud platforms such as AWS, Azure, or Google Cloud

- Experience with Java, Scala, Spark

- Strong database integration and management skills

- Strong ETL skills, including database performance tuning and optimization

- Familiar with Shell scripting skills and Unix/Linux knowledge

- Exceptional analytical and problem-solving skills

- Running diagnostic tests and performing debugging procedures

- Documenting application development processes, procedures, and standards

- Excellent leadership and interpersonal skills

- Strong communication and team working skills

- Able to analyze complex situations and derive workable actions

- Flexibility to adjust to multiple demands, shifting priorities, ambiguity and rapid change

Preferred Technical And Professional Expertise :

- Knowledge of DevOps and related tooling such as Jenkins, Ansible etc.

- Experience with Big Data technologies such as Apache Hadoop, HBase, Hive, Spark, Flink or Flume

- Strong coding skills in one of Python/Java/Scala

- Familiar with event management technologies such as Apache Kafka or Apache Airflow

- Familiar with container technologies such as Docker or Kubernetes

- Demonstrable experience coaching junior members of your teams