Data Engineer

Knowledge of Talend studioTalend is needed and AWS knowledgeTalend to Python/PySpark conversion of ETL jobs, including associated Autosys scheduling jobs.Spark developer who knows how to fully exploit the potential of our Spark cluster and special emphasis to database development skills across Migration projects.5 to 8 years of experience on client projects to deliver AWS, PySpark, Databricks based Data engineering Analytics solutions Build and operate very large data warehouses or data lakes ETL optimization, designing, coding, tuning big data processes using Apache Spark Build data pipelines applications to stream. Scala (with a focus on the functional programming paradigm), Scalatest, JUnit, Mockito {{ , Embedded Cassandra }}, Apache Spark 2.xSpark query tuning and performance optimizationSQL database integration {{ Microsoft, Oracle, Postgres, and/or MySQL }}Experience working with {{ HDFS, S3, Cassandra, and/or DynamoDB }}