Sr. Software Engineer

"JD: Strong programming skills in Python. Proficiency with Hadoop ecosystem technologies like HDFS, Spark. Good knowledge on Big Data querying tools, such as Hive and Impala Working Knowledge of data ingestion using spark for supporting various file formats like Json , Csv and parquet with its compression techniques. Perform analysis of vast data stores and uncover insights. Experience in Shell Script . Roles and Responsibilities: Hadoop development and implementation. Translate complex functional and technical requirements into detailed design. Loading from disparate data sets and processing the data as per business need Organizing data into tables, performing transformations, performance tuning and simplifying complex queries with Py spark. Monitoring data performance and modifying infrastructure as needed Maintain security and data privacy. "