Data Engineer - PySpark

Develop, automate, and maintain data processing pipelines using Python/Pyspark for data manipulation and automation.

Collaborate with internal teams to understand and define the data needs of the company or client.

Design and build efficient database models in collaboration with the development team.

Lead database implementation efforts with the development team to ensure seamless integration and performance.

Manage relational databases (MySQL, PostgreSQL) and NoSQL databases to handle structured and unstructured data.

Work with NoSQL databases for handling semi-structured data, ensuring scalability and flexibility.

Utilize cloud platforms such as AWS and Azure to implement and manage data solutions.

Implement and manage real-time data processing using streaming technologies like Kafka.

Collaborate with the business team to understand and define data reporting requirements.

Adjust data access permissions and reports as needed to meet evolving business needs.

Partner with the development team to implement and maintain data warehouses and reporting systems.

Lead data migration efforts, ensuring smooth transitions and data integrity.

Collaborate with data scientists to define metadata querying requirements and implement efficient querying solutions.

Contribute to data cleaning processes and help manage data quality issues.

Implement data security solutions and ensure compliance with data security standards.

Positive work environments and stellar reputations attract and retain top talent. Find out why Luxoft stands apart from the rest.

Req. VR-111619