Senior Data Engineer - Python/Spark (8-12 yrs)

Company Description :

BlueCloud was founded in 2004,

Blue Cloud enables enterprises to transform to cloud. As a Cloud-Only / Cloud-Forward transformation company, we help organizations to enable and speed their journey to cloud,we work with companies to understand their specific business needs to develop and implement practical business strategies and technology solutions

Role Description :

- Design, Architect, and Develop solutions leveraging cloud big data technology to ingest, process and analyse large, disparate data sets to exceed business requirements.

- Develop systems that ingest, cleanse, and normalize diverse datasets, develop data pipelines from various internal and external sources and build structure for previously unstructured data.

- Interact with internal colleagues and external professionals to determine requirements, anticipate future needs, and identify areas of opportunity to drive data development.

- Develop good understanding of how data will flow & stored through an organization across multiple applications such as CRM, Broker & Sales tools, Finance, HR etc.

- Unify, enrich, and analyze variety of data to derive insights and opportunities.

- Design & develop data management and data persistence solutions for application use cases leveraging relational, non-relational databases and enhancing our data processing capabilities.

- Develop POCs to influence platform architects, product managers and software engineers to validate solution proposals and migrate.

- Develop data lake solution to store structured and unstructured data from internal and external sources and provide technical guidance to help migrate colleagues to modern technology platform.

- Contribute and adhere to CI/CD processes, development best practices and strengthen the discipline in Data Engineering Org

- Mentor other members in the team and organization and contribute to organizations' growth.

Qualifications :

What we are looking for :

- 6+ years' work experience and bachelor's degree in information science, Computer Science, Mathematics, Statistics or a quantitative discipline in science, business, or social science.

- Hands-on engineer who is curious about technology, should be able to quickly adopt to change and one who understands the technologies supporting areas such as Cloud Computing (AWS, Azure(preferred), etc.), Micro Services, Streaming Technologies, Network, Security, etc.

- 3 or more years of active development experience as a data developer using Python-spark, Spark Streaming, Azure SQL Server, Cosmos DB/Mongo DB, Azure Event Hubs, Azure Data Lake Storage, Azure Search etc.

- 3+ years of experience working with source code control systems and Continuous Integration/Continuous Deployment tools.

- Independent and able to manage, prioritize & lead workload