Senior Professional- Data Technical Engineer

• Harness modern application best practices with code quality, test Coverages, Agile Development, DevOps, and Observability and support.
• Design, develop, and deploy microservices using Java and Python to facilitate data processing, transformation, and integration.
• Create APIs and endpoints to interact with data sources, databases, and other services within the data ecosystem.
• Data Pipeline Design and Implementation:
• Design and implement data pipelines to ingest, process, and deliver data from diverse sources into our data lake or data warehouse.
• Ensure data quality, consistency, and reliability throughout the data pipeline.
• Develop data transformation workflows to clean, enrich, and aggregate data as per business requirements.
• Implement Extract, Transform, Load (ETL) processes to integrate data from various sources and systems.
• Identify performance bottlenecks and optimize data processing workflows to improve overall system efficiency.
• Develop and execute unit tests, integration tests, and end-to-end tests to ensure the reliability and accuracy of data pipelines and microservices.
• Set up monitoring and logging systems to proactively identify and resolve issues in data processing workflows.
• Investigate and troubleshoot data-related problems, providing timely resolutions.
• Collaboration and Communication:
• Work closely with data scientists, analysts, and other stakeholders to understand their data requirements and provide data solutions accordingly.
• Collaborate with the development and operations teams to deploy and manage microservices effectively.
• Implement OAuth Okta integration for communication between API producers and consumers.

Qualifications & Experience

• 12+ Years of experience with Bachelors / Masters degree in Computer Science, Engineering, or a related field.
• Solid experience in developing microservices using Java and Python.
• Proficiency in data processing frameworks and libraries, such as Apache Spark, Kafka, or Flink.
• Strong understanding of data modeling, data structures, and algorithms.
• Experience with ETL tools and methodologies.
• Familiarity with data warehousing concepts and databases (e.g., PostgreSQL, MySQL, etc.).
• Knowledge of cloud-based data services and platforms (AWS, Azure, GCP) is a plus.
• Understanding of DevOps practices and containerization technologies (Docker, Kubernetes) is advantageous.
• Excellent problem-solving and analytical skills with a keen eye for detail.
• Strong communication skills and ability to work collaboratively in a team-oriented environment.
• Significant experience of designing, deploying, and supporting production cloud environments like Azure and Kubernetes
• Experience with Azure DevOps CI/CD Tools to build and deploy Java/API packages and API deployment automation.
• Hands-on experience in designing and developing high volume REST using API Protocols and Data Formats.
• Deep Understanding of Databases, API Frameworks, Governance Frameworks, and expertise in hosting and managing platform environments like: Spark, Flink, Kafka, SpringBoot, BI Tools like Tableau, Alteryx, Governance Tools like Callibra, Soda, Amazon DeeQu

Additional Qualifications

• Financial proven experience: Capital markets/Private Equity and/or securities processing
• Familiar in NoSQL\NewSQL databases
• Strong documentation capability and adherence to testing and release management standards
• Design, development, modification and testing of databases designed to support Data Warehousing and BI business teams
• Familiarity with SDLC methodologies, defect tracking (JIRA, Azure DevOps, ServiceNow etc.)
• Monitor and fine-tune microservices for optimal performance and scalability.
• Implement data security best practices and ensure data privacy compliance in all data engineering processes.
• Collaborate with the security and compliance teams to uphold data governance standards.

Soft Skills:

• Candidate must have an analytical and logical thought process for developing project solutions
• Strong interpersonal and communication skills; works well in a team environment
• Ability to deliver under competing priorities and pressures.
• Excellent organizational skills in the areas of code structuring & partitioning, commenting and documentation for team alignment and modifications