340 Spark Jobs in Hyderabad / Secunderabad

As a Data Engineer in Data Infrastructure and Strategy group, you will play a key role in transforming the way Operations Finance teams access and analyse the data. You will work to advance 3 Year Data Infrastructure Modernisation strategy and play a key role in adopting and expanding a unified data access platform and scalable governance and observability frameworks that follow modern data architecture and cloud-first designs.

Your responsibilities will include supporting and migrating data analytics use cases of a targeted customer group and new features implementations within the central platform: component design, implementation using NAWS services that follow best engineering practices, user acceptance testing, launch, adoption and post-launch support. You will work on system design and integrate new components to established architecture. You will be engaged into cross-team collaboration by building reusable design patterns and components and adopting designs adopted by others. You will contribute to buy vs “build” decision by evaluating latest product and features releases for NAWS and internal products, perform gap analysis and define feasibility of their adoption and the list of blockers.

The ideal candidate possess a track record of creating efficient AWS-based data solutions; data models for both relational databases and Glue/Athena/EMR stack; developing solution documentation, project plans, user guides and other project documentation. We are looking into individual contributor inspired to become data systems architects. Track of production level deliverables leveraging GenAI is big plus.

Key job responsibilities
* Elevate and optimize existing solutions while driving strategic migration. Conduct thorough impact assessments to identify opportunities for transformative re-architecture or migration to central platforms. Your insights will shape the technology roadmap, ensuring we make progress towards deprecation goals while providing best customer service;
* Design, review and implement data solutions that support WW Operations Finance standardisation and automation initiatives using AWS technologies and internally built tools, including Spark/EMR, Redshift, Athena, DynamoDB, Lambda, S3, Glue, Lake Formation, etc;
* Support data solutions adoption by both, finance and technical teams; identify and remove adoption blockers;
* Ensure the speed of delivery and high-quality: iteratively improve development process and adopt mechanisms for optimisation of the development and support;
* Contribute into engineering excellence by reviewing design and code created by others;
* Contribute to delivery execution, planning, operational excellence, retrospectives, problem identification, and solution proposals;
* Collaborate with finance analysts, engineers, product and program managers and external teams to influence and optimize the value of the delivery in data platform;
* Create technical and customer-facing documentation on the products within the platform.

A day in the life
You work with the Engineering, Product, BI and Operations teams to elevate existing data platforms and implement best-of-class data solutions for Operations Finance organization. You solve unstructured customer pain points with technical solutions, you are focused on users productivity when working with the data. You participate in discussions with stakeholders to provide updates on project progress, gather feedback, and align on priorities. Utilizing AWS CDK and various AWS services, you design, execute, and deploy solutions. Your broader focus is on system architecture rather than individual pipelines. You regularly review your designs with Principal Engineer, incorporate gathered insights. Conscious of your impact on customers and infrastructure, you establish efficient development and change management processes to guarantee speed, quality and scalability of delivered solution.

About the team
Operations Finance Standardization and Automation improves customer experience and business outcomes across Amazon Operations Finance through innovative technical solutions, standardization and automation of processes and use of modern data analytics technologies.

Basic qualifications

• MS or BS in Computer Science, Electrical Engineering, or similar fields;
• Strong AWS engineering background, 3+ years of demonstrated track record designing and operating data solutions in Native AWS. The right person will be highly technical and analytical with ability to drive technical execution towards organization goals;
• Exceptional triaging and bug fixing skills. Ability to assess risks, implement fix without customer impact;
• Strong data modelling experience, 3+ years of data modeling practice is required. Expertise in designing both analytical and operational data models is a must. Candidate needs to demonstrate the working knowledge of trade-offs in data model designs and platform-specific considerations with concentration in Redshift, MySQL, EMR/Spark and Athena;
• Excellent knowledge of modern data architecture concepts - data lakes, data lakehouses, - as well as governance practices;
• Strong documentation skills, proven ability to adapt the document to the audience. The ability to communicate the information on levels ranging from executive summaries and strategy addendums to detailed design specifications is critical to the success;
• Excellent communication skills, both written and oral. Ability to communicate technical complexity to a wide range of stakeholders.

Preferred qualifications

• Data Governance frameworks experience;
• Compliance frameworks experience, SOX preferred;
• Familiarity or production level experience with AI-based AWS offerings (Bedrock) is a plus.