Senior Engineer Consultant-AI/ML Engineering

You will be responsible for migrating our feature set preparation code from BTEQ scripts to hive compatible hql or spark in an optimized way. You will incorporate possible automation techniques to hasten this migration project. You will make sure migration is delivered in phases on time with high code quality and standards followed.

• Migrating bteq scripts to hql or spark.
• Documenting the complete migration end to end in VZ Grid.
• Identifying, designing, and implementing internal process improvements by automating manual processes, optimizing data delivery and re-designing infrastructure for greater scalability, etc.
• Building the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and big data technologies.
• Working with partners including the Executive, Product, Data and Design teams to assist with data-related technical issues and supporting their data infrastructure needs.
• Rewriting our big data ETL pipeline in BTEQ to hql/spark, to create datasets for our modeling efforts.
• Wrangling with raw data from large, diverse data sets from our distribution partners.
• Mentoring team members on need basis.

What were looking for...

You are excited to work in a cloud environment, supporting development and deployment in the Verizon Grid. You are self-directed and comfortable supporting the data needs of multiple teams, systems and products. You are excited by the prospect of optimizing or even re-designing the architecture to support our next generation of products and dataset creation for modelling purposes.

You'll need to have:

• Bachelor's degree or four or more years of work experience.
• Four or more years of relevant work experience.
• Experience with big data tools: Hadoop Eco System ( Hive, Pig, OOZie, Spark, Kafka, Elastic search, Kibana).
• Working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working with a variety of databases.
• Experience in scripting languages: Unxi shell scripts, Python or Scala.

Even better if you have one or more of the following:

• Masters degree.
• Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
• Experience with data pipeline and workflow management tools: NIFI.
• Experience with stream-processing systems: Spark-Streaming, Storm etc.
• Experience transforming complex data into easily understandable and actionable information.
• Experience working in a fast-paced environment.
• Ability to quickly adapt to changing priorities.