Data Engineer II - Scala/Python (2-4 yrs)

About Media.net :

Media.net is a leading online ad-tech company that develops innovative products for advertisers and publishers. Media.net has one of the most comprehensive portfolios of advertising technology in the industry across search, native, display, mobile, local, products and video.

We manage high-quality ad supply on over 500,000 websites and our platform and products are licensed by some of the largest publishers, ad networks and other ad tech companies worldwide.

Media.net is one of the largest contextual advertising company worldwide. Media.net has key operation centers across - North America, Europe and Asia. Media.net's US HQ is based in New York, and Global HQ in Dubai.

What does the team do ?

- Every single web page view hits one or more services and hosts high scale service to handle this large volume of requests, across 5 million unique topics.

- Some of the platforms built and managed by the team.

- To achieve this we use cutting edge Machine Learning and AI technologies on a large Hadoop cluster.

- Tech Stack - Java, Elastic Search/Solr, Kafka, Spark, Machine Learning, NLP, Deep Learning, Redis, Big Data technologies like Hadoop, HBase, YARN etc.

Roles and Responsibilities :

- Design, execution and management of a large and complex distributed data systems.

- Monitoring of performance and optimizing existing projects.

- Researching on and integrating any Big Data tools and frameworks required to provide requested capabilities.

- Understanding Business/Data requirements and implementing scalable solutions.

- Creating reusable components and data tools which help all the teams in the company to integrate with our data platform.

Who should apply for this role?

- 2 to 4 years of experience in big data technologies (Apache Hadoop) and relational databases (ms sql server/oracle/MySQL/postgres).

- Proficiency in at least one of the following programming languages - Java, Python or Scala.

- Expertise in SQL (T-SQL/PL-SQL/SPARK-SQL/HIVE-QL).

- Proficiency in Apache Spark.

- Hands on knowledge of working with Data Frames, Data Sets, RDDs, Spark SQL/PySpark/Scala APIs with deep understanding of Performance Optimizations.

- Good Understanding of Distributed Storage (HDFS/S3).

- Strong analytical/quantitative skills and comfortable working with very large sets of data.

- Experience with integration of data across multiple data sources.

- Good understanding of distributed computing principles.

It would be good if you have the below skills:

a)Experience with Message Queues (e.g., Apache Kafka)

b)Experience with MPP systems (e.g., Redshift/Snowflake).

c)Experience with NoSQL storage (e.g., MongoDB)