ResearchFox - Data Engineer (3-5 yrs)

Job Description :

We need Elasticsearch, Gen AI, LLMs/AI engineering skills, and strong API engineering and test driven development skills with python. Kubernetes based deployments desirable. Cloud skills are a must. Knowledge of shipping production grade code is essential.

Key Responsibilities :

- Should be self-motivated and with a go-getter attitude, working under ambiguity and with at times incomplete requirements.

- Strong abilities to deliver innovative new solutions as a full stack engineer working on data science and data engineering POCs and projects

- Good working knowledge of Azure Cloud or AWS and various aspects of these platforms such as IAM, storage, compute, services and building applications on them.

- Knowledge of SQL and the ability to work with a SQL database in the backend, such as PostgreSQL.

- Knowledge of Apache Spark, Apache Dask and similar distributed data processing frameworks and the Databricks platform

- Conversant with the development and deployment of Python web services and APIs for different architectures, such as synchronous and asynchronous REST API calling, event-driven ML product architectures and batch processing architectures for reporting and BI use cases

- Knowledge on JavaScript/TypeScript and the ability to develop simple and functional front-ends with frameworks such as React or Vue

- Conversant with one statically typed language and the development of applications in it. C# .NET, or Java knowledge desirable but not necessary

- Some knowledge of Elastic stack (Elastic, Logstash, Kibana), Databricks, vector databases such as Qdrant, and caches such as Redis preferred

- Ability to understand, develop and maintain code bases with performant, well tested and readable code, good test coverage and adherence to code standards such as PEP8 and .NET coding conventions

- Conversant with Git, Docker, CI/CD and the deployment of applications on Azure

- Logging and monitoring using tools such as Azure Log Analytics, Monitor, Prometheus

- Familiar with the sequential development, testing and promotion of code through different environments, leading up to production release.

- Desirable if the candidate is conversant with parallel processing and concurrency paradigms in Python and .NET

- Effectively collaborates with different team members such as data scientists, engineers, and architects to build and release data and AI applications.

- Clear verbal and written communication skills in English, and the ability to interact well with a globally distributed and diverse data science and engineering team.