Senior Elasticsearch Engineer

Job Title: ELK Stack Engineer

Position Overview:

We are seeking a skilled and experienced ELK Stack Engineer to join our team. The ideal candidate will be responsible for designing, developing, and maintaining data pipelines using the ELK Stack (Elasticsearch 7.1 or higher, Logstash, Kibana). The ELK Stack Engineer will play a key role in managing large datasets, optimizing complex queries, and maintaining self-hosted ElasticSearch clusters.

Responsibilities:

1. Design, develop, and maintain data pipelines using the ELK Stack.
2. Handle large datasets (3 billion+ records, 100+ fields per index) with efficiency and accuracy.
3. Develop and optimize complex Elasticsearch queries for efficient data retrieval and analysis.
4. Manage and maintain self-hosted ElasticSearch clusters.
5. Implement DevOps practices, including containerization with Docker.
6. Perform backups, disaster recovery, and index migrations to ensure data security and integrity.
7. Execute data processing and cleaning techniques, including writing custom scripts to extract and transform data into usable formats.
8. Collaborate with the engineering team to integrate ElasticSearch with other systems.

Requirements:

1. Bachelor's degree in Computer Science, Engineering, or related field.
2. Minimum 5+ years of experience in designing, developing, and maintaining data pipelines using the ELK Stack.
3. Hands-on experience handling large datasets (3 million+ records, 50+ fields per index).
4. Strong proficiency in developing and optimizing complex Elasticsearch queries.
5. Experience managing and maintaining self-hosted ElasticSearch clusters.
6. Proficiency in DevOps practices, including containerization with Docker.
7. Knowledge of security best practices, including backups, disaster recovery, and index migrations.
8. Experience with database administration and data processing techniques.
9. Excellent communication and teamwork skills.

Benefits:

1. Competitive salary and benefits package.
2. Opportunities for professional development and growth.
3. Collaborative and inclusive work environment.
4. Flexible work hours and remote work options.