Data Engineer

Permanent contract

Paris

A few days at home

Salary: Not specified

Experience: > 3 years

We're looking for an experienced Data Engineer to reinforce our Data team and contribute to our mission: transforming a critical mass of information into actionable insights for the entire company.

You master the full data lifecycle, from development to production. You will join a team in its maturation phase to tackle a major challenge: transitioning from internal data management to massive real-time processing (500,000 events per second). Alongside our Data Ops and our Head of Data, you will be a driving force in the implementation of our Spark environment.

Your missions:

• Design and implement the Spark environment on Kubernetes to process massive real-time data streams.

• Centralize and harmonize heterogeneous sources (Product, Finance, Marketing) to make data accessible and reliable for all Sekoia teams.

• Develop, test, and orchestrate pipelines (via Airflow).

• Propose innovative solutions to ensure the resilience and performance of our platform.

• Empower collaborators by providing the tools they need to leverage high-quality data independently.

Preferred experience:

• You have 3 to 5 years of professional experience in Data Engineering or Python development with a strong focus on data processing.

• You are proficient in Python and Spark, and you have a solid understanding of Kubernetes environments.

• You are a proactive doer: we are looking for someone autonomous and mature who takes ownership of subjects rather than waiting for instructions.

• You demonstrate a strong interest in Big Data challenges, especially in handling massive real-time data streams and high-availability infrastructures.

• You ensure code quality by following software engineering best practices, including Continuous Integration and automated testing.

• You are familiar with orchestration tools, and having experience with Airflow is a significant plus.

• You are driven by efficiency and resilience, capable of building systems that stay performant under heavy loads (500k events/sec).

• You are curious about new trends in the data ecosystem and willing to learn and grow alongside the team.

• You write and speak English fluently.

Bonus points: You have experience in cybersecurity