Data engineer Databricks

About the Role:

Are you passionate about building modern data platforms and turning complex data into trusted insights? We are looking for a Data Engineer to help build and evolve our security data platform.

In this role, you will be responsible for integrating data from technical and business systems into a centralized data platform. You will design robust data pipelines, transform raw data into analytics-ready datasets, and ensure data quality across the platform. You will work with various stakeholders to convert the wishes within the organization into datasets that can be effectively read and consumed by data analysts and data scientists.

This position sits at the intersection of Data Engineering and Analytics Engineering, making it ideal for someone who enjoys both platform development and creating reliable datasets that power reporting, analytics, and operations.

What you’ll do:

• Design, build, and maintain scalable data ingestion (ETL/ELT) pipelines.
• Build and optimize data models for analytics, monitoring, and operational reporting.
• Ensure platform reliability, performance, and scalability.
• Work closely with analysts and stakeholders to understand data requirements.
• Develop SQL-based transformations.
• Create clean, reusable, and business-friendly data models.
• Implement automated data quality testing and monitoring.
• Apply software engineering best practices such as version control and CI/CD.
• Drive adoption of analytics engineering practices across the data platform.

What we’re looking for:

• 4+ years of experience as Data Engineer, Data Warehouse Expert, Analytics Engineer, or BI Developer.
• Strong Python and SQL skills with experience developing complex transformations.
• Experience building and maintaining data pipelines and data integration solutions.
• Solid understanding of data modeling principles and dimensional modeling.
• Experience implementing data quality checks and validation frameworks.
• Strong problem-solving skills and attention to detail.
• Ability to work independently while collaborating across technical and non-technical teams.
• Experience with git, CI/CD, Docker, Linux and Infrastructure as Code.
• Experience with modern data lakehouse architectures.
• Knowledge of orchestration tools such as Airflow, Azure Data Factory, or similar platforms.

Preferred Experience:

• Hands-on experience with Azure, Databricks and dbt (Data Build Tool).
• Experience working with both on-premises and cloud-based environments.
• Experience working with cybersecurity, risk, or compliance data
• Experience handling high-volume log and event data.
• Knowledge in stream technology (Kafka).

Soft skills:

• Strong communication and interpersonal skills.
• Ability to collaborate effectively and asynchronously with cross-functional teams including analysts, data engineers, and business stakeholders.
• Ability to work independently and manage multiple projects simultaneously.
• Strong attention to detail and ability to deliver high-quality work under tight deadlines.
• Ability to adapt to new technologies and processes quickly.

What success looks like:

• A scalable data platform that consolidates data from various internal sources.
• Reliable, tested, and well-documented data products.
• High-quality datasets that enable analysts and stakeholders to make informed decisions.
• Automated data quality monitoring and governance mechanisms.
• Strong collaboration between engineering, analytics, and teams.