Data Engineer

Key Responsibilities

• Design, develop, and maintain ETL/ELT pipelines using Databricks, Fabric, or Snowflake.
• Build and optimize data workflows for performance, scalability, and cost efficiency in cloud environments (Azure/AWS/GCP).
• Build data transformation and processing solutions using Python, PySpark and SQL.
• Implement Bronze/Silver/Gold or Medallion Architecture and scalable data models.
• Implement and manage data lakes, data warehouses, or lakehouse architectures.
• Integrate data engineering solutions with Power BI and downstream analytics applications.
• Develop and maintain Power BI dashboards and reports to support business insights and decision-making.
• Collaborate with cross-functional teams to define data requirements, governance standards, and best practices.
• Ensure data quality, integrity, and security across all platforms.
• Automate workflows and support CI/CD deployments for data solutions.
• Monitor and troubleshoot pipelines and dashboards to ensure high availability and reliability.

Experience

• Bachelor’s/Master’s degree in Computer Science, Information Technology, Engineering, or related field.
• 3-10+ years of experience in Data Engineering.

Core Technical Requirements

• Hands-on experience with at least one of the following — Microsoft Fabric, Azure Data Factory (ADF), or Databricks.
• Strong Python programming skills for data processing.
• Strong PySpark / Apache Spark experience for large-scale data processing and transformation.
• Strong SQL skills and experience with complex queries.
• Good understanding of data modeling — Star Schema, dimensional modeling and Medallion Architecture.
• Experience designing and developing scalable ETL/ELT data pipelines.
• Experience working with at least one major cloud platform — Azure, AWS or GCP.
• Experience with data migration projects such as Informatica → Fabric, SSIS → ADF, legacy platforms → Databricks/Fabric, etc..
• Experience delivering multiple data engineering projects; 6–8 Databricks projects preferred for Databricks-focused roles.
• Proven experience as a Data Engineer with expertise in Databricks OR Fabric OR Snowflake.
• Hands-on experience with Power BI (data modeling, DAX, dashboard creation, performance optimization).
• Hands-on experience with Dataflow Gen2 for low-code data ingestion and transformation within Microsoft Fabric.
• Strong proficiency in PySpark for distributed data processing, building scalable ETL pipelines, and large-scale data transformations.
• Strong proficiency in SQL and at least one programming language (Python/Scala/Java).
• Experience with data modeling and building scalable pipelines in cloud environments.
• Knowledge of Azure, AWS, or GCP and their data ecosystem.
• Strong problem-solving, analytical, and communication skills.

Platform & Data Ecosystem

• Hands-on experience with Databricks / Microsoft Fabric / ADF.
• Experience with Delta Lake and modern cloud data architectures is preferred.
• Snowflake or Azure Synapse experience is a plus.
• Understanding of Power BI integration, including data models and consumption of data engineering outputs.

Engineering & Advanced Skills

• Experience with CI/CD, Git and DevOps practices for data engineering.
• Understanding of Ai Ops pipelines and GenAI data workloads is a plus.
• Knowledge of Databricks platform optimization and the broader Databricks ecosystem is preferred.
• Understanding of data governance, lineage, security and compliance.
• Kafka / Event Hubs / Kinesis experience for streaming pipelines is a plus.

Preferred Qualifications

• Experience with streaming data technologies (Kafka, Event Hubs, Kinesis).
• Knowledge of Delta Lake, Synapse, or Snowflake performance optimization.
• Familiarity with DevOps practices for data (CI/CD, Git, Infrastructure as Code).
• Exposure to machine learning pipelines or advanced analytics.
• Understanding of data governance, lineage, and compliance frameworks

Preferred Certifications

• Databricks Professional Certification is preferred, but not mandatory.