Data Engineer with Data Bricks Full Time NEW

Position: Data Engineer with Data Bricks

Relevant years of Experience: 8 Years

Job Type: Contract

Shift Time: UK Shift (2 PM to 10 PM)

Work Location: Chennai (OMR – 2 Days a Week)

Salary Range: Max 24L PA

Key Responsibilities and Requirements:

• Pipeline Development: Design, build, and optimize Databricks pipelines for transportation and supply chain data processing (e.g., last-mile tracking and truckload optimization).
• Data Modeling: Develop semantic models and Unity Catalog metric views to enable business self-service KPIs, such as on-time delivery and utilization metrics.
• ETL/ELT Workflows: Implement workflows in Databricks (Spark, Delta Lake) to manage large-scale logistics datasets.
• Collaboration: Work remotely with the US pod (TPM, BAs) within Agile sprints, ensuring hybrid offshore delivery meets quality standards and timelines.
• Governance: Perform data modeling, cleansing, and governance to support KPI definitions and scalable analytics.
• Optimization: Monitor jobs, troubleshoot Databricks clusters, and optimize performance for cost-efficiency.
• Integration & Compliance: Integrate with BI tools for self-service dashboards while adhering to healthcare data compliance requirements (e.g., HIPAA).