Senior Data Architect – Databricks Platform

About the role

Caylent is seeking a Senior Data Architect to design and harden enterprise‑scale Databricks platforms for client engagements. The role focuses on building guardrails, automation, and standards that enable business teams to safely extend the platform.

Key responsibilities

• Lead architectural design for Databricks‑based data platforms, including workspace consolidation, permission models, and config‑as‑code patterns.
• Design high‑volume batch and real‑time streaming pipelines using PySpark and Structured Streaming.
• Integrate Unity Catalog with external metadata repositories such as AWS Glue Catalog.
• Implement automation, CI/CD, and guardrail patterns to ensure governance while supporting self‑service and future agent‑driven use cases.
• Produce architectural documentation, assessments, and roadmaps for technical and business stakeholders.

Required profile

• 6+ years in data engineering, database engineering, or architecture roles.
• 3+ years designing cloud‑based data solutions on AWS.
• 3+ years hands‑on experience with Databricks, including workspace, Unity Catalog, and Delta Lake.
• Strong business acumen and ability to communicate with client stakeholders.
• Advanced English (minimum C1).

Required skills

• Databricks platform architecture (workspace, Unity Catalog, lakehouse patterns).
• PySpark, Structured Streaming, Spark configuration and optimization.
• AWS services and cloud‑native system design.
• Terraform and Infrastructure as Code.
• CI/CD pipelines and automation (ConfigOps).

What we offer

• Competitive pay in USD.
• 100% remote work with flexible PTO and generous holidays.
• Paid exams, certifications, and professional development plan.
• State‑of‑the‑art laptop, equipment stipend, and peer bonus awards.