Big Data Developer - Data Lake Pipelines

About the role

Talan, an international consulting group focused on innovation and business transformation through technology, is looking for a Big Data Developer to join an Agile environment. You will design, develop and evolve data lake solutions for Market Risk use cases, working on large-scale data processing pipelines. The role combines hands-on technical development with collaboration across Agile squads, functional teams and production support teams.

Key responsibilities

• Design, develop and maintain data lake pipelines using Apache Spark, Scala and SQL.
• Build and maintain scalable data processing solutions for large volumes of data.
• Orchestrate batch workflows and dependencies with tools such as Control-M and Airflow.
• Implement data ingestion, transformation, validation and publication processes for Market Risk use cases.
• Analyse and optimise the performance, reliability and scalability of Spark-based workloads.
• Analyse technical and functional requirements, identify gaps and propose solutions.
• Contribute to delivery planning and work with different teams to ensure successful implementation.
• Support production incidents, perform root-cause analysis and help improve operational processes.
• Follow software engineering and data development best practices.

Required profile

• 2+ years of professional experience in Big Data, Data Lakes or cloud-based data platforms.
• Strong hands-on experience with Apache Spark, Scala and SQL for large-scale data processing.
• Good understanding of data lake architecture, data quality controls, data lineage and production-grade pipelines.
• Strong analytical and problem-solving skills, and excellent communication and teamwork skills.
• English at C1 level or equivalent professional fluency; Spanish is a plus.

Required skills

• Apache Spark, Scala and SQL (mandatory)
• Nice to have: Control-M and Airflow, Databricks, Azure and AWS, YARN, CI/CD tools such as GitHub, Jenkins and SonarQube, banking production environments and ServiceNow, knowledge of Market Risk processes (P&L, valuations, risk metrics, regulatory reporting)

What we offer

• 100% remote setup
• Training and career development
• The chance to work in a multicultural team on international projects