Senior Data Software Engineer/ AI Agents, Azure, Spark İş İlanı

We are looking for an experienced Senior Data Software Engineer with strong hands-on expertise in Azure and the Spark ecosystem , primarily focused on building and maintaining data transformation pipelines. The ideal candidate should be comfortable with any Spark-adjacent technology rather than being locked into one specific flavor. The candidate should also combine strong technical skills with leadership capability, contributing to architecture, design, development, and mentoring of engineering teams — preferably in complex enterprise or financial services environments. Responsibilities Lead the design, development, and optimization of scalable data engineering solutions on Azure, using Spark-based processing (PySpark, SparkSQL, Scala) Own end-to-end data transformation pipelines, including ingestion, transformation, storage, and analytics Work with Azure-native data services such as Data Factory, Databricks, and Synapse Support high-performance data access patterns using Cosmos DB (NoSQL API) when applicable Collaborate with data scientists, AI engineers, and product stakeholders to enable data-driven analytics and insights Mentor and guide junior engineers, setting coding standards and best practices Ensure data quality, security, governance, and performance across platforms Contribute to technical decision-making and solution architecture discussions Requirements 3+ years of experience in data engineering roles, preferably in complex enterprise or financial services environments Expertise in Azure and Azure-native data services (Data Factory, Databricks, Synapse) Proficiency in the Spark ecosystem, including PySpark, SparkSQL, and Scala Spark Background in building and maintaining end-to-end data transformation pipelines covering ingestion, transformation, storage, and analytics Understanding of data quality, security, governance, and performance best practices Capability to contribute to architecture and solution design discussions Skills in mentoring engineering teams and establishing coding standards English proficiency at an Upper-Intermediate level (B2) or higher Nice to have Hands-on experience with Microsoft Fabric and OneLake (Delta / OpenLake) Familiarity with Cosmos DB (NoSQL API) Knowledge of financial instruments and financial services data Exposure to AI-assisted development tools (e.g., GitHub Copilot) and awareness of industry-standard LLMs Data Science fundamentals and collaboration experience with DS teams We offer CONTINUOUS UPSKILLING, LEARNING & DEVELOPMENT Diversity of tasks and projects Assessment center for objective review of competency level Personal development plan Mentoring programs and leadership development Certification and professional development support Access to learning platforms including more than 2,500 internal courses English courses taught by certified teachers CORPORATE BENEFITS Extra leave days Referral bonuses COMPENSATION PACKAGE Competitive compensation paid in USD Regular salary and performance reviews MEDICAL & HEALTHCARE Private health insurance Well-being events WORKING ENVIRONMENT Recreation areas and kitchens Tea, coffee and snacks Sports equipment and game consoles IT Equipment Microsoft’s Software Assurance Home Use Program (HUP) Please note that our Talent Attraction Team reviews applications and CVs submitted in English. EPAM is global leader in AI transformation engineering and integrated consulting, serving Forbes Global 2000 companies and ambitious startups. With over thirty years of expertise in custom software, product and platform engineering, we empower our clients to become AI-Native enterprises, driving measurable value from innovation and digital investments.