Lead real-time data integration using Azure Databricks, PySpark, Kafka, ADF, and ADLS. Build and optimize medallion-style ETL pipelines processing over 10 million daily records. Improve transformation efficiency, data quality, validation, and downstream reliability by identifying upstream data issues, implementing root-cause fixes, and collaborating with stakeholders.
Real-time data pipelines using Azure Databricks, PySpark, Kafka, Azure Data Factory (ADF), and Azure Data Lake Storage (ADLS), enabling reliable ingestion and transformation of 10M+ records daily. • Developed and optimized SQL and PySpark transformations within medallion-style ETL workflows, improving processing efficiency, maintainability, and data quality for downstream analytics. • Identified requirement gaps and data quality issues stemming from upstream data sources and solutioned root-cause fixes by implementing validation rules, refining transformation logic, and collaborating with stakeholders to improve downstream data reliability.
Hexaware Technologies Mississauga, Ontario, CAN Office
600 Matheson Blvd West Unit 5, Mississauga, Canada, L5R 4C1
Similar Jobs
Information Technology • Consulting
Leads hands-on data integration and engineering initiatives using Azure Databricks, Azure Data Factory, PySpark, Python, and SQL. Modernizes legacy on-premises systems into cloud-native architectures, implements Medallion Architecture and data warehousing, improves pipeline reliability and compute efficiency, develops CI/CD workflows, and mentors engineering teams delivering secure, actionable analytics for banking environments.
Top Skills:
AzureAzure Data FactoryAzure DatabricksCi/CdData WarehousingMedallion ArchitecturePysparkPythonSQL
Information Technology • Consulting
Leads hands-on data integration and engineering initiatives using Azure Databricks, Azure Data Factory, PySpark, Python, and SQL. Modernizes legacy on-premises systems into cloud-native architectures, implements Medallion Architecture and data warehousing, improves pipeline reliability and compute efficiency, establishes CI/CD workflows, supports banking-grade security, and mentors engineering teams delivering actionable analytics.
Top Skills:
Azure Data FactoryAzure DatabricksCi/CdData WarehousingMedallion ArchitecturePysparkPythonSQL
Artificial Intelligence • Information Technology • Professional Services • Consulting
Designs, develops, and maintains end-to-end integrations between Oracle EPM Cloud applications and source or target systems. Responsibilities include data mapping, transformation, validation, automation with scripts and REST APIs, integration monitoring, troubleshooting, stakeholder collaboration, and technical documentation. The role requires extensive Oracle EPM integration experience, SQL proficiency, scripting or development skills, and familiarity with testing and Oracle Integration Cloud.
Top Skills:
Ansi SqlArcsC#Cloud Data ManagementEpbcsEpm AutomateEpm Integration AgentFccsFdmeeGroovyJavaJythonOdiOracle Epm CloudOracle Epm Rest ApisOracle Integration CloudOracle SqlPcmcsPipelinesPowershellPythonRelational DatabasesT-SqlVbscript
What you need to know about the Toronto Tech Scene
Although home to some of the biggest names in tech, including Google, Microsoft and Amazon, Toronto has established itself as one of the largest startup ecosystems in the world. And with over 2,000 startups — more than 30 percent of the country's total startups — Toronto continues to attract new businesses. Be it helping entrepreneurs manage their finances, simplifying business operations by automating payroll or assisting pharmaceutical companies in launching new drugs, the city's tech scene is just getting started.
