Lead real-time data integration using Azure Databricks, PySpark, Kafka, ADF, and ADLS. Build and optimize medallion-style ETL pipelines processing over 10 million daily records. Improve transformation efficiency, data quality, validation, and downstream reliability by identifying upstream data issues, implementing root-cause fixes, and collaborating with stakeholders.
Real-time data pipelines using Azure Databricks, PySpark, Kafka, Azure Data Factory (ADF), and Azure Data Lake Storage (ADLS), enabling reliable ingestion and transformation of 10M+ records daily. • Developed and optimized SQL and PySpark transformations within medallion-style ETL workflows, improving processing efficiency, maintainability, and data quality for downstream analytics. • Identified requirement gaps and data quality issues stemming from upstream data sources and solutioned root-cause fixes by implementing validation rules, refining transformation logic, and collaborating with stakeholders to improve downstream data reliability.
Hexaware Technologies Mississauga, Ontario, CAN Office
600 Matheson Blvd West Unit 5, Mississauga, Canada, L5R 4C1
Similar Jobs
Information Technology • Consulting
Leads data integration development using Azure, Hadoop, Spark, Scala, Java, SQL, and relational databases. Responsibilities include data modeling, analytics, solution design, requirements analysis, development, maintenance, and quality delivery in an agile, client-focused environment. The role requires strong problem-solving, interpersonal, communication, and technical skills, along with adaptability, motivation, and the ability to learn quickly.
Top Skills:
AgileSparkAzure CloudAzure Data Factory (Adf)Azure Data Lake Storage (Adls)Azure Databricks (Adb)Data AnalyticsData ModelingDelta LakeHadoopHbaseHdfsHiveIbm Db2J2EeJavaJdbcJSONMapreduceMicrosoft Sql ServerOracleScalaSpark SqlSpringSqoopWeb ServicesXMLYarn
Information Technology • Consulting
Leads hands-on data integration and engineering initiatives using Azure Databricks, Azure Data Factory, PySpark, Python, and SQL. Modernizes legacy on-premises systems into cloud-native architectures, implements Medallion Architecture and data warehousing, improves pipeline reliability and compute efficiency, develops CI/CD workflows, and mentors engineering teams delivering secure, actionable analytics for banking environments.
Top Skills:
AzureAzure Data FactoryAzure DatabricksCi/CdData WarehousingMedallion ArchitecturePysparkPythonSQL
Information Technology • Consulting
Leads hands-on data integration and engineering initiatives using Azure Databricks, Azure Data Factory, PySpark, Python, and SQL. Modernizes legacy on-premises systems into cloud-native architectures, implements Medallion Architecture and data warehousing, improves pipeline reliability and compute efficiency, establishes CI/CD workflows, supports banking-grade security, and mentors engineering teams delivering actionable analytics.
Top Skills:
Azure Data FactoryAzure DatabricksCi/CdData WarehousingMedallion ArchitecturePysparkPythonSQL
What you need to know about the Toronto Tech Scene
Although home to some of the biggest names in tech, including Google, Microsoft and Amazon, Toronto has established itself as one of the largest startup ecosystems in the world. And with over 2,000 startups — more than 30 percent of the country's total startups — Toronto continues to attract new businesses. Be it helping entrepreneurs manage their finances, simplifying business operations by automating payroll or assisting pharmaceutical companies in launching new drugs, the city's tech scene is just getting started.
