Design, build, automate, optimize, and support complex cloud data pipelines and ETL/ELT processes. Develop unified Databricks data platforms, data lakes, lakehouses, warehouses, and dimensional models. Integrate and migrate data across Oracle, Azure, Databricks, and Microsoft Fabric. Create Azure Data Factory pipelines, implement CI/CD and DataOps practices, and perform performance tuning, monitoring, troubleshooting, and data provisioning for large-scale enterprise environments.
Responsibilities:
- Designing and developing data pipelines from source to end
user
- Optimizing data pipelines
General Skills:
- Experience with Cloud data platforms, data management and
data exchange tools and technologies
- Experience with Commercial and opensource data and database
development and management and specializing in data storage setting and
managing cloud Data As a Service (DaaS). application Database as a Service
(DBaaS), Data Warehouses as a Service (DWaaS), and other storage platforms
(both in the cloud and on-premise).
- Experience with Data pipeline and workflow development,
orchestration, deployment, and automation and specializing in programming and
pipelines to create and managing the dataflow and movement of data.
- Experience with Cloud data engineer must be familiar and
experienced with different programming languages and be able to integrate with
many different platforms to create data pipelines, automate tasks, and write
scripts.
- Experience in DataOPS principle, best practice, and
implementation and Agile project development and deployment
- Experience in Continuous Integration/Continuous
Development/Deployment (CI/CD) and Data provisioning Automation
- Experience with digital product, data analysis, data
exchange, data provisioning, and data security
- Extensive expert experience in designing/developing and
implementing data conversation and migration of VLD (Very large Data) of Online
analytical processing (OLAP) and online transaction processing (OLTP)
environments to Cloud Software-as-a-service (SaaS), Platform-as-a-service
(PaaS) and Infrastructure-as-a-service (IaaS) environments.
- Experience in design, development and implementation of
fact/dimension model, data mapping, data warehouse, data lake and data
lakehouse for enterprise
- Experience managing Cloud Data services for project
delivery, including storage, repositories, Data Lake, Data Lakehouse, key
vault, virtual machine, disk, etc.
- Experience with structured, semi-structured, unstructured
data collection, ingestion, provisioning and exchange technological development
of enterprise data warehouse and data lake and data lakehouse solutions and
operational support.
- Experience with DataOPS performance monitoring and tuning
- Excellent analytical, problem-solving and decision-making
skills; verbal and written communication skills; presentation skills;
interpersonal and negotiation skills
- A team player with a track record for meeting deadlines
Requirements
Experience and Skill Set Requirements:
Must Haves:
- Design,
build, automate and optimize complex data ETL/ELT processes using
Databricks, Delta Live Tables (DLT) and equivalent.
- Development
of unified data platforms using Databricks and Unity catalog or its
equivalent.
- Relational
databases (Oracle, MySQL, SQL Server), data modelling (relational &
dimensional), advanced SQL, query optimization, data replication,
administration.
- Advanced
SQL skills (PL/SQL, TSQL).
- Big
Data processing Frameworks (PySpark).
- Proficient
in creating Azure Data Factory pipelines for copy activity and custom data
pipelines in Databricks with CI CD (Azure DevOPS Git Integration) .
- Data
migration and data integration across different platforms Oracle to Azure
Data Lake, Oracle to Databricks, Oracle to Microsoft Fabric.
Skill Set Requirements:
Mandatory Skills and Experience (8+ years experience):
- Design,
build, automate and optimize complex data ETL/ELT processes using
Databricks, Delta Live Tables (DLT) and equivalent.
- Development
of unified data platforms using Databricks and Unity catalog or its
equivalent.
- Relational
databases (Oracle, MySQL, SQL Server), data modelling (relational &
dimensional), advanced SQL, query optimization, data replication,
administration.
- Advanced
SQL skills (PL/SQL, TSQL).
- Big
Data processing Frameworks (PySpark).
- Proficient
in creating Azure Data Factory pipelines for copy activity and custom data
pipelines in Databricks with CI CD (Azure DevOPS Git Integration) .
- Data
migration and data integration across different platforms Oracle to Azure
Data Lake, Oracle to Databricks, Oracle to Microsoft Fabric.
Desirable Skills and Experience:
- Data
lake design and development with focus on extracting, transforming data
coming from various data sources and loading into Lakehouse in medallion
architecture.
- Advanced
skills and hands-on experience with Oracle databases, Azure SQL Server and
Azure Data Factory.
- Cloud
technologies (Azure, Google, AWS).
- PL/SQL
for data extraction, transformation, and loading, initial setup and other
ETL experience management and support, such as troubleshooting,
performance tuning, failover and recovery.
- Scripting
languages (Python, Unix shell, Scala).
Similar Jobs
Fintech • Machine Learning • Payments • Software • Financial Services
Leads fulfillment and customer experience process operations for credit card customers. The role drives operational excellence, compliance, automation, data-informed transformation, vendor partnerships, performance management, and cross-functional initiatives. This leader establishes management systems for KPIs, quality, capacity, and budgets while developing high-performing operational teams and influencing enterprise strategy.
Top Skills:
AgileAIAlteryxLean Six SigmaPower BISaaSSQL
Information Technology
Drives revenue growth through partner-focused sales initiatives, pipeline creation, opportunity development, and strategic business development. Serves as a subject matter expert and liaison among sales teams, solution architects, customers, and technology partners. Delivers sales enablement, manages partner relationships, supports quoting and pricing, advances opportunities, and executes joint go-to-market initiatives while meeting revenue, profit, and customer acquisition targets.
Top Skills:
Salesforce CRM
Information Technology
Drives Apple sales growth by supporting account managers, advising customers, managing partner relationships, and providing Apple product expertise. Supports presales consultations, opportunity development, quotes, customer presentations, sales enablement, training, marketing campaigns, partner events, and pipeline reporting. Builds relationships with Apple channel teams and promotes Apple hardware, software, deployment, lifecycle, financing, and professional services solutions.
Top Skills:
AppleApple Business ManagerApplecareCrm ToolsMicrosoft 365Mobile Device Management (Mdm)
What you need to know about the Toronto Tech Scene
Although home to some of the biggest names in tech, including Google, Microsoft and Amazon, Toronto has established itself as one of the largest startup ecosystems in the world. And with over 2,000 startups — more than 30 percent of the country's total startups — Toronto continues to attract new businesses. Be it helping entrepreneurs manage their finances, simplifying business operations by automating payroll or assisting pharmaceutical companies in launching new drugs, the city's tech scene is just getting started.
