Develop and operate scalable enterprise data solutions on the Ontario Data Platform. Responsibilities include building batch and near-real-time pipelines with Azure Data Factory, Databricks, ADLS, Spark, and Delta Lake; implementing medallion architecture, data quality controls, governance, and integrations; automating deployments through Azure DevOps and CI/CD; troubleshooting workloads; documenting solutions; and collaborating with technical and business stakeholders in Agile teams.
Role Overview:
- The Government of Ontario is undertaking a large-scale Data
& AI transformation initiative, with the Ontario Data Platform (ODP)
serving as the enterprise technology ecosystem for data-driven decision-making,
analytics, and artificial intelligence. ODP provides a modern and scalable data
environment supporting data ingestion, storage, processing, governance,
quality, marketplace capabilities, and advanced analytics across ministries and
government organizations.
- We are seeking a Data Engineer / Software Developer to
support the design, development, and operation of enterprise data solutions on
the Ontario Data Platform. The successful candidate will work with modern cloud
technologies, including Databricks, Azure Data Factory (ADF), and Azure Data
Lake Storage (ADLS), to build scalable data pipelines, enable analytics use
cases, and support the delivery of high-quality data products across
government.
Requirements
Experience and Skill Set Requirements:
Must Haves:
- Degree or diploma in Computer Science, Software Engineering,
Information Technology, or a related field.
- Experience developing data solutions using Azure Data
Factory (ADF), Azure Data Lake Storage (ADLS), Databricks, Delta Lake and
Lakehouse architecture principles.
- Experience working with Azure DevOps, Git, CI/CD pipelines,
and automated deployment practices.
- Strong proficiency in Python, SQL, and Spark-based data
processing frameworks.
- Knowledge of data governance, security, privacy, and access
control principles.
- Experience with REST APIs, application integration patterns,
and automation frameworks.
Skill Set Requirements:
Key Responsibilities:
Data Engineering & Pipeline Development
- Design, develop, and maintain scalable batch and near
real-time data pipelines using Azure Data Factory (ADF), Databricks, and
related cloud technologies.
- Develop data ingestion frameworks to onboard data from
various source systems into the Ontario Data Platform.
- Build and optimize ETL/ELT processes to transform, cleanse,
and prepare data for analytics and reporting.
- Implement data validation, monitoring, and reconciliation
processes to ensure data quality and reliability.
- Databricks Development
- Develop and maintain notebooks, workflows, and data
processing jobs using Databricks.
- Build data transformation pipelines leveraging Spark and
Databricks Lakehouse architecture.
- Support the implementation of medallion architecture
patterns (Bronze, Silver, Gold).
- Monitor and troubleshoot Databricks jobs, workflows, and
performance issues.
- Develop and manage Delta Lake tables and optimize data
processing workloads for performance and scalability.
- Data Storage & Integration
- Design and manage data solutions utilizing Azure Data Lake
Storage (ADLS).
- Collaborate with source system teams to establish secure and
efficient data integration patterns.
- Implement data partitioning, optimization, and archival
strategies to improve performance and scalability.
- Support metadata management, data cataloguing, and
governance initiatives.
- Software Development & Automation
- Develop reusable frameworks, scripts, and utilities to
automate data engineering processes.
- Participate in DevOps practices, including CI/CD pipelines,
code reviews, testing, and deployment automation.
- Maintain technical documentation, design specifications, and
operational procedures.
- Contribute to platform enhancements and continuous
improvement initiatives.
- Apply software engineering best practices including version
control, test automation, code quality standards, and secure coding principles.
- Collaboration & Stakeholder Engagement
- Work closely with architects, product managers, data
scientists, analysts, and business stakeholders to understand requirements and
deliver technical solutions.
- Participate in Agile teams and contribute to sprint
planning, estimation, and delivery activities.
- Provide technical guidance and knowledge sharing to team
members and partners.
- Participate in solution architecture reviews and contribute
to technology roadmaps and design decisions.
Evaluation breakdown:
- Technical(Power BI)
- Communication
- Public Sector exp
Similar Jobs
Blockchain • eCommerce • Fintech • Payments • Software • Financial Services • Cryptocurrency
Own end-to-end analytics engineering for AI and developer experience data. Instrument telemetry, build governed production pipelines, define trusted metrics, and create executive-facing dashboards and analyses. Partner with data engineering, AI, and data science teams to improve data quality and connect raw signals to decision-making. Present clear recommendations on AI investment, developer productivity, and ROI while using AI coding agents to build and maintain pipelines, dashboards, and analyses.
Top Skills:
Ai Coding AgentsCi/CdGitLlm-Based Classification ToolingMetrics StorePythonReactSnowflakeSQLTypescript
2 Hours Ago
Blockchain • eCommerce • Fintech • Payments • Software • Financial Services • Cryptocurrency
Graduate research interns will own end-to-end projects to build customer world models and proactive intelligence: developing representation learning, foundation model, RL/agentic methods, evaluation frameworks, and shipping research into production to improve customer decision-making.
Top Skills:
Deep LearningFoundation ModelsJaxOptimizationPythonPyTorchRepresentation Learning
Blockchain • eCommerce • Fintech • Payments • Software • Financial Services • Cryptocurrency
Build and deploy agentic and generative AI systems that automate risk and compliance investigations. Own full-stack ML systems, including data integrations, prompt tuning, orchestration, evaluation, productionization, monitoring, and transformer-based modeling across transaction, graph, text, and profile data. Collaborate with product, engineering, platform, and operations teams while leading strategic roadmaps and delivering scalable production systems.
Top Skills:
Agentic AiAmazon Web Services (Aws)Apache AirflowGenerative AiGoogle Cloud Platform (Gcp)Large Language Models (Llms)Natural Language Processing (Nlp)PythonPyTorchSnowflakeTransformer Models
What you need to know about the Toronto Tech Scene
Although home to some of the biggest names in tech, including Google, Microsoft and Amazon, Toronto has established itself as one of the largest startup ecosystems in the world. And with over 2,000 startups — more than 30 percent of the country's total startups — Toronto continues to attract new businesses. Be it helping entrepreneurs manage their finances, simplifying business operations by automating payroll or assisting pharmaceutical companies in launching new drugs, the city's tech scene is just getting started.
