As a Senior Data Engineer at Provectus, you will tackle technical challenges in data engineering, collaborating with cross-functional teams to build data pipelines, manage large datasets, and contribute to open source projects while ensuring effective data governance and API development.
We are seeking a talented and experienced Data Engineer to join our team at Provectus. As part of our diverse practices, including Data, Machine Learning, DevOps, Application Development, and QA, you will collaborate with a multidisciplinary team of data engineers, machine learning engineers, and application developers. You will encounter numerous technical challenges and have the opportunity to contribute to Provectus’ open source projects, build internal solutions, and engage in R&D activities, providing an excellent environment for professional growth.
Requirements
- Experience in data engineering;
- Experience working with Cloud Solutions (preferably AWS, also GCP or Azure);
- Experience with Cloud Data Platforms (e.g., Snowflake, Databricks);
- Proficiency with Infrastructure as Code (IaC) technologies like Terraform or AWS CloudFormation;
- Experience handling real-time and batch data flow and data warehousing with tools and technologies like Airflow, Dagster, Kafka, Apache Druid, Spark, dbt, etc.;
- Proficiency in programming languages relevant to data engineering such as Python and SQL;
- Experience in building scalable APIs;
- Experience in building Generative AI Applications (e.g., chatbots, RAG systems);
- Familiarity with Data Governance aspects like Quality, Discovery, Lineage, Security, Business Glossary, Modeling, Master Data, and Cost Optimization;
- Advanced or Fluent English skills;
- Strong problem-solving skills and the ability to work collaboratively in a fast-paced environment.
Nice to Have:
- Relevant AWS, GCP, Azure, Databricks certifications;
- Knowledge of BI Tools (Power BI, QuickSight, Looker, Tableau, etc.);
- Experience in building Data Solutions in a Data Mesh architecture;
- Familiarity with classical Machine Learning tasks and tools (e.g., OCR, AWS SageMaker, MLFlow, etc.).
Responsibilities:
- Collaborate closely with clients to deeply understand their existing IT environments, applications, business requirements, and digital transformation goals;
- Collect and manage large volumes of varied data sets;
- Work directly with Data Scientists and ML Engineers to create robust and resilient data pipelines that feed Data Products;
- Define data models that integrate disparate data across the organization;
- Design, implement, and maintain ETL/ELT data pipelines;
- Perform data transformations using tools such as Spark, Trino, and AWS Athena to handle large volumes of data efficiently;
- Develop, continuously test and deploy Data API Products with Python and frameworks like Flask or FastAPI.
Top Skills
Python
SQL
Similar Jobs
Be an Early Applicant
As a Data Engineer at Provectus, you will build and maintain data pipelines, collaborate with Data Scientists, and create data models. Your role includes working with cloud platforms and various data tools to manage large data sets and support Generative AI applications and internal projects.
As an Associate in Product Analytics, you will help prioritize the team's direction, create analytical frameworks, assess potential opportunities, and translate complex datasets into actionable insights, all while enabling cross-functional collaboration across various departments.
Be an Early Applicant
The Analyst II position at Affirm involves owning the end-to-end analytics workflow, leading cross-functional metric reviews, and exploring data to enhance user engagement and profitability. This role emphasizes developing reporting tools and experiments while fostering a data-driven culture across the organization.
What you need to know about the Toronto Tech Scene
Although home to some of the biggest names in tech, including Google, Microsoft and Amazon, Toronto has established itself as one of the largest startup ecosystems in the world. And with over 2,000 startups — more than 30 percent of the country's total startups — Toronto continues to attract new businesses. Be it helping entrepreneurs manage their finances, simplifying business operations by automating payroll or assisting pharmaceutical companies in launching new drugs, the city's tech scene is just getting started.
