Design, orchestrate, and optimize large-scale LLM pre-training across 1,000+ GPUs. Implement 3D parallelism, manage GPU clusters (SLURM/Kubernetes), optimize InfiniBand/RDMA networking and memory, and automate checkpointing and failure recovery for long training runs.
We are seeking a highly skilled LLM Pre-training & Distributed Systems Engineer. This role is essential for orchestrating large-scale machine learning training runs and optimizing distributed infrastructure. The ideal candidate will have a deep understanding of GPU clusters and extensive experience in system engineering to ensure efficient and reliable training processes.
Responsibilities:
- Orchestrate distributed training runs across 1,000+ GPUs using PyTorch, DeepSpeed, or Megatron-LM.
- Optimize networking (InfiniBand/RDMA) and memory management to prevent out-of-memory errors.
- Automate checkpointing and failure recovery during month-long training runs.
Required Skills:
- Deep expertise in 3D parallelism (Data, Tensor, Pipeline).
- Experience managing SLURM or Kubernetes-based GPU clusters.
- Strong systems engineering background (C++, CUDA, Python).
Similar Jobs
HR Tech • Information Technology • Professional Services • Sales • Software
Own the full sales cycle for mid-market SaaS customers in Canada, from prospecting and relationship development through product demonstrations, forecasting, negotiation, and closing. Build pipeline through outbound activity, engage decision-makers, assess buying readiness, manage sales data in Salesforce, and network with consultants, partners, influencers, and industry events. Collaborate with Sales Engineers and Business Development representatives to drive new business.
Top Skills:
HrisSaaSSalesforce
Digital Media • Gaming • Information Technology • Software • Sports • Esports • Big Data Analytics
Develop and maintain relationships with VIP gaming customers, drive sales metrics and net revenue, promote loyalty, and deliver exceptional player experiences. The role executes responsible gaming policies, analyzes player data and trends, recommends customer solutions, and resolves escalations with internal teams. Candidates need a related bachelor's degree or five years of relevant gaming, hospitality, sales, or marketing experience, plus the ability to obtain and maintain required gaming licenses.
Top Skills:
AI
Cloud • Software
Own end-to-end B2B demand generation, including integrated campaigns, ABM, email nurture, organic social, website initiatives, conversion optimization, and performance reporting. Partner with Sales, Product Marketing, Customer Success, agencies, design, and product teams to generate measurable pipeline. The role requires strategic planning, hands-on execution, data-driven optimization, strong writing, and management of multiple concurrent programs, with emphasis on SaaS, technical audiences, open source, and product-led growth.
Top Skills:
Claude CodeHubspotLinkedIn
What you need to know about the Toronto Tech Scene
Although home to some of the biggest names in tech, including Google, Microsoft and Amazon, Toronto has established itself as one of the largest startup ecosystems in the world. And with over 2,000 startups — more than 30 percent of the country's total startups — Toronto continues to attract new businesses. Be it helping entrepreneurs manage their finances, simplifying business operations by automating payroll or assisting pharmaceutical companies in launching new drugs, the city's tech scene is just getting started.
