Top Tech Jobs & Startup Jobs in Toronto

The Software Engineer will lead the development of Baseten Voice AI, focusing on real-time model serving systems, collaboratng with teams on architecture and implementation.

Lead design and build of a real-time observability and root-cause analysis system for high-density GPU fabrics. Collect, reduce, correlate telemetry from switches, hosts, NICs, GPUs, and services; model topology and ownership; separate fabric faults from host/driver/RDMA/scheduler issues; and produce operator workflows for triage and remediation.

As a Senior Software Engineer, you will manage Baseten's billing and revenue infrastructure, ensuring reliability and collaboration with finance and sales teams while evolving the billing platform.

Own and operate Baseten's internal AI stack, build and deploy custom AI agents and LLM-powered tools, evaluate third-party AI developer tools, instrument usage and measure impact on engineering productivity, and support engineering teams in adopting AI workflows.

As a Forward Deployed Engineer, you will develop and maintain ML solutions, collaborate with teams on AI/ML pipelines, and manage projects end-to-end.

Lead and mentor a team of Forward Deployed Engineers in building and optimizing LLM inference workloads, delivering high-performance AI applications, and collaborating with cross-functional teams.

The role involves developing and managing infrastructure for distributed LLM inference systems, ensuring performance and usability, debugging complex systems, and collaborating with performance engineers.

As a Software Engineer, you'll build and maintain Baseten's core product, design APIs, implement features, fix issues, and collaborate with teams.

Design and implement RDMA/InfiniBand networking primitives for distributed GPU inference, optimize communication for Disaggregated KV Cache and WideEP, enable sub-10s model startup, validate hardware performance on H100/H200/B200/B300/GB300 clusters, build observability tools, and optimize or author communication kernels (NCCL/NVSHMEM/UCX).

Develop and maintain components for machine learning inference platform. Focus on Kubernetes deployments, resource management, and performance monitoring.