HeyMilo AI Logo

HeyMilo AI

Technical Customer Success Manager (AI Tuning)

Reposted 24 Days Ago
Be an Early Applicant
Hybrid
Toronto, ON, CAN
Mid level
Hybrid
Toronto, ON, CAN
Mid level
The role involves ensuring the performance of AI agents in production, defining success metrics with clients, and debugging issues.
The summary above was generated by AI

HeyMilo is building AI interviewers that run real conversations with candidates. We help companies screen faster and make better hiring decisions using conversational AI.

We work closely with customers to deploy AI into real hiring workflows and ensure it delivers measurable outcomes.

The Role

We’re hiring a Forward Deployed AI Engineer to own the performance and success of AI agents in production.

This role sits at the intersection of engineering, product, and customer success. You’ll work directly with customers post-deployment to ensure their AI agents are performing at a high level and continuously improving.

What you’ll do

  • Work closely with customers and customer success to define and track success metrics

  • Monitor performance of deployed AI agents and identify gaps

  • Tune prompts, workflows, and agent behavior to improve outcomes

  • Iterate on agent logic, tool usage, and context management

  • Debug issues across prompts, integrations, and system behavior

  • Translate real-world usage into product and system improvements

What we’re looking for

  • Strong technical background with ability to code when needed

  • Deep experience with prompt engineering and LLM behavior

  • Strong understanding of agent systems (tool calling, context management, orchestration)

  • Heavy user of AI tools (Cursor, Claude, Codex, etc.)

  • Ability to debug ambiguous issues across AI and system layers

  • Strong ownership and communication skills

  • Comfortable working under pressure and tight timelines

  • Able to own outcomes, not just tasks

  • Experience implementing agentic workflows or AI agents in enterprise environments

  • Programming fundamentals + Python/JS experience are a huge plus

Bonus

  • Experience with conversational AI across voice, SMS, or chatbots

  • Experience with telephony or voice systems

  • Experience evaluating and improving production AI systems

  • Familiarity with recruiting, HR tech, or workflow automation

Why join

  • Direct ownership over how AI performs in the real world

  • Work on high-impact problems tied to customer outcomes

  • High visibility across product, engineering, and customer teams

  • Fast-paced, execution-driven environment

Similar Jobs

11 Minutes Ago
Hybrid
Toronto, ON, CAN
Mid level
Mid level
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Develop secure technology solutions for financial institutions, focusing on vulnerability management. Responsibilities include full-stack engineering, collaboration, and Agile methodologies.
Top Skills: AngularAWSAzureDockerGCPJava SpringKubernetesOpenshiftReact
11 Minutes Ago
Hybrid
Toronto, ON, CAN
Junior
Junior
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
The Junior Full Stack Developer will assist in delivering vulnerability remediation initiatives for financial institutions, working with various teams on development, testing, and production support activities.
Top Skills: AngularAWSAzureDockerGCPJava SpringKubernetesOpenshiftReact
11 Minutes Ago
Easy Apply
Remote or Hybrid
Canada
Easy Apply
Senior level
Senior level
eCommerce • Healthtech • Kids + Family • Retail • Social Media
The Senior Mobile Engineer will lead mobile app projects, collaborate with teams, and enhance the Babylist Health experience. Responsibilities include building features, improving performance, and ensuring code quality.
Top Skills: Auto LayoutAWSJetpack ComposeJetpack LibrariesMySQLRedisRuby On RailsSidekiqSwiftuiUikit

What you need to know about the Toronto Tech Scene

Although home to some of the biggest names in tech, including Google, Microsoft and Amazon, Toronto has established itself as one of the largest startup ecosystems in the world. And with over 2,000 startups — more than 30 percent of the country's total startups — Toronto continues to attract new businesses. Be it helping entrepreneurs manage their finances, simplifying business operations by automating payroll or assisting pharmaceutical companies in launching new drugs, the city's tech scene is just getting started.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account