Principal AI Engineer, Trust & Evaluation
Notice: Equinix is aware of scams involving fake employment offers. Read more.
Principal AI Engineer, Trust & Evaluation
- JR-164772
- ハイブリッド
- Toronto
- Redwood City
- Technology
- Full time
Who are we?
Equinix is the world’s digital infrastructure company®, shortening the path to connectivity to enable the innovations that enrich our work, life and planet.
Job Summary
Sets the technical direction for the evaluation and trust capabilities of Equinix's GenAI platform, and designs, develops and deploys them. Builds platform features that let teams evaluate their agents at scale: golden datasets, synthetic test data, scoring with LLM-as-judge, and monitoring of agent quality in production. Designs and tests the trust controls that agents run on, including agent identity, access, guardrails and data boundaries. Leads the investigation of the hardest agent failures and fixes them in the platform. Serves as the hands-on technical lead of the Trust & Evaluation team, and partners with product, platform, security and governance leaders to turn ambiguous problems into robust, secure, observable and performant solutions.
Knowledge and Application Applies expert knowledge of software engineering practice and of LLM and agent concepts (prompting, retrieval, tool and function calling, orchestration, MCP). Has deep expertise in evaluation methods (code-based checks, LLM-as-judge, human review, golden datasets, regression suites) and evaluation statistics (sampling, variance across runs, agreement between reviewers). Has a deep understanding of AI security risks such as prompt injection and data leakage, and of identity concepts such as OAuth, token exchange and per-user access. Sets the standards and practices that the team and other teams follow, and keeps them current with industry research.
Complexity and Problem Solving Works on complex, open-ended problems that span teams, where the right approach is not yet known. Leads the investigation of the hardest agent failures (retrieval, tool call, prompt, model, data or infrastructure) and follows them until they are fixed. Decides which metrics reflect real quality and sets the bar for release. Makes architecture decisions with long-term impact on the platform, and resolves trade-offs among quality, cost, latency and security. Work is reviewed at design and milestone gates for results against long-term goals.
Collaboration and Interaction Partners with product, platform, security, legal and governance leaders, and with forward-deployed engineers and agent builders across the company. Turns the needs of agent builders into a platform roadmap. Explains results, risks and trade-offs to senior and non-technical stakeholders. Mentors engineers, raises the technical bar of the team, and leads through influence across teams.
Responsibilities
Evaluation platform (primary focus)
- Own the architecture and roadmap for platform evaluation features, and build them: dataset creation and versioning, golden datasets, synthetic test data and reusable metric templates
- Design and build the scoring approach for agents (code-based checks, LLM-as-judge, tool-call accuracy and grounding checks), and decide which methods apply where
- Design and build human-review features that compare judge scores against human reviewers, and set the agreement level required before teams rely on a judge
- Define how evaluations run at scale: before release, on every change to a prompt, model or tool, and on agents in production
- Define the evaluation standard and playbooks that every agent release follows, so agent builders can evaluate their agents without specialist help
- Keep results traceable to the agent, model, prompt and dataset versions tested, with a record of who approved each release, ready for audit
Trust and safety
- Design the platform's trust architecture and lead its build: agent identity, per-user access to connected systems, tool permissions, guardrails and data boundaries
- Lead adversarial testing and red-teaming (prompt injection, out-of-scope requests, data-boundary tests), and turn findings into fixes and regression tests
- Partner with security and governance leaders to turn policy requirements into automated checks
Root-cause analysis and quality
- Lead the investigation of the hardest agent failures and performance problems end to end, and follow them until the cause is fixed
- Set the approach for test and performance tooling: end-to-end agent tests, load and latency tests, and failure diagnostics
- Make sure production failures and user feedback become regression tests, so the same defect does not return
Quality and responsible practice
- Write tested, reviewed and documented code, contribute to shared libraries and tooling, and review the team's designs and code
- Make sure the team tests and builds to security, data-privacy and responsible-AI requirements
Qualifications
Required
- 7+ years in software, ML or platform engineering, including 3+ years building ML or LLM systems in production
- Bachelor's degree in computer science, engineering, data science, mathematics or a related field, or equivalent experience
- Expert Python, with solid habits: version control, testing, code review. Track record of designing backend services or platform features used by many teams
- Track record of designing and leading the build of evaluation, testing or quality platforms for ML or LLM applications, used by many teams or at scale
- Deep knowledge of LLM applications and agents (prompting, retrieval, tool calling) and how they fail, including tracing hard failures to their root cause through logs and traces
- Deep knowledge of evaluation statistics (sampling, confidence intervals, variance across runs) and of measuring agreement between reviewers
- Ability to write SQL and analyze results
- Experience leading technical work across teams and influencing without direct authority
- Clear written and verbal communication, including with senior and non-technical stakeholders
Nice to have
- Experience with LLM evaluation and observability tools, and tracing standards such as OpenTelemetry
- Experience with red-teaming or adversarial testing of LLM systems
- Experience with identity and access (OAuth 2.0, OIDC, delegated access)
- Experience with performance, load or test engineering
- Experience with agent frameworks, MCP or A2A
- Experience with Google Cloud (Vertex AI, GKE) and Kubernetes
- Familiarity with AI risk frameworks (NIST AI RMF, ISO 42001)
- Published research, patents, talks or open-source contributions in evaluation or AI safety
- MS or PhD in a related field
What success looks like
- The evaluation and trust roadmap is delivered on schedule, with tests and documentation
- Teams can evaluate their agents on the platform without specialist help
- Every agent release carries an evaluation record with criteria, results, open defects and approver
- Judge scores agree with human reviewers, and teams use them to make release decisions
- Trust controls are enforced by the platform, covered by automated tests, and pass security and governance reviews
- Defects found in evaluation become regression tests, and the same defect does not return
- Engineers on the team grow in scope and skill, and the team's standards are adopted by other teams across Equinix
The targeted pay range for this position in the following location is / locations are:
Canada - Toronto Office TRO : 158,000 - 236,000 CAD / Annual
United States - Redwood City Office GHQ : 246,000 - 370,000 USD / Annual
Our pay ranges reflect the minimum and maximum target for new hire pay for the full-time position determined by role, level, and location.The pay range shown is based on our compensation structure in place at the time of posting and may be updated periodically based on business needs. Individual pay is based on additional factors including job-related skills, experience, and relevant education and/or training.
The targeted pay range listed reflects the base pay only and does not include bonus, equity, or benefits. Employees are eligible for bonus, and equity may be offered depending on the position.
Equinix Benefits
As an employee, you become important to Equinix’s success. We ensure all your benefits are in line with our core values: competitive, inclusive, sustainable, connected and efficient. We keep them competitive within the current marketplace to ensure we’re providing you with the best package possible. So, wherever you are in your career and life, you’ll be able to enhance your experience and bring your whole self to work.
Employee Assistance Program: An Employee Assistance program is available to all employees.
US Benefits: - Insurance: You may enroll in health, life, disability and voluntary plans that are designed for you and your eligible family members. - Retirement: You and Equinix may contribute to a retirement plan to help you plan for your financial future. - Paid Time Off (PTO) and Paid Holidays: You will receive an accrued amount of PTO each pay period along with various paid holidays for you to rest and recharge. Eligibility requirements apply to some benefits. Benefits are subject to change and may be subject to specific plan or program terms. Canada Core Benefits: - Insurance: You may enroll in healthcare coverage that is designed to complement the provincial healthcare system, along with life, disability and optional benefit plans that are designed for you and your eligible family members. - Retirement: You may also enroll in Equinix-sponsored retirement or savings plans: Defined Contribution Pension Plan (DCPP), Group Retirement Savings Plan (RRSP) and Tax-Free Savings Plan (TSFA). - Vacation and Paid Holidays: Equinix offers both vacation and personal time, along with various paid holidays for you to rest and recharge. Eligibility requirements apply to some benefits. Benefits are subject to specific plan or program terms, and to change at Equinix discretion.Equinix is committed to ensuring that our employment process is open to all individuals, including those with a disability. If you are a qualified candidate and need assistance or an accommodation, please let us know by completing this form.
Equinix is an Equal Employment Opportunity and, in the U.S., an Affirmative Action employer. All qualified applicants will receive consideration for employment without regard to unlawful consideration of race, color, religion, creed, national or ethnic origin, ancestry, place of birth, citizenship, sex, pregnancy / childbirth or related medical conditions, sexual orientation, gender identity or expression, marital or domestic partnership status, age, veteran or military status, physical or mental disability, medical condition, genetic information, political / organizational affiliation, status as a victim or family member of a victim of crime or abuse, or any other status protected by applicable law.
We use artificial intelligence in our hiring process. Learn more here.
This posting is a new position within our organization.