Senior Engineer – AI Platform

Role Overview We are looking for a hands-on Senior Engineer to design, build, and operate an enterprise AI platform for a digital bank. Our goal is to provide a fully governed, safe, and efficient AI environment that staff actually want to use. You will take ownership across three core areas: an enterprise-grade inference gateway, identity and distribution systems, and self-hosted GPU inference clusters. Core Responsibilities Inference Serving & On-Premise GPU Operations Size hardware capacity and collaborate with infrastructure teams to handle hypervisors, rack/power setups, and GPU passthrough across bare-metal or data center environments. Deploy, operate, and fine-tune model serving frameworks using techniques like continuous batching, quantization, tensor parallelism, and KV cache optimization. Implement tenancy admission controls, quota management, queuing, and failover mechanisms to cloud inference when local capacity is saturated. Track and publish key performance indicators—such as p95 time-to-first-token, tokens per second, and cost per million tokens—to evaluate fixed OpEx break-even points against cloud alternatives. Governed Gateway Architecture & Identity Integration Construct and maintain an inference gateway that handles rate limiting, per-user key management, model allowlists, and strict zero data retention policies. Connect governance systems directly to corporate single sign-on (OIDC / OAuth2) and automate onboarding, offboarding, and role claims via SCIM. Ensure total financial transparency by attributing real-time API spend, token usage, and operational costs down to individual users and departments. Tooling Distribution & User Enablement Package, distribute, and manage agent configurations and execution harnesses across a diverse enterprise machine fleet. Maintain a secure, CI-driven registry for versioned skills, knowledge-base bundles, and agent modules featuring seamless rollback capabilities. Onboard, train, and support non-technical teams (such as risk, finance, and operations) to maximize productivity using approved AI tools. What We Are Looking For Essential Qualifications & Skills Production Platform Engineering: 5+ years of experience managing critical production infrastructure, with direct involvement in incident response and on-call rotations. Core Technical Stack: Proficiency in Python (primary) and TypeScript, alongside hands-on experience with Kubernetes, Linux, containers, Infrastructure as Code, and CI/CD pipelines. Enterprise Identity Systems: Strong grasp of JWT claims, OIDC, OAuth2, SCIM protocols, and enterprise directory integrations. Gateway & Multi-Tenancy Architecture: Background in building or maintaining API/LLM gateways, rate limits, quota systems, usage metering, and multi-tenant isolation. On-Premise Infrastructure Comfort: Experience navigating capacity planning, hypervisors, hardware passthrough, and enterprise change management processes. FinOps & Attribution Mindset: Ability to evaluate platforms through unit economics, capacity utilization, and cost attribution. Technical Enablement: Skill in simplifying complex AI tools and training non-technical colleagues to use them effectively. Preferred / Bonus Qualifications Experience with Model Context Protocol (MCP), sovereign hosting constraints, model evaluation, or highly regulated financial services environments. Hands-on expertise tuning high-throughput GPU model serving stacks such as vLLM, SGLang, Triton, or TGI. Platform Guiding Principles Usability First: Approved pathways must offer a better user experience than unapproved workarounds to prevent control bypasses. Attribution Before Scale: Total visibility into spend and usage per user is required before expanding system scale. Allocated Capacity: GPU resources are scarce and access is governed through explicit capacity allocation policies. Pragmatic Ownership: Open-source or licensed solutions are used for standard components, while internal focus is dedicated to security and governance boundaries.