Chapter 13: Building the AI Ops Team

Key Qualities for Building an AI Operations Team

Building an effective AI Operations (AI Ops) team requires a unique blend of technical expertise, strategic thinking, and adaptability. This team must bridge the gap between AI capabilities and business goals, ensuring AI initiatives are both impactful and sustainable. Remember from earlier in this book the concept that AI has been in the back office of companies and then suddenly post-2022 it's in the front office. The AI Operations team needs to be a business/operations-biased team, not just a technical one.

Team building for AI Operations is going to look different based on the stage of AI Operations ("Wow," Easy Scale & Education, or AI Ready Data) and also different based on the company/industry/etc. So let's start by taking a look at the types of qualities you will need on the team instead of the actual roles and headcount.

STACK-B: The Core Qualities of an AI Ops Team

To structure these qualities, we introduce the STACK-B framework:

S – Systems Thinker: Understands complex relationships and foresees cross-departmental impacts.
T – Technologist: Ability to develop, implement, and optimize AI solutions.
A – Adaptable: Thrives in a fast-changing AI landscape, adjusting strategies as needed.
C – Communicative: Bridges AI teams and business leaders, articulating AI concepts effectively.
K – Knowledge-Seeker: Constantly exploring new technologies, questioning assumptions, and testing solutions.
B – Bold: Willing to take risks, challenge conventional thinking, and push boundaries in AI problem-solving.

Breaking Down the STACK-B Framework

Systems Thinker

At the core of a great AI Operations team is a systems thinker. In short, a systems thinker is someone who can see the whole, not just the parts. Systems thinking is not "systematic thinking" either. It's a way of solving problems and thinking that understands complex relationships. Your AI Operations team lead needs to approach challenges holistically, ensuring AI initiatives align with business goals without creating unintended negative impacts across departments.

Technologist

The nature of the AI Operations team and projects requires someone of a technical nature—a developer who can code and work with AI. In some cases, there will be multiple skill sets here—Infrastructure, DevOps, front-end, etc. It will differ for each company and each AI Operations stage, but there is no way of getting around needing someone with a technical skill set.

Adaptable

Being adaptable is highly important for AI Operations team members. Not just for the fast nature and context switching that will come with the role itself but to keep up with the broader AI landscape, which is evolving at a technological pace we haven't experienced before. It's not uncommon for some new capability, feature, or product to be released before a project is complete using a now outdated approach.

Communicative

Communicating well is essential for AI Operations. This is not just about being a good talker—it's about understanding the nuances of conversation, asking questions, guiding discussions, and digging deeper. The AI Ops team must work with multiple departments to understand their processes and collaborate effectively. Communication is equally critical for AI prompting, as generative AI thrives on structured and precise input. Effective communication builds trust, reduces fear of AI-related job displacement, and ensures project success.

Knowledge-Seeker

At its core, AI Operations is about solving problems, and that requires curiosity. Curiosity about how things work, possible solutions, and how those solutions perform. Whether during the interview process, development, or education stages, AI Ops team members need to be obsessed with acquiring and testing new information.

Bold

AI Operations teams need to be bold. They must be willing to take calculated risks, challenge established workflows, and experiment with new AI-driven solutions. Boldness fuels innovation, encouraging the team to push beyond conventional thinking and explore novel applications of AI that can drive business transformation.

The Role of the Business/Department Expert

One other characteristic to consider is one that may or may not reside on an AI Operations team: the Business/Department Expert. This person deeply understands the purpose and operations of the team the AI Operations team is collaborating with. In some companies, it may make sense to have this as a full-time AI Operations team member, and in others, it may be a designated liaison. Regardless, having deep knowledge of the specific function is critical to successful AI Operations outcomes.

Conclusion: STACK-B for AI Success

Building an AI Ops team isn't just about technical proficiency; it requires STACK-B—Systems Thinking, Technologist skills, Adaptability, Communication, Knowledge-Seeking, and Boldness. By ensuring your team embodies these characteristics, your organization will be well-equipped to drive AI initiatives that are innovative, practical, and impactful.

Building the AI Ops Team for Each Stage

Building an AI Operations (AI Ops) team is not a one-size-fits-all endeavor. The structure, skills, and focus of the team must evolve in alignment with the organization's stage of AI adoption. As discussed in Chapter 7, AI Ops adoption progresses through three distinct phases: the "Wow" Phase, the "Easy Scaling & Education" Phase, and the "AI-Ready Data" Phase. Each stage requires a different team composition, skill emphasis, and operational strategy.

AI Ops Team Composition Across Stages

Phase 1: The "Wow" Phase

At this initial stage, the primary goal is to generate excitement and showcase AI's potential through quick wins. The AI Ops team should be lightweight, highly experimental, and focused on rapid prototyping.

Low-Budget Team Structure

AI Ops Lead (Fractional or 1 FTE) – Drives AI strategy and ensures alignment with business needs.
Freelance AI Engineer (Contractor) – Develops quick prototypes and proof-of-concept AI tools.
AI Enthusiast (Internal Advocate) – A tech-savvy employee who helps promote AI within the company.

High-Budget Team Structure

AI Ops Lead (1 FTE) – Acts as a champion and strategist, ensuring alignment between AI efforts and business priorities.
AI Engineer (1 FTE) – Develops quick prototypes and proof-of-concept AI tools.
AI Evangelist (1 FTE or fractional) – Bridges the gap between AI and business teams, helping employees understand AI's capabilities.
AI Analyst (1 FTE or fractional) – Identifies key pain points where AI can drive efficiency, working closely with business units.

Key Focus Areas

Rapid development of AI-powered tools to showcase quick value.
Conducting AI demos and workshops to generate curiosity and excitement.
Identifying enthusiastic early adopters within the organization.
Gathering feedback to inform future AI Ops strategy.

Phase 2: Easy Scaling & Education

As AI adoption grows beyond early experiments, the AI Ops team must shift toward structured education, repeatable processes, and scalable deployment.

Low-Budget Team Structure

AI Ops Lead (1 FTE or fractional) – Focuses on AI education and structured deployment.
AI Engineer (1 FTE or fractional) – Enhances early AI solutions for scalability.
AI Champion (Internal Advocate or Business Liaison) – Educates employees and promotes AI usage.

High-Budget Team Structure

AI Ops Lead (1 FTE) – Expands focus from quick wins to structured AI strategy.
AI Engineer (2 FTEs) – Supports growing AI tool development and integration.
AI Evangelist (1 FTE) – Establishes training programs and AI literacy initiatives across teams.
AI Analyst (1 FTE) – Monitors AI effectiveness and identifies optimization opportunities.
Business Liaison (1 FTE or fractional) – Ensures AI adoption aligns with departmental needs and processes.

Key Focus Areas

Scaling AI solutions across multiple teams and functions.
Implementing AI governance frameworks to ensure responsible AI use.
Developing AI training programs to bridge the knowledge gap.
Standardizing AI tool adoption across different business units.
Establishing feedback loops to refine AI applications.

Phase 3: AI-Ready Data & Enterprise Integration

At this stage, AI is no longer an isolated experiment; it becomes a core part of business operations. The AI Ops team must focus on deep integrations, enterprise-wide governance, and optimizing AI-driven decision-making.

Low-Budget Team Structure

AI Ops Lead (1 FTE) – Ensures enterprise AI alignment and strategy.
AI Engineer (1-2 FTEs) – Focuses on integrating AI with existing infrastructure.
AI Data Consultant (Contractor or Part-Time Employee) – Supports data quality improvements.
AI Governance Advisor (Fractional Role) – Establishes lightweight AI policies and compliance guidelines.

High-Budget Team Structure

AI Ops Director (1 FTE) – Oversees AI strategy at the enterprise level, ensuring alignment with business objectives.
AI Engineers (3-5 FTEs) – Develop and maintain robust AI infrastructure and applications.
Data Engineers (2-3 FTEs) – Ensure high-quality data pipelines and integration into AI models.
AI Governance Lead (1 FTE) – Establishes ethical AI guidelines and compliance measures.
AI Evangelist (1-2 FTEs) – Drives continued AI literacy and adoption across the company.
AI Analysts (2-3 FTEs) – Measures AI performance and business impact.
Business Liaison (1 FTE per major department) – Embeds AI within key functions such as HR, marketing, finance, and operations.

Key Focus Areas

Integrating AI into enterprise systems such as CRM, ERP, and customer service platforms.
Optimizing AI solutions for large-scale automation and decision-making.
Enhancing AI governance and compliance to align with regulatory requirements.
Continuous iteration and optimization of AI-driven business workflows.
Measuring ROI and impact to inform long-term AI strategy.

Building an AI Ops team is not a static process—it evolves with the organization's maturity in AI adoption. In the early stages, the focus is on agility, experimentation, and internal advocacy. As AI adoption scales, the team expands to include more structured education, governance, and integration roles. By aligning the AI Ops team with the appropriate phase of adoption, organizations can maximize AI's impact and ensure a smooth transition from experimentation to enterprise-wide implementation.

Identifying Internal Candidates

Building an AI Ops team doesn't always require looking externally—some of the best candidates may already exist within the organization. Identifying employees who demonstrate the characteristics outlined in the STACK-B framework (Systems Thinking, Technologist, Adaptable, Communicative, Knowledge-Seeker, Bold) is essential for assembling a high-functioning team.

What to Look for in Internal Candidates

Curiosity and Initiative: Candidates should demonstrate an eagerness to learn, experiment, and explore AI's potential in their current roles.
Process Optimization Mindset: Look for employees who naturally seek efficiency improvements in their workflows.
Technical Fluency: While they may not be developers, candidates should be comfortable with data, automation tools, and technology adoption.
Cross-Departmental Thinkers: Individuals who understand how different departments interact and are capable of seeing the big picture.
Effective Communicators: Those who can translate AI concepts into clear, actionable insights for non-technical colleagues.
Bias Toward Action: Employees who take initiative and test new tools or methods without needing direction.
AI Champions: Those already engaging with AI tools, participating in AI-related discussions, or advocating for AI-driven solutions.

Methods for Identifying Internal Talent

Observation & Performance Reviews: Managers can identify employees who exhibit AI Ops traits in their current roles.
AI Literacy Training Participation: Employees who show enthusiasm in AI training sessions are strong candidates.
Internal Hackathons & Innovation Challenges: Employees who actively engage in AI-driven innovation challenges demonstrate a natural fit for AI Ops roles.
Peer Nominations: Colleagues can nominate individuals who consistently embrace technology and efficiency.
Pilot Project Involvement: Employees who have contributed to AI or automation projects in any capacity.

Internal Screening Exercises

To ensure the right fit, consider assessing internal candidates through practical exercises:

Process Mapping Task: Have candidates identify inefficiencies in their department and propose an AI-driven solution.
AI Prompting Challenge: Ask candidates to interact with a generative AI tool (e.g., ChatGPT) and refine their prompts for better outcomes.
Cross-Functional Scenario Exercise: Present a business challenge and evaluate their ability to apply AI thinking across departments.

Transitioning internal candidates into AI Ops roles ensures a smoother cultural shift and leverages institutional knowledge, creating a well-rounded and committed team.

Sourcing External Candidates

While internal candidates provide valuable institutional knowledge, external hiring may be necessary to fill specific gaps in expertise, particularly in technical roles or advanced AI strategy.

Key Skills to Look for in External Candidates

AI & Machine Learning Experience: Strong understanding of AI model development, automation, and integration.
Systems Thinking & Process Optimization: Ability to analyze complex workflows and implement AI solutions efficiently.
Change Management Skills: Comfortable leading AI adoption and overcoming resistance in organizations unfamiliar with AI Ops.
Cross-Functional Collaboration: Experience working across departments to align AI initiatives with business goals.
Software & Infrastructure Expertise: Proficiency in cloud-based AI solutions, APIs, and automation tools.
AI Governance & Ethics Awareness: Understanding of compliance, data privacy, and responsible AI deployment.

Best Sources for External Talent

AI & Data Science Communities: Platforms like GitHub, Kaggle, and AI-focused LinkedIn groups.
Industry Conferences & Meetups: Attending AI and automation events can help identify passionate and skilled candidates.
University Partnerships: Collaborating with academic institutions to source AI researchers and students specializing in AI Ops.
Professional Networks: Leveraging AI-focused communities or referrals from existing AI practitioners.
Recruiting Firms Specializing in AI & Automation: Agencies that specialize in sourcing AI and data professionals.
Freelance Marketplaces: Platforms like Upwork or Toptal can be useful for hiring AI contractors to test-fit roles before committing to full-time hires.

Interview Questions & Exercises to Identify AI Ops Candidates

Behavioral Questions

"Tell us about a time you automated a process. What was the impact?"
Assesses process optimization and problem-solving skills.
"Can you explain AI to a non-technical stakeholder?"
Tests communication ability.
"Describe a situation where you had to implement AI in a skeptical or resistant environment. How did you overcome the challenge?"
Evaluates change management capabilities.

Technical Assessments

Live AI Use Case Exercise: Provide a business scenario and ask them to propose an AI-driven solution, detailing their approach and trade-offs.
AI Model Analysis: Present a pre-built AI model and ask them to critique its performance and suggest improvements.
Prompt Engineering Task: Evaluate their ability to refine prompts for AI tools to maximize output quality.

Hiring external candidates can bring fresh perspectives, specialized expertise, and industry best practices to an AI Ops team. Balancing internal promotions with external hires ensures the team has both deep organizational knowledge and cutting-edge technical skills.

Reporting Structure

One of the most crucial decisions in structuring an AI Operations (AI Ops) team is determining its reporting hierarchy. The placement of the AI Ops leader within an organization can significantly impact the team's effectiveness, strategic alignment, and ability to drive AI adoption across the enterprise. The reporting structure should ensure that AI Ops remains operationally relevant, aligns with business priorities, and has executive sponsorship for long-term success.

Preferred Reporting Structures

The AI Ops team should ideally report to a senior executive who has both operational oversight and strategic influence. Below are the three primary reporting structures, along with their benefits and potential challenges:

Reporting to the Chief Operating Officer (COO) (Recommended)
- Benefits:
  - Operational Focus: The COO oversees company-wide operations, making AI Ops a natural fit under this leadership.
  - Broad Scope: AI Ops spans multiple departments (HR, finance, customer service, IT, etc.), and the COO is best positioned to ensure AI solutions align with overall business operations.
  - Efficiency and Scalability: AI Ops can integrate directly into existing operational workflows, reducing silos and ensuring AI adoption at scale.
  - Cross-Functional Impact: Since AI Ops influences multiple business areas, a COO-led structure allows for a seamless connection between departments.
- Potential Challenges:
  - If the COO is unfamiliar with AI and technology trends, they may not fully leverage AI Ops' potential.
  - May require additional technical oversight from a CTO or CIO to ensure AI models and infrastructure are implemented correctly.
Reporting to the Chief Technology Officer (CTO)
- Benefits:
  - Technical Expertise: The CTO understands AI's technological complexities and can ensure its proper integration with IT infrastructure.
  - Innovation Leadership: AI Ops will have direct access to R&D initiatives, fostering a culture of continuous improvement.
  - Scalability: The CTO can oversee technical scaling and cloud-based AI implementations.
- Potential Challenges:
  - AI Ops may become too focused on technology rather than business outcomes.
  - Limited influence over operational and process-driven teams, making AI adoption in non-technical areas more challenging.
Reporting to the Chief Executive Officer (CEO)
- Benefits:
  - Top-Level Strategic Priority: AI Ops will be directly aligned with the company's overall vision and long-term strategy.
  - Company-Wide AI Adoption: With CEO backing, AI Ops initiatives are more likely to receive funding and executive buy-in.
  - Cross-Departmental Reach: Direct oversight from the CEO ensures AI Ops influences the entire organization.
- Potential Challenges:
  - Bandwidth Issues: CEOs often lack the time to be directly involved in AI Ops, which can lead to bottlenecks in decision-making.
  - Delegation Challenges: Without technical or operational expertise, AI Ops may require additional oversight from other C-suite members.

Choosing the Right Reporting Structure

The best reporting structure depends on the company's size, AI maturity, and strategic priorities. However, in most cases, reporting to the COO provides the optimal balance between operational integration, cross-functional collaboration, and scalability. If the COO lacks AI expertise, a dual-reporting relationship to the CTO for technical oversight may be beneficial.

For highly technical companies where AI innovation is a core product, reporting to the CTO may be more effective. In AI-first organizations where executive-level prioritization is essential, reporting to the CEO could be considered, but only if the CEO is highly engaged in AI initiatives.

Final Recommendation

For most organizations, AI Ops should report to the COO to maintain an operational focus, drive business impact, and scale AI adoption across the enterprise. However, a close partnership with the CTO ensures that AI Ops solutions are technologically sound and strategically implemented. Establishing regular executive check-ins with the CEO can also help align AI Ops with broader business objectives.