The Skills a Small Company Needs to Run AI in Production

A small company needs someone to understand the work, someone to build and maintain its integrations, and a clear way to judge and operate the result. Data preparation, evaluation, security, and vendor support also need named owners. Several people can share these responsibilities, but each responsibility needs demonstrated skill, time, and a backup.

Define the production workflow before staffing it

Running AI in production means staff or customers rely on the result during normal work. Someone has to maintain that service when a source system changes, a model returns an incorrect answer, or the original builder takes leave. A working demonstration establishes much less than that operating commitment.

Describe one supported workflow. For example, an assistant reads incoming service requests, prepares a draft job record, and sends uncertain cases to a staff member. State who uses it, where the records come from, and which system receives the approved result. Specify what it can do without a person's approval.

The required skills follow that scope. A read-only search assistant needs a different integration and recovery plan from software that changes bookings or sends customer messages. A company using an approved hosted assistant with no custom integrations may need configuration and operational oversight rather than a full application team. Reassess the staffing when the scope changes.

Anthropic's engineering guidance on effective agents recommends starting with a simple solution and adding complexity when the task requires it. Apply that principle to the operating burden too. Each extra component needs someone who understands its failure modes and can maintain it.

A Reddit discussion about production monitoring asks how teams track AI quality alongside ordinary technical health. It identifies a reader concern, rather than evidence of how most companies operate. Your staffing plan should answer who detects a wrong result even when the application stays online.

Use the capabilities below as a coverage map. They don't imply six new hires or a requirement to train your own model. They describe work that somebody must cover for the workflow you choose. Keep the responsible person's name, availability, and escalation route beside each capability.

Keep the data fit for the task

Data work starts with identifying the authoritative source. A customer account, service address, and current contract may live in different systems. Someone needs to establish which record takes precedence and how the application matches them. An AI model cannot settle an unresolved business rule about which contract applies.

The person covering data engineering should be able to trace a record from source to output. They need to recognise missing values, duplicates, inconsistent identifiers, and delayed updates. For a small workflow, a controlled export and validation script may be sufficient. A larger or faster-changing workflow may need scheduled pipelines and stronger monitoring.

Agree freshness requirements with the process owner. A procedure document that changes monthly and a dispatch board that changes every few minutes need different update arrangements. Show staff when the source last refreshed. If a refresh fails, keep that state visible and decide whether the workflow can continue using older information.

Plan for corrections and deletions. If staff correct a customer's address, the next run should use the corrected record. If the system keeps a separate search index or cache, identify how it receives that change. Store enough provenance to explain which version supported a result without copying unnecessary sensitive records into logs.

Check the skill through a practical demonstration. Ask the responsible person to follow an incorrect draft back to its source, correct the input through the approved process, and show how the next run changes. If they only edit the output, the underlying problem may recur. The demonstration should also show how staff report a source error they cannot fix themselves.

You may share this work with an existing analyst or systems administrator if they have the required skills and time. Use a data engineer when the pipeline complexity exceeds that coverage. Our data-readiness guide explains how to inspect the inputs before adding more automation.

Build and maintain the software connections

The integration owner connects the model or hosted product with your actual workflow. That can include authentication, input validation, a staff review interface, and controlled writes to the receiving system. They need to understand the whole request path, including what happens when only part of it succeeds.

Duplicate actions are a practical risk. If a request times out after the job system has created a record, repeating it may create a second job. The engineer should design a way to recognise the original request and reconcile its state before retrying. Staff need a clear view of whether an action failed, succeeded, or has an unresolved outcome.

Set limits for waiting and retrying. A provider error should not leave staff staring at an indefinite loading indicator, and repeated retries should not spend an unlimited budget. Agree the response for a temporary failure and preserve the staff member's entered work. Test the receiving system's errors as well as the model provider's errors.

Keep releases reproducible. Store the application code and configuration in company-controlled repositories and accounts, with an agreed review process. Record the model, prompt, and integration version used by each release. Give another authorised engineer a way to deploy and restore the approved version.

AI coding tools can assist the implementation, but the engineer still needs to explain and validate it. GitHub's current guidance for reviewing AI-generated code calls for functional checks and review against the project's context and intent. Hiring someone who can generate code is insufficient if they cannot investigate its behaviour in your environment.

Assess the engineer using your actual failure scenarios. Can they handle an expired credential, a changed API response, and an uncertain write outcome? Can they show that the application preserves permissions? A full stack engineer, integration specialist, or forward deployed engineer may cover the work; choose from evidence of delivery and support rather than the title alone.

Evaluate outcomes with a domain expert

Evaluation needs technical implementation and knowledge of the job. An engineer can automate a test, but an experienced service coordinator may know that a proposed visit duration is impossible or that two customers share a trading name. Assign both contributions. The process owner should resolve disagreements about what counts as correct.

Build a set of representative cases with expected outcomes. Include ordinary requests, incomplete records, and the exceptions that staff encounter. Use approved data and appropriate access controls; synthetic examples can cover cases that you cannot share. Record what the test set represents and what it leaves untested.

Judge the final business state as well as the answer. A draft that names the right customer but attaches the wrong site fails the workflow. A successful API response doesn't prove the record contains the right information. Check the receiving system and the review queue against the acceptance criteria.

Anthropic's January 2026 evaluation guide distinguishes an agent's recorded interaction from the final state it produces, and combines automated checks with human judgement where needed. It also explains why automated evaluations need production monitoring and user feedback. Use those methods together rather than treating a test score as complete operating evidence.

Run the relevant cases before changing prompts, models, retrieval, or integrations. Record the prior result and investigate failures by case type. Repeat cases when output variation affects the decision. A small suite can reveal a major regression, but it cannot justify a claim about rare failures that it doesn't cover.

Give evaluation maintenance real time. Someone needs to add new failures, review ambiguous expected answers, and check whether the graders still match the business rule. Keep domain review separate from the builder's own assurance where the consequence warrants it. Our evaluation-set guide describes the practical test-building process.

Control access and recover from failures

The security responsibility covers who can use the system, which records it can read, and which actions it can take. A small company may use its existing IT provider for parts of this work. Confirm that somebody can review the application-specific controls and understand how the AI component interacts with them.

Enforce permissions in the application and source systems. Don't rely on a prompt telling the model to avoid confidential records. Use scoped service accounts, keep credentials out of browser code, and ensure staff can only retrieve information they are authorised to access. Test cross-user and cross-customer access where the workflow serves separate groups.

Keep external content within its intended role. An incoming email or retrieved document can contain instructions that conflict with your workflow. The application should treat that content as data, validate proposed actions, and apply the same permission checks regardless of what the model requests. Give the engineer a specific threat scenario to demonstrate.

Agree where data goes and how long the system keeps it. Review the actual service configuration and contract terms with the appropriate owner. Logs, test datasets, and support tickets can each create additional copies. Collect the evidence needed to diagnose failures while restricting unnecessary disclosure.

NIST's AI Risk Management Framework playbook calls for clear risk-management responsibilities and communication routes, and addresses third-party system failures. It provides a framework for planning these duties; it doesn't establish that a particular implementation complies with every requirement that applies to your company.

Recovery needs a named operator too. Test how they disable an unsafe action, route work to the manual process, and restore the approved release. Record the access they need and the person they can call. A recovery document has limited value if its author is the only person who can follow it.

Give a process owner authority and time

The product responsibility starts with defining what the company needs the workflow to achieve. A small business may assign it to an operations manager rather than hire a product manager. They need authority to decide scope, access to the people doing the work, and time to review evidence.

Establish a baseline before promising efficiency gains. Record the current volume, staff effort, and rework for the supported task. Agree which quality measures the rollout must preserve. Measure the whole process after launch, including review and downstream corrections, so the team can distinguish faster drafting from a better operation.

Decide where people approve or intervene. The process owner sets the rule; the engineer implements it; reviewers need enough context to apply it. If reviewers disagree, investigate the business rule rather than asking the model to conceal the uncertainty. Explain how staff can stop or escalate a case.

Make priorities explicit. A request to add another document type may compete with fixing a recurring mapping error. The owner should weigh operational value, delivery effort, and support obligations. Keep a short backlog with a reason for each priority and an acceptance condition.

Training belongs in this responsibility too. Staff need to recognise unsupported cases, check drafts, and report problems. Review whether the new workflow actually fits their workday. If the team spends more time repairing output than the task previously took, narrow the supported scope or change the design before expanding it.

The owner also decides whether continued operation is justified. Agree a review point and the evidence required to continue, pause, or expand. A technically available service can still fail to meet the business objective. Our guide to ownership after launch explains how to carry that responsibility beyond the pilot.

Manage vendors and ongoing support

A vendor can operate the model service or hosted application while your company owns the local integration and business process. Write that boundary down. Ask who investigates an incorrect result, who repairs a connector, and who communicates with staff during an incident. A supplier's status page doesn't identify the source of every local failure.

Check the support agreement before relying on it. Identify the service hours, escalation route, and the evidence the vendor needs. Distinguish a response commitment from a promise to restore the workflow. Give your operator a fallback for periods when the supplier cannot resolve the problem in time.

Track changes to the dependencies you use. Someone should review model availability, API changes, and product notices, then schedule the required evaluation and release work. Record the account owner and renewal decision. A credit card payment keeps a service funded but doesn't keep an integration compatible.

Monitor cost alongside completed work. Count provider usage, retries, review labour, and support time for the workflow. An alert for a spending threshold helps only if someone receives it and has authority to act. Agree what the operator can pause and what requires the process owner's decision.

Prepare for departure and supplier changes. Your company should control the production accounts, repository access, and operational documentation it needs. Confirm what another provider or engineer can take over, subject to the rights in the agreement. Maintain a working manual path while you assess alternatives; don't promise an instant vendor switch without testing the dependencies.

For a PE portfolio, a shared specialist can coordinate vendor reviews and common connectors across several companies. Each company still needs local process ownership and an incident contact. For a venture-backed startup, show how support of existing customers fits beside new delivery work when planning engineering capacity.

Assemble a small team and test its coverage

Consider an illustrative 25-person service company deploying a request-to-draft-job assistant. This is a staffing example, with no claimed client result or market benchmark. Its operations manager owns the workflow, a coordinator contributes domain cases and training, and a contracted engineer maintains the application and integration.

The engineer covers data validation and evaluation automation where they have the demonstrated skills. The company's IT provider reviews identity and access configuration, with specialist application-security help when required. The model or software vendor supports its own service. A second authorised engineer provides continuity for release and recovery work.

These assignments describe coverage, rather than a fixed hiring formula. Check the workload against the people available. If an illustrative weekly plan allocates 12 hours to engineering, four to domain evaluation, three to product decisions, and three to support and access work, it needs 22 hours across those responsibilities. Those figures are assumptions for planning, not recommended staffing levels.

Reserve time for continuing work after the initial build. Updates, new exceptions, and support compete with new development. Review actual hours and incidents during the first operating period, then adjust the coverage. If the same person owns several duties, make sure an urgent incident doesn't leave every other responsibility unattended.

Before expanding, ask the receiving team to demonstrate a complete supported case and a failure case without the original builder directing each step. Check the output, the source records, and the final system state. Record any gap as a named task with an owner and a completion condition.

Use the gaps to decide where to invest next. Missing business criteria call for process-owner and domain time. A fragile connector calls for integration expertise. Inadequate access controls call for security review and remediation. Fund the capability that blocks reliable operation, and require evidence that the team can continue covering it after launch.