Research / Technical implementation
RAG, Fine-Tuning, or a Better Database Query?
Choose RAG, fine-tuning, SQL, or explicit rules for a business task. Compare concrete examples, failure modes, evaluation, permissions, and operating costs.
Choose the architecture by examining the work the system must get right. A question about an order total needs a complete calculation. A question about a policy needs the applicable source. A document extraction task may need better model behaviour. A fixed approval limit needs a rule you can enforce. These needs can occur in the same workflow.
Understand what each approach changes
Retrieval-augmented generation, or RAG, supplies information to a model when it answers. Fine-tuning continues training a model on task or domain data. SQL queries select and calculate over database records. Explicit rules apply decisions that your business can state in code.
A Reddit question about structured Excel data asks whether counting across spreadsheets needs RAG or a SQL database. It captures a practical architecture question; the replies aren't evidence of guaranteed accuracy or cost savings. Start by writing the result your own system must produce.
| Required result | Starting point | Check before expanding |
|---|---|---|
| You need an exact count or total. | Query the authorised records and calculate in the database. | Confirm definitions, joins, and coverage. |
| You need an answer supported by a current document. | Retrieve the applicable passages and show the evidence. | Confirm version, access, and support for the claim. |
| You need a model to perform a recurring task more consistently. | Test instructions and examples, then evaluate tuning if errors persist. | Confirm training quality and performance on unseen cases. |
| You need to enforce an agreed condition. | Apply a versioned rule to validated inputs. | Confirm boundaries, exceptions, and approval authority. |
These labels overlap in an application. A model can call a SQL tool and use its result as context; that is also retrieval in the broad sense. The important implementation choice is how you obtain the records and calculate the answer. Don't assume that every retrieval task needs vector similarity search.
Use database queries for complete records and totals
Suppose a distributor asks how many open orders a customer has and their total value. Define “open”, select the authorised customer's records, and calculate over the complete matching set. PostgreSQL's aggregate documentation explains how counts and sums operate over rows. Returning a few similar text passages doesn't establish a complete total.
Write the business definition before the query. Decide whether the total includes tax, cancelled lines, or partial shipments, and state the currency. An order joined to several order lines can repeat an order-level amount. The query can run successfully and still calculate the wrong measure.
For a bounded question, start with an approved query or API operation and validated parameters. A natural-language interface can select the operation and ask the reader to clarify an ambiguous customer. Keep the calculation in the service, and show the relevant date and scope beside the result.
If users need broader text-to-SQL exploration, treat generated SQL as a proposal that needs controls. Restrict the accessible data, enforce query limits, and test the resulting measures against reviewed answers. Read-only access prevents ordinary writes but doesn't stop a query from returning confidential records.
PostgreSQL row-security policies can restrict visible rows. Its documentation also identifies roles that bypass those policies, including superusers and roles with BYPASSRLS; table owners normally bypass them too. Test with the application's actual role and tenant identity, rather than assuming a policy protects every connection.
Use RAG for answers that need document evidence
A question about a returns policy needs the applicable rule and its conditions. Microsoft's RAG documentation describes retrieving content and supplying it to the model as grounding. Search can use keywords, vectors, or a combination. RAG doesn't require changing the model's trained parameters.
For the distributor, retrieve the approved policy for the product and region. Keep its effective date and document identifier with the passage. If an amendment changes a condition, the retrieval process needs to select that amendment and the relevant original text together.
Choose search according to the question. An exact product code can need an exact lookup, while a paraphrased policy question may need semantic matching. Test whether the returned evidence contains the answer before changing the model. If the source itself omits the condition, ask the policy owner to resolve it.
Microsoft also notes that grounding can still produce inaccurate answers and that retrieval needs access controls. Apply permissions before content reaches the model, and treat document text as untrusted input. A passage that tells an assistant to ignore its instructions doesn't acquire authority because search returned it.
Review whether the cited passage supports each material claim. A correct link beside an unsupported answer can mislead a reviewer. Where evidence conflicts or doesn't answer the question, return the uncertainty and route the case to the responsible person.
Consider fine-tuning for repeated task behaviour
Hugging Face's fine-tuning guide describes continuing training from a pretrained model on a smaller task or domain dataset. The training changes model parameters; some methods train additional adapter parameters. This can adapt task behaviour, but it doesn't create a live connection to your order system.
Consider an extractor that repeatedly confuses a requested delivery date with an invoice date despite clear instructions and representative examples. A supervised tuning experiment can use reviewed input-and-output pairs to teach the intended distinction. Compare it with the existing model on documents it hasn't seen.
First separate output shape from meaning. A schema check can reject an invalid date format or a missing required field. It can't establish that a valid-looking date came from the right part of the document. Tuning needs a demonstrated task error to address, and the application still needs validation.
Prepare examples that include exceptions and incomplete inputs. Remove near-duplicate documents from the evaluation split, and consider holding out customers or document templates to test the kind of novelty you expect. Confirm the right to use the training records and who owns corrections to their labels.
Google's supervised-tuning documentation requires a prepared dataset and supports validation data. Supported models and deployment requirements depend on the service. Check the current constraints for the model you intend to operate before estimating the work.
Keep changing facts in a maintained source you can query or retrieve. A tuned model may learn factual associations, but that doesn't establish their current value, access rights, or provenance. If the delivery date changes today, fetch today's record rather than expecting a training run to keep it current.
Keep explicit business rules in code
Some work has an agreed decision procedure. If an invoice identifier already exists for the same supplier, flag a possible duplicate. If a requested discount exceeds the approved limit, require the named approver. Write those conditions into the application and test their boundaries.
A model may help extract the identifier from an email or explain why a case needs review. Validate that extracted identifier before applying the rule. Where text is ambiguous, preserve the ambiguity and request a correction instead of inventing a value to keep the workflow moving.
Keep the rule version and the inputs used for each decision. When a limit changes, the business owner should approve the change and its effective date. Test the value just below the boundary, the boundary itself, and the value just above it.
Don't force a judgement-heavy task into a brittle rule merely to avoid AI. A disputed delivery commitment may require a person to assess the customer's circumstances. Record which conditions the software can decide and which questions it must escalate.
Combine the approaches in a customer workflow
Consider an illustrative request: “My order hasn't arrived. Can you cancel it?” The assistant first needs the authenticated customer's identity and an unambiguous order. If several orders fit, ask for clarification before fetching or discussing their details.
An authorised query supplies the order and shipment status. Retrieval supplies the applicable cancellation policy and any amendment. The application applies explicit conditions, such as whether the shipment has passed a cancellation cutoff. The model can then draft a response using those results.
A tuning experiment belongs only where a measured task error persists, such as reliably extracting the requested action from a recurring email format. You don't need to tune a model simply because the workflow already has retrieval. Each component should have an assigned job and evidence that it performs it.
Keep the customer commitment separate from the draft. A policy explanation doesn't authorise a cancellation or refund. Give the action service its own permission checks and approval path, then record the outcome. Use the system-mapping guide to show the boundaries between these components.
When evidence is missing, stop the affected step. A missing shipment record shouldn't turn into a confident claim that the order hasn't shipped. Return the missing information to the reviewer and preserve the completed parts of the case.
Test against a simpler starting point
Build a reviewed set of actual questions and tasks. Include exact totals, amended policies, ambiguous requests, and incomplete documents. Add access tests where the user must receive no record. State what a correct answer contains and when the correct behaviour is to ask or escalate.
Compare an approved query, a search result, or a prompt with examples before adding another component. Keep the same cases and source snapshot across each comparison. Otherwise a model change can appear to improve accuracy simply because the underlying records changed.
Inspect failures by stage. Did the application resolve the wrong customer, choose the wrong query definition, miss a policy amendment, or misread evidence it had? If you replace the model for every failure, you can spend money without repairing the part that caused the error.
For tuning, evaluate on unseen examples and check important behaviours outside the tuned task. For retrieval, check evidence coverage and citation support. For calculations and rules, compare exact expected results and boundary cases. Report serious errors separately from an average score.
Measure the cost of accepted work, including retries, review, and fallback. As an illustrative calculation, 20,000 monthly requests at $0.015 each cost $300 in request charges; at $0.035 each they cost $700. The $400 difference excludes setup, training, indexing, maintenance, and staff time. These figures aren't provider prices or a complete business case.
Use a baseline for the whole workflow to compare the operating result. Faster model output doesn't establish that the team finishes the task sooner, especially if reviewers now need to resolve more exceptions.
Choose a plan your team can operate
Write a decision record that names the required result, the source of truth, and the failure the architecture addresses. Assign owners for query definitions, policy versions, and any training dataset. Include the fallback and the evidence that would justify changing the design.
For an SME, an approved query behind an existing screen may solve the current problem with little new infrastructure. A PE portfolio team should compare business definitions before sharing queries across companies. A startup investor can ask for a trace from customer request to accepted outcome, including manual repairs and permission checks.
Ask a vendor to demonstrate an exact total, an amended policy, and a request the user can't access. Those cases test different responsibilities. Assess each result against your own records; a fluent answer to the policy question doesn't establish that the database total is complete.
Choose the smallest design that meets the task's quality and access requirements with an operating plan your team can maintain. Add retrieval when the task needs evidence, and test tuning when a recurring model error justifies it. If the obstacle is an unclear metric or missing source record, repair that input first.
