Purchasing Guides / Custom Text Classification Model

Custom Text Classification Model

A purchasing guide to buying a small AI model trained to sort your text into your own categories, running in your environment.

Yearly running cost

Assuming 12,000 batches a year

By hand
$240K / year
10,000 hours of work
Hosted service
$18K – 36K+ / year
This workflow
$100 / year
Machine usage only; setup, hosting and review are extra.
Your modelDo you also need a way to retrain it?

Start with a model trained on your labels. Add a repeatable training pipeline if your examples or categories will change.

What is included

Adapt a small open model to your approved labels, evaluate it on held-out examples and deploy it with a review threshold.

Your checked examples train a small model for up to 10 categories. The starting scope covers single-label English text up to 128 model tokens per item. You receive model artefacts, a working deployment and test results under the agreed licences.

AI-assisted delivery adapts reusable components. Data labelling, training compute and new integrations are agreed separately. New models need approval before release.

AccuracyHow well should it recognise each category?

Measure every category, not just the most common ones. Unknown or ambiguous text should go for review.

How accuracy is measured

These are acceptance targets, not measured results. Test on examples kept out of training and tuning. Macro F-score balances precision and recall equally across categories; it is not the percentage of all texts labelled correctly.

Targets for your selected checks
What is checkedTarget
Balanced classification scoreCalculate precision and recall for each category, combine them into its F-score, then average equally across categories. Abstentions on labelled in-scope text count as misses.≥90%
Minimum category recallCorrect predictions for a category divided by all held-out texts labelled with that category. Every category must meet the target; do not hide rare categories in an overall average.≥85%
Out-of-scope texts sent for reviewCorrect review states divided by all labelled out-of-scope texts. Report false review flags on in-scope text separately and include them in classification scoring.≥90%
Valid prediction recordsValid records divided by all submitted records. Missing results, malformed fields and unrecognised labels fail.≥100%

Include missed predictions and in-scope review cases in the scores. Agree enough examples for every category and inspect the confusion matrix before acceptance.

SpeedHow quickly should it classify a batch?

Choose the time to return labels for up to 1,000 texts. Training time is separate.

Timing details

At least 95% of test batches should meet the target, with 1 in progress at a time. Include model loading, tokenisation, queueing, retries and export. Agree hardware and input lengths before testing; a faster target is not an achieved benchmark.

Running-cost targetHow much should each batch cost?

Set your inference budget for up to 1,000 texts. This may affect the model, hardware and batching.

Cost details

Reference machine cost: $0.01 – 0.02 per batch at 1,000 batches a month. Includes classifier inference, retries and allocated infrastructure. Training, labelling, human review, agent orchestration and maintenance are separate.

Your budget is a target, not a measured cost. Confirm it alongside accuracy and speed on the deployed model. Local hardware costs need measurement.

Your environmentWhere should your model run?

Run your model in your cloud account or on your own hardware. Your text does not go to an external model API.

Data and model access

Deploy the model in your cloud account. Inference runs there without sending text to an external model API.

Training and inference use locally loaded model weights. Model downloads are approved during setup; production text is not sent to an external model API.

Check the base model licence, data permissions and retention. Keeping inference inside your environment does not remove your cloud infrastructure bill.

InterfaceHow do you want to use it?

Connect your agent, use a dedicated agent or upload batches in a web page. Choose more than one if needed.

Anything else your provider should know?

Optional. Describe your text, labels, volume or budget. We include your selections automatically.

Common questions

When should I buy this instead of using a general AI model?

Buy when your categories repeat, you have checked examples, and you need predictable behaviour in your own environment. Use a general model or existing service when the task changes often or you cannot yet define the labels.

Do I need labelled examples?

Yes. Supply authorised text and agreed labels, including ambiguous and out-of-scope cases. The provider separates training, tuning and acceptance data. Creating and checking a new dataset is a separate piece of work.

Will I receive the model or just API access?

You receive the agreed model artefacts, label mapping and a working deployment, subject to the base model's licence. Agree source access and reuse rights before hiring.

Will it learn automatically from every correction?

Not by default. The retraining option turns approved corrections into a candidate model and comparison report. You approve release; the previous version stays available for rollback.

Can it classify longer documents or other languages?

Yes, describe them in your brief. More labels, longer inputs, multilabel output and extra languages require their own data, tests and estimate.

Download RenX

Get the app.

Install, sign up, and start on your free plan with welcome credit included. No credit card required.

On a platform not listed? Leave your email and we'll notify you when a build is available.