Private AI on your own servers
How organizations run open-weight models such as Llama, Qwen, DeepSeek and GLM on servers they control, what it takes, and when it is worth it.
AI Solutions
Private AI and self-hosted AI models
Some organizations cannot send client files, health records or confidential plans to an outside AI provider, whatever the contract says. We deploy open-weight models on GPU servers you control, in your own cloud account or your own data centre, connect them to your documents and applications, and secure and run them like any other critical system. Your prompts, files and outputs stay in your environment.
Who this service is for
A good fit if
- Your policies, clients or regulators do not allow confidential or personal information to be processed by an outside AI provider.
- You need full control over where AI runs, what is logged and how long anything is kept.
- You want AI over sensitive documents, such as legal files, health records or financial data, without a third party seeing them.
- Your AI use is steady and high enough that paying per request is becoming expensive.
- You want to reduce your dependence on one AI vendor's prices, terms and model changes.
Another approach may suit you better if
- Enterprise terms from a commercial AI provider, with no training on your data and zero data retention, meet your requirements. That is usually simpler, and for light use cheaper, and we will tell you so.
- You need the most capable commercial model for the hardest reasoning tasks, and no open-weight model meets your accuracy bar on your own tests.
- Use is light or occasional. A GPU server that sits idle most of the day costs more than paying per request.
- Nobody will own the system after launch. A self-hosted model needs patching, monitoring and updates like any server, from your team or ours.
What this service is
Most organizations use AI through a provider's service: their prompts and documents travel to the provider, are processed there and come back. Business terms from the major providers exclude your data from model training, and some offer zero data retention. For many organizations that is enough. For others, such as law firms, health organizations, financial institutions and companies protecting their own product designs, the requirement is simpler and stricter: confidential information does not leave our environment.
Private AI meets that requirement. We deploy an open-weight model on GPU servers that you control, connect it to your documents and systems, and secure it like any other critical system. Your staff get a capable assistant, and your applications get an AI endpoint, without any outside AI provider seeing a prompt.
Three ways to keep AI data private
| Option | Where the model runs | Who can see your prompts | Best for |
|---|---|---|---|
| Enterprise AI service | The provider's cloud | The provider processes them under business terms; with zero data retention, they are not stored after the response, with limited exceptions | Most organizations, when the terms meet your requirements |
| Managed model in your cloud account | Your cloud provider's AI service, in a region you choose | Your cloud provider, under your existing cloud agreement, not the model developer | Organizations already committed to one cloud with data residency needs |
| Self-hosted open-weight model | GPU servers you control, in the cloud or on premises | Only your organization | Information that must not leave your environment, steady high volumes, full control |
We are not tied to one approach. Many organizations end up with a mix: a commercial assistant for everyday work and a private model for the information that must stay in-house. Our comparison of self-hosted models and AI APIs explains how to decide.
What we deploy
A private AI platform is more than a model on a server. A typical deployment includes:
- GPU servers sized for the model and your peak use, in your cloud account, on a dedicated server or on your own hardware.
- An inference server, such as vLLM or SGLang, that runs the model efficiently for many users and offers an OpenAI-compatible API.
- A gateway that handles sign-in through your identity provider, usage limits and logging under your retention rules.
- Retrieval over your documents, so answers come from your own policies, files and records, with sources. Our guide to how AI assistants answer from your documents explains the approach.
- Applications on top, such as a staff assistant, AI features in your portal or ERP, or AI agents that work on your private model.
Choosing a model
There is no single best open-weight model. We shortlist models by the tasks you need, the languages you work in, the hardware they need and their licence terms, then test them on your own questions and documents. Smaller models answer faster and need less GPU memory, and are often good enough for document questions and extraction. Larger models handle harder reasoning but need more powerful servers. New releases arrive often, so we keep your evaluation set and re-test when a better candidate appears.
Security, privacy and residency
- Closed by default. The model endpoint is reachable only from your network or approved applications, behind single sign-on.
- Trusted model files. We obtain weights from the developer's official source, use the safetensors format, which stores only the model's numbers rather than code that could run when it loads, avoid running model code we have not reviewed, and record exactly which files are in production.
- Your retention rules. You decide what is logged and for how long, including keeping no prompt content at all.
- Canadian residency. Where data must stay in Canada, we use Canadian cloud regions, such as DigitalOcean Toronto, AWS Montréal or Google Cloud Montréal and Toronto, Canadian facilities or hardware on your premises, and check that no component routes requests outside the country.
- Privacy law still applies. Self-hosting keeps data in your control, but PIPEDA and provincial laws, such as Quebec's Law 25, still apply to personal information used in an AI system. This is general information, not legal advice. See AI governance and security.
Running it after launch
A private model is a system you own, so it needs owning. We can hand over to your team with a runbook, or operate the platform for you as part of managed IT services: patching, monitoring, capacity, backups and testing new model releases before they go live.
What is included
The exact list is agreed in writing for each project. These are the usual deliverables and the usual boundaries.
Typical deliverables
- A requirements review covering data sensitivity, residency, users, workloads, response times and accuracy targets.
- A shortlist of open-weight models tested on your own questions and documents, with licence terms checked for commercial use.
- A hosting design, whether GPU servers in your own cloud account (such as AWS, Microsoft Azure, Google Cloud or DigitalOcean), a dedicated GPU server or hardware on your premises, sized for your users.
- An inference server, such as vLLM or SGLang, that exposes an OpenAI-compatible API, so applications and tools can use your private endpoint instead of a public one.
- Private networking, single sign-on, encryption and no public exposure of the model endpoint.
- Logging set to your retention rules, including the option to keep no prompt content at all.
- Retrieval over your documents (RAG) and connections to Microsoft 365, file shares, Prometheus or ERPNext and other systems.
- Model files from the developer's official source in the safetensors format, pinned and recorded, and testing for prompt injection and data leakage.
- Monitoring of availability, response times, GPU use and cost, and a runbook for updates and model changes.
Not included unless agreed separately
- GPU servers, cloud usage and hardware, which you buy from the provider. We can manage them for you.
- Training new models from scratch.
- Legal advice on model licences or privacy law. We flag terms for your legal adviser.
- Ongoing operation after handover, unless agreed in a support or managed services agreement.
What we will need from you
Most delays in this kind of work come from access and decisions, not from the technical build. Knowing these early keeps the project predictable.
- Your data classification and your privacy, security and residency requirements.
- A cloud account or data centre where the servers will run, or approval to set one up.
- Example questions and documents, with the answers you expect.
- A business owner who decides whether answers are good enough.
Delivery
How the work is delivered
Each stage ends with something you can review before the next one starts.
Assess
Confirm what data the AI will touch, where it must stay and who will use it, and whether private hosting is needed at all or enterprise terms are enough.
Output: Recommendation, model shortlist and hosting options.
Prove
Deploy candidate models on a temporary GPU server in your environment and test them on your own questions, alongside a commercial model where policy allows.
Output: Accuracy, response time and running cost results.
Build
Set up production servers, private networking, sign-in, logging, retrieval over your documents and the integrations you need.
Output: Production-ready private AI platform.
Pilot
Release to a pilot group, review answers and feedback, and tune models, prompts, retrieval and capacity.
Output: Pilot report and improvements.
Run and improve
Monitor, patch and back up the platform, and test new model releases against your evaluation set before switching.
Output: Runbook, monitoring and an update plan.
Testing and handover
- The model endpoint is reachable only from your network or approved applications, and every user signs in.
- Tests confirm that no prompts, files or outputs are sent to an outside AI provider.
- Accuracy meets the agreed threshold on your evaluation set.
- Response times meet the agreed target at expected peak use.
- Logs follow your retention rules, and administrators know how to change them.
- Running costs are documented, with a plan for scaling up or down.
What affects the cost
We do not publish package prices. Each estimate is based on an agreed scope, in Canadian dollars, with taxes shown separately. These are the things that move the number most:
- The size of the model and the GPU memory it needs.
- The number of users, peak load and response time target.
- Whether servers run around the clock or only when needed.
- Cloud GPU servers, a dedicated server or hardware you own.
- Retrieval over documents, integrations and security requirements.
- Ongoing operation, monitoring and model updates.
Questions buyers usually ask
What is an open-weight model?
A model whose developer publishes the trained weights, so anyone can download it and run it on their own hardware under a licence. Well-known families include Meta's Llama, Alibaba's Qwen, DeepSeek, Z.ai's GLM, Mistral, Google's Gemma, OpenAI's gpt-oss and Cohere's Command, from Toronto. Licences differ, even between versions of the same family, from permissive ones such as Apache 2.0 and MIT to custom licences with conditions, so we check the licence of the exact version before you rely on it.
Are open-weight models as good as ChatGPT or Claude?
For many business tasks, such as answering from documents, summarizing, drafting and extracting data, current open-weight models perform well. Independent evaluators report that the best open-weight models trail the leading commercial ones by a few months on average, and remain further behind on the hardest reasoning and coding work. We test candidate models on your own questions before you commit, so the decision rests on evidence rather than benchmarks.
If we run DeepSeek or Qwen, does our data go to China?
Not when you host the model yourself. A self-hosted model runs entirely on your servers and does not send prompts to its developer. Using the developer's own app or API is different. The federal government and British Columbia restricted the DeepSeek app on government devices in 2025; those measures targeted the app, not self-hosted models, but your own policies or clients may restrict certain developers, so we check before recommending one and test every model's security.
Where can the model run?
On GPU servers in your own cloud account, on a dedicated GPU server, or on hardware in your data centre or office. In Canada, GPU servers are available from providers such as DigitalOcean in Toronto, AWS in Montréal and Google Cloud in Montréal and Toronto, though GPU types differ by region. Where data residency matters we confirm the region of every component and document where everything runs.
Is self-hosting cheaper than paying per use?
Only at steady, high volumes. A GPU server costs the same whether it answers ten questions or ten thousand, while an AI service charges per use. For light use, an enterprise service with zero data retention is usually cheaper. Our comparison of self-hosted models and AI APIs explains how to work it out.
Can our existing tools use the private model?
Usually, yes. The inference servers we deploy offer an OpenAI-compatible API, so many applications, coding tools and frameworks that work with commercial AI services can be pointed at your private endpoint instead. Compatible is not identical, since chat formats and tool calling differ between models, so we test each integration.
Related
Self-hosted AI model or AI service?
When to run an open-weight model on servers you control and when an AI service with strong terms is the better choice, compared on privacy, quality, cost and effort.
Generative AI development and custom AI assistants
Custom AI assistants and applications that answer from your approved documents and data, with sources, permissions and testing built in.
Need AI that never leaves your environment?
Tell us what data the AI must work with, where it must stay and who will use it. We will reply to arrange a conversation and suggest whether private hosting, enterprise terms or a mix of both fits best.