Buyer guide

Using AI for IT questions: where it helps and where it fails

AI assistants explain, draft and brainstorm well, but fail where the answer depends on your systems, current vendor behaviour or live data. Check production changes with someone who sees your environment.

The short answer

AI helps with questions that have a general answer: terms, how a technology works, usual options, policy wording. It fails where the answer depends on what it cannot see: your configuration, licence tier, vendor's current interface, compliance obligations and what else breaks. The rule: use AI to understand, a professional to decide and change, especially near live systems, security or personal information.

Where AI genuinely helps

We happily tell clients to start with AI here: a wrong answer is cheap to spot and undo.

Learning the vocabulary

Conditional access, VLANs, RPO versus RTO, an ERP "document type".

Comparing approaches

Cloud or on-premise backup, per-user or per-device licensing, fully or co-managed IT.

Drafting for review

Acceptable-use policies, onboarding checklists, project briefs, staff emails about a change.

Reading and summarizing

A long vendor article cut to the three points you need to discuss.

Preparing questions

Before meeting a vendor, auditor or IT provider.

Personal, reversible tasks

A spreadsheet formula, a rule on your own inbox, a keyboard shortcut.

Where it predictably fails

The failures cluster in a few places:

Failure modeLooks likeWhy
Out-of-date stepsMoved or renamed menusInterfaces change; training lags
Generic adviceAssumes a licence you lackCannot see your environment
Confident wrong commandsScript runs at the wrong scopeCannot see output or tenant
Missing side effects"Fix" disables logging, backups or a controlOptimizes for your symptom
Invented detailNon-existent setting or policy namePlausible is not verified
No Canadian contextAnother country's privacy lawGeneral training

The Canadian Centre for Cyber Security lists inaccurate content and "buggy code" among generative AI risks and advises verifying against credible sources (Cyber Centre ITSAP.00.041).

How to ask better questions

  1. Context, not secrets"Microsoft 365 Business Premium, about 40 users, hybrid", not your tenant configuration.
  2. Ask for options"Three ways to do this, and what could each break?"
  3. Ask its gapsWhat must it confirm about your setup? Often the most valuable answer.
  4. Get the sourceThen read the vendor's own documentation before acting.
  5. Remove personal informationUse anonymized or de-identified information where possible, as the OPC suggests.
  6. No credentialsNever paste passwords, API keys, licence keys or logs containing user data.

A traffic-light guide

Green: fine to rely on AI, with a quick sense check

  • Explaining a concept or term
  • Drafting text a person will review
  • Personal productivity tasks you can undo

Amber: use AI to prepare, then confirm with the vendor's documentation or an IT professional

  • Choosing between products or licence plans
  • Planning a change to a shared system
  • Interpreting an error message on a business system

Red: do not act on AI output alone

  • Changes to firewalls, identity, email authentication or security policies
  • Anything touching backups, retention or deletion
  • Scripts that run across all users, mailboxes or records
  • Decisions about privacy, breach reporting or regulatory obligations
What usually goes wrong in practice

Rarely a badly wrong AI answer; usually a reasonable one applied by someone who could not tell when it did not fit:

PatternResult
Invisible side effectTurning off a protection to stop a nuisance prompt fixes the complaint and quietly weakens security
Half-applied changeStep four fails, nobody can roll back steps one to three, and the system is left undocumented
No recordMonths later nobody knows who changed a setting or why, slowing every future fix

Rather talk it through? If you have an AI answer that falls in the amber or red zone, we can check it against your actual systems before anyone runs it. Talk to a Promatics specialist

When to bring in help

AI plus common sense covers learning and low-risk, reversible tasks. Bring in a professional when a change affects more than one person, involves security or personal information, is hard to undo, or needs planning across systems. That is IT consulting: someone who knows your environment, weighs trade-offs with you and is accountable for the recommendation.

For AI that answers staff IT and policy questions from your own documentation, with sources and permissions, see generative AI development and our AI Solutions practice.

Sources and further reading

Product capabilities and guidance change. These are the primary sources this article relies on, checked on the review date above.

  1. Generative artificial intelligence (ITSAP.00.041), Canadian Centre for Cyber Security
  2. Principles for responsible, trustworthy and privacy-protective generative AI technologies, Office of the Privacy Commissioner of Canada

This article is general information, not legal, accounting or security advice for your specific situation. Examples are hypothetical unless stated otherwise.

Talk to Promatics

Get a second opinion before you change production

AI answers are a good start, but you carry the risk. We check the question against your environment and say what we would do.

  • Advice checked against your real configuration and licences
  • A clear recommendation, including what we would not do
  • Help making the change safely, if you want it