Telling an AI agent what not to do usually backfires

We were testing an HR assistant built in Copilot Studio when a tester asked a simple question: "If my last day is on 15 August 2026, when will I receive my pay?" The agent answered with confidence that final salary would be paid on 25 August 2026.

The answer was wrong, and as far as the agent's instructions were concerned, it was allowed. That one answer changed how we write rules for agents.

A ban only covers what you listed

The staff handbook says salary is paid on the 25th of each month, but it says nothing about staff who are leaving. The agent took a real clause and stretched it to a case the handbook never covers.

We had already written a rule to stop the agent working out final pay. It said: no calculations, no pro-ration formulas, no dollar amounts and no day counts.

It said nothing about dates, so the agent gave a date. It broke none of our rules, yet it did exactly what the rule was there to prevent. A ban only blocks what you remembered to list.

Ban a phrase and you get a synonym

Next, we added a line forbidding the agent from saying the handbook "does not specify" something. Its very next answer said the handbook "only specifies the regular monthly pay day".

Different words, same meaning. If we banned that phrase too, the agent had plenty of others to choose from. It could say the handbook "covers regular pay but not final pay", or that it "is silent on final pay". You can keep adding to the list, but the model will always find one more way to say it.

Your rules can end up in the answer

One routing rule told the agent not to send login problems to the IT team. The agent passed that on to a staff member, almost word for word, as if it were advice.

A prohibition in the instructions is still text the model can repeat. Unless you tell the agent a rule is internal, assume a user may one day read it.

In a description, "exclude" is a keyword

In Copilot Studio, a topic is a ready-made conversation for one kind of question, and each topic has a short description of what it covers. When a user asks something, the agent reads those descriptions and picks the topic that sounds closest to the question. It matches on meaning, much like a search engine, rather than following the description as an instruction.

Our agent had a topic for claiming professional memberships, such as a subscription to a professional body. Staff asking whether they could claim their gym membership kept landing in that topic, because both questions are about claiming a membership.

To stop this, we added the line "Exclude gym membership." to the topic's description, and the problem got worse. The agent does not treat "Exclude" as a rule. What it does notice is that the description now contains the words "gym membership", so the topic looks like an even better match for gym questions. It is like printing "No cold callers" on your business card: you have still put cold callers on the card.

Microsoft's guidance does suggest stating what a topic does not cover. That helps the agent choose between two similar topics that both exist, but it does not help when there is nothing else to choose. In that case the agent still picks the closest match it has.

The real fix was to give gym questions somewhere better to go. We created a separate topic for self-help claims, the category that covers gym membership in the client's policy, so the agent had a closer match to pick. We also moved the exclusion into the agent's main instructions, where a "do not" is read as a rule rather than as a keyword.

What worked instead

Rather than listing bad answers, we described the shape of a good one. The rule that finally held was this:

Pointing staff to a document, page or link is fine. Describing what a document contains, covers, limits or omits is not. Never explain why you are routing.

The last sentence does most of the work. It closes off the whole category, including phrasings nobody has thought of yet, without needing a list.

The second fix was finding the rules that pulled the other way. Elsewhere in the same instructions, we had asked the agent to cite the document it used and to close with where to find more detail. Both invited exactly the document-talk we were trying to ban, and an agent told to do two opposite things will do a bit of both.

Add SharePoint lists on their own

Not every gap can be fixed with better rules. If some of the answers your agent needs live in a SharePoint list, pointing the agent at the whole SharePoint site is not enough, because the agent will not pick up the list.

A site URL covers the site's modern pages and its Word, PowerPoint and PDF files, not the lists inside it. In Copilot Studio, each list has to be added as its own knowledge source, and Copilot Studio then ingests it into Dataverse and indexes it. Microsoft covers the steps in Add SharePoint lists as a knowledge source. Dataverse search also needs to be turned on in the environment, or the list returns nothing.

By the end of testing, nearly every test case passed. Not all of that came from the instructions: stale documents and missing sources caused more failures than the model ever did. But the instruction fixes that held were the ones that described the answer we wanted, not the ones that listed what we did not.