Organization administrator configuration
Ada does not include a hosted AI service. After installing the extension, an Azure DevOps organization administrator connects it to a compatible backend that the organization operates (see DEPLOY.md). Until then, users see "AI features are not configured for this Azure DevOps organization" and the non-AI features keep working.
Configure
- In Azure DevOps, open Organization settings > Ada AI, tab Backend settings (or select Ada settings > Admin settings at the right end of Ada's bar). Azure DevOps decides who can open Organization settings; Ada adds no check of its own.
- Enter:
- Backend URL:
https://only. It must not contain credentials, a query string or a fragment; trailing slashes are removed. - Display name.
- Expected API version: default
1.0. Backends at 1.1 or later of the same major version also pass; features added in 1.1 appear when the backend offers them. - Trusted hostnames (optional): an allow-list policy such as
*.your-organization.example. - Enable AI features.
- Backend URL:
- Select Test connection. Ada calls
GET /api/metadatawithout following redirects. It checks the response status, content type, service identifier, API version, required capabilities and that authentication is required. - Select Save configuration.
- A backend is enabled only after a successful test.
- Unreachable backend: a backend that cannot be reached can be saved only by ticking the override, and then stays disabled.
- Replacing a backend: needs an explicit confirmation. Every user must then confirm the new backend before data is sent to it.
Only non-secret settings are stored. Never enter API keys, client secrets, tokens or Key Vault values here; these fields don't take them, and the stored document rejects unknown fields. Provider keys belong in your backend's secret store, or, on a backend someone runs for you, in the card below.
Your AI provider's key (a backend run for you)
Where someone runs your backend for you, such as Valor Consulting with Ada as a managed service, the backend can take your AI provider's key from Ada, so the key never has to be sent to anyone. Backend settings then shows Your AI provider's key under the backend's address.
- Paste the key, and the one-time setup code you received from whoever runs the backend.
- Select Test and save. The backend tries the key with your provider first and keeps it only if it works: encrypted, never shown again and not stored in Azure DevOps. Ada then shows when it was set and its last four characters.
- A setup code works once and expires, usually after 7 days. To change or remove the key later, ask for a new code. Where the backend knows you as an administrator (it has its own Azure DevOps access), no code is needed.
- The provider's address is set where the backend runs and can't be changed from Ada: whoever could change it could send everyone's questions somewhere else.
- Where the embeddings for search have a key of their own, a second field takes it.
Each change records who made it and when. The last 20 changes are listed under Change history.
Where settings are stored
| Data | Storage | Scope |
|---|---|---|
| Backend URL, display name, enabled, API version, trusted hostnames, change history | Azure DevOps extension data, collection scope | This organization only |
| "I confirm this backend" acknowledgement | Azure DevOps extension data, user scope | Per user |
| Backend session token | The browser tab's sessionStorage (memory only where the browser blocks it) |
Until it expires (SESSION_TTL_MINUTES), the tab closes or the backend rejects it |
| Project templates | Azure DevOps extension data, collection scope, one document per project | That project; changes need review and approval in Project Templates |
| Pull-request check settings (repositories, Wiki scope, reviewers) | The organization's backend | This organization only |
| AI provider key entered in Ada (a backend run for you) | The organization's backend, encrypted (AES-256-GCM); never returned | That backend only |
Nothing else is stored in the browser: no localStorage and no cookies.
Who can change the configuration: platform limitation
Azure DevOps extension data offers no access control: any user who can run the extension can technically write collection-scoped extension data with their own credentials. The page lives in Organization settings, which Azure DevOps shows according to its own permissions, but that does not restrict the stored document either.
Ada protects users with checks that do not depend on who wrote the configuration:
- User confirmation. Before any content is sent, each user sees the backend's name and hostname, plus who configured it and when. They must confirm it the first time and after every change.
- The backend authenticates and authorizes every request.
- Without a separate sign-in (
AUTH_MODE=azure-devops, the default), Azure DevOps itself confirms each user in the organization when a session starts; the backend accepts only the configured Azure DevOps organizations. - With a Microsoft sign-in (
AUTH_MODE=entra), users sign in with the organization's Microsoft Entra ID, and the backend accepts only its own tenant, the configured Azure DevOps organizations, and the configured users, groups or roles. It verifies organization membership with Azure DevOps at sign-in. - A rogue URL can't impersonate your backend's sign-in. Restricting user consent to unverified applications in Entra ID blocks look-alike sign-ins.
- Without a separate sign-in (
- Trusted hostname policy, stored URL validation and HTTPS are rechecked on every load.
- Tamper detection. A malformed or tampered configuration document is rejected.
Recommended: restrict user consent in Entra ID, use a trusted hostname policy, and review the change history regularly.
Setup
Organization settings > Ada AI > Setup shows what your organization's backend offers:
- What the features build on: an AI provider Ada's agents can run on (Claude), storage that survives restarts, and the backend's own Azure DevOps access, each Ready or Missing.
- Features, each On or Off. For a feature that is off, the (i) next to it says what is missing, by the name of the backend's setting.
- Check the Azure DevOps access asks Azure DevOps with the backend's own identity whether you are an administrator of the organization, and says what it found. Until that identity is among your organization's users (Organization settings > Users > Add users, access level Basic), the check fails and Ada knows no administrators.
Nothing is changed from this page. The settings are the backend's own: whoever runs it sets them where it runs (in Azure: App Service > Settings > Environment variables) and restarts it. Everyone who can open Ada's organization settings sees this page; it shows no setting's value.
Pull-request documentation checks
Shown under Organization settings > Ada AI > Pull-request documentation checks when the connected backend offers them; its operator switches them on (see DEPLOY.md). Changing them requires membership in Project Collection Administrators (when the backend has its own Azure DevOps access) or the backend's administrator app role (Ada.Admin by default); everyone else sees them read-only.
- Repositories. Add repositories and choose per repository whether completed pull requests are checked automatically or only on demand.
- Wiki scope. The Wikis, optionally limited to a path, that Ada reads and proposes changes for.
- New pages are proposed under. The parent path for release notes and upgrade and migration instructions.
- Reviewers to notify. Their e-mail addresses. Ada resolves them to Azure DevOps users and @mentions them on the completed pull request when it has prepared proposals.
Proposals and the pull-request metadata behind them are visible to everyone in the organization who uses Ada. Enable only repositories and Wikis whose documentation impact may be shared that widely.
Sprint documentation
Shown under Organization settings > Ada AI > Sprint documentation when the backend has its own Azure DevOps access. Who may change it is decided as for the pull-request documentation checks above. Nothing runs until you add a team.
- Add a team. Choose the project and team, the Wiki and the folder for the pages, the documents (sprint documentation and release notes are preselected; checklists, testing summary and the other types are available), and optionally the days to wait after the sprint.
- Automatic. When a team's sprint is over, Ada reads its completed work items and prepares each document by itself. No one has to start it. Saving checks that the sprints can be read, so a missing permission shows at once.
- Approval. The documents wait under Documentation > Review Pull Request Documentation (origin "End of sprint"). A page is created when someone approves it, with their own Wiki permissions. Nobody is notified, so someone needs to look there at the end of the sprint.
- Generate last sprint. Prepares the documents of the team's last ended sprint now: for trying it out, and for a sprint that ended before the team was added, which is otherwise left out.
- Latest documents. What happened to each one, including why one was not generated: no completed work items, or a page with that name already exists.
Azure DevOps keeps no sprint goal, so the documents mark it TODO. A sprint counts as over when its last day has ended in UTC; set the days to wait if your team closes work later than that. Each team needs its own Wiki folder, because sprints of different teams can have the same names.
Search and diagnosis
Shown under Organization settings > Ada AI > Search and diagnosis when the backend has Azure AI Search (DEPLOY.md). Changing it requires the same administrator rights as the pull-request checks.
- Index status. Ready, the passages indexed, the last run's counts and problems, and Rebuild the index.
- Projects to index and Repositories left out. Everything else of these projects is indexed: Wikis, default branches, work items.
- Boards and their keywords. A board is an area path, the same list as Create Sprint Schedule's Board field; it covers the areas below it. With several indexed projects, Board's project chooses whose boards and repositories are listed (default: the project Ada is open in). Questions and tickets that name its keywords are searched in its area, repositories and log services first.
- Logs and traces. Grafana's data sources Ada may read, with a Test that shows a masked sample and the service names found; a failed test names Grafana's reason. Tempo is read through Grafana's data source proxy (Tempo's own search and trace API, as Explore does). It follows trace IDs; for other request IDs, set Attribute that holds the request ID (for example
span.http.request_id). - Diagnose a Ticket. Where new Bugs and User Stories go (project, default area from that project's areas, iteration) and the L2 support group named in handover notes. When the board a diagnosis points to is in another project, the work item gets the default area instead.
Ada's agent
Shown under Organization settings > Ada AI > Ada's agent when the backend has Ada's agent in the chat. Only organization administrators see and change what is here.
Skills
A skill is your organization's own guidance for a kind of work: how you write release notes, what a bug report needs, how you name branches. Skills are files in one of your repositories, in the open Agent Skills format that coding tools read too.
In a repository, make a folder for the skills, for example
/skills. Each skill is a folder in it that holds aSKILL.md:--- name: release-notes description: How we write release notes. Use for release notes and changelogs. --- Start with what changed for users. One line per change, newest first.The name takes lower-case letters, digits and hyphens; where it is missing or isn't one, Ada takes the folder's name. The description says when to use the skill: Ada decides by it.
Under Skills, enter the project, the repository and the folder, and choose Check the folder. Ada reads it with your own access and lists the skills, and the folders that aren't one with the reason (no
SKILL.md, no description).Save. Stop using skills removes the folder from Ada's settings; the files stay.
What to know:
- What the AI model gets. The names and descriptions of all skills go with every message, which costs tokens: keep descriptions short. A skill's text is read only when a task matches, up to 30,000 characters, with values that look like keys or passwords masked. Ada takes the first 30 skills of a folder and reads the repository's default branch. A change to the files shows within five minutes.
- Whose access. In the chat, Ada reads skills with the asking person's own access: someone who can't read the repository gets no skills. Routines read them with the backend's access.
- Who decides how Ada works. Whoever can change the files in that folder changes how Ada works for everyone. Protect the branch with a policy, as you do for code. A skill guides how Ada works; it never lets her change anything.
Routines
Shown when the backend offers them: they need Claude as the AI provider, the backend's own Azure DevOps access and storage that survives restarts. Where they are missing, the Setup tab says which.
A routine is work Ada's agent does on its own, on a schedule or when a pull request merges: a weekly list of stale bugs, a note on what each merged change means for the people who run the service.
- Add a routine. Give it a name, the project it works in, and the task, in words, as you would ask it in the chat. Nobody is there to answer a question back, so say what matters: the area, the period, the repository.
- Choose what starts it:
- A schedule: every day, Monday to Friday or once a week, and the hour it starts, in UTC. A run starts within a minute or two of its hour; if the backend was stopped then, it runs once when the backend is back.
- A merged pull request: the routine runs once for each pull request that completes in its project. You can limit it to one repository and to merges into one branch, for example
main. Ada is told which pull request it was and reads it (title, description, changed files, commits, linked work items); the task says what she should do about it. Pull requests that merge while a run is under way wait their turn, up to 20; more are left out and noted in the backend's log.
- Choose who sees its reports: administrators only, or everyone who can see the routine's project. A routine that a merged pull request starts can also post the report as a comment on the pull request (ticked for a new routine).
- Save routine. Run now starts it at once, also while it is switched off; a routine that waits for pull requests then takes the one that merged last. The list shows when it runs next or how many pull requests wait, its last report or why the last run failed, and under Earlier reports the ones before it.
Routine Reports. Each routine's history, run by run, is on the Routine Reports page on the Others tab of Ada: the routines of the project Ada is open in, what started each run (with a link to the pull request), whether the report was posted there, and the report. Administrators see every routine; everyone else sees the routines shown to the project.
A merged pull request can start a routine only where Azure DevOps tells the backend about pull requests: whoever runs the backend sets PR_DOCS_HOOK_SECRET and adds the service hook in the project (DEPLOY.md). It is the hook the pull-request documentation checks and Repository Documentation use; where it is missing, the form says so and offers schedules only.
What to know:
- Whose access. A routine reads Azure DevOps with the access you gave Ada's backend, not with a person's. Its report can hold whatever that identity can read, so the reports are for administrators unless you show them to the project. Shown to the project, or posted on a pull request, what looks like personal data, a key or a password is masked; administrators see the report as it was written. Show a routine to the project only when its reports are for everyone there. The newest thirty reports of a routine are kept; removing the routine removes them.
- The comment on the pull request is posted with the backend's own Azure DevOps identity, which needs Contribute to pull requests on the repository. Where it can't post, the report is still kept, and its run says so.
- It changes nothing. A routine has only tools that read: search, files, work items, pull requests, Wiki pages, logs, and the tools of your connections ticked for Ask Ada. It prepares no proposals. Where the task asks for a change, the report says what should change and where.
- Skills and project notes apply to a routine as they do in the chat.
- Cost and limits. Each run costs tokens like a question in the chat, is counted under Monitoring as "Ada's agent (routines)" and against the quotas of the administrator who last saved it. A run ends after 40 steps of the AI model or 12 minutes, and says so in its report. An organization can have 10 routines. A routine that every merged pull request starts runs as often as your teams merge: limit it to a repository or a branch where that is too often.
Connections
Shown under Organization settings > Ada AI > Connections when the backend has Ask with search and an AI provider that works with tools (Claude). A connection is an outside system that offers a Model Context Protocol (MCP) server: monitoring, logs, or any other. Ada may ask it while she answers in Ask Ada and Diagnose a Ticket. Only organization administrators see and change connections.
- Add a ready-made kind, or Another MCP server.
- Azure Managed Grafana: enter your Grafana's address followed by
/api/azure-mcp. Ada signs in with the backend's own identity: give that identity the Grafana Viewer role on your Grafana. Azure public cloud only, and not through a private endpoint. - Another MCP server: its HTTPS address, and how Ada signs in: without a sign-in, or with a secret in a header you name (sent as a bearer token in
Authorization, as it is in any other header).
- Azure Managed Grafana: enter your Grafana's address followed by
- Test and list the tools. Ada asks the server for its tools and shows each with what it does: Reads (its server says so), Its server doesn't say whether it only reads, or Changes something: not offered.
- Tick the tools Ada may use, at most 20 per connection. Every ticked tool is described to the AI model with each question, so tick what is needed and no more.
- Choose where it is used (Ask Ada, Diagnose a Ticket, and, where the backend offers Ada to coding tools, Coding tools) and Save connection.
What to know before you tick:
- Ada's agent in the chat uses the tools ticked for Ask Ada too.
- Whose access it is. Ada asks with the connection's sign-in, not with the asking person's. Everyone who can use Ask Ada or Diagnose a Ticket gets answers that draw on the ticked tools.
- Coding tools. Ticked for coding tools, the tools are offered to every developer who connected their coding tool to Ada (Ada in Your Coding Tool), next to Ada's own. Their coding tool calls them with the connection's sign-in, as Ask does. What a tool returns then goes to the AI their coding tool uses, which is not your backend's AI provider: personal data and what looks like a key or a password are masked and stay masked. Tick for coding tools only what every developer may see.
- Tools that change something are never offered: those their server marks as changing, those named like it (create, update, delete, send and the like), and the write tools of the ready-made kinds. A tool whose server says nothing is yours to judge: read its description (the (i) next to it) before you tick it. Query tools that run a statement the AI writes, such as SQL, are only as safe as the rights of the account behind them.
- What the AI model is told about a tool is what the server said when you saved the connection. If the server changes a tool later, Ada keeps the saved description until you save the connection again.
- Secrets are kept encrypted in the backend's storage, never shown again, and sent only to the server they were saved for: after changing the address to another server, enter the secret again. Where the page says the backend can't keep secrets, whoever runs it sets
SETTINGS_ENCRYPTION_KEY(DEPLOY.md). - Addresses must be HTTPS and may not lead to the backend's own machine. Redirects aren't followed.
What goes where: DATA-HANDLING.md.
Code
Shown to organization administrators under Organization settings > Ada AI > Code when the backend offers Code with Ada (DEPLOY.md).
- Who has it, besides organization administrators. Nobody by default: only Project Collection Administrators (and the backend's admin application role) see the card. Teams and groups added here see it too, nested groups included, on the Code tab; the backend refuses everyone else on every route. This is the feature's own rule: Access by tab neither opens nor closes it, so a team allowed here gets the Code tab with this card even where the tab is limited to others.
- Projects and repositories. What Ada may change. Nobody can run it on anything else, and a person also needs Contribute on the repository in Azure DevOps.
- Reviewers of the pull requests. Teams and groups added to each pull request, required or optional. Without any, the branch policies decide; who asked for the change is always added as an optional reviewer.
- Target branches. Pull requests go to the first of these that exists in a repository, otherwise to its default branch.
- Protected folders and files. Changed only when the approved plan or the person names them. Pipelines and agents' instruction files always are.
- Models. For the plan and its check, for code and tests, and for review, at high effort. Where none is set, the person's own choice or the persona's, and otherwise the largest the backend offers. A smaller model for code and tests costs about half, though a failed round can cost more than that saves.
- Fix rounds. How often Ada fixes what her reviewer found (2 by default) and what failed in the checks (3 by default) before a person decides.
Repository documentation
Shown under Organization settings > Ada AI > Repository documentation when the backend offers Repository Documentation (DEPLOY.md). Everyone sees the settings; changing them requires the same administrator rights as the pull-request checks. Until they are saved once, no repository is enabled.
Projects whose repositories may get documentation, and per project the repositories to leave out. Users choose among the enabled repositories they can see.
Reviewers of the pull requests. Search Azure DevOps teams and groups and add up to 10; each pull request gets them, as Required reviewers unless you clear that option. The person who started the run is added as an optional reviewer. At least one team or group is needed: the settings can't be saved without one once projects are enabled, a run doesn't start without one, and when none of them can be found any more, a repository's pull request isn't opened (its files can be downloaded, and Retry uses the settings as they are then).
Documentation folder,
docsby default: where the documentation set goes in each repository. Files people wrote there keep their text; Ada's part of a file sits between markers.Also correct what the code contradicts in README files, CLAUDE.md and the folder's documents (on by default): a second commit in the same pull request.
Open as drafts: reviewers are notified when someone publishes the pull request.
Security assessment: Full detail (the default), Summary only (findings without code or analysis) or Left out. The documentation goes into the repository, so everyone who can read the repository reads the findings: choose Summary only where that audience is wider than the people who should see how to exploit them.
Keeping the documentation up to date. Update the documentation when a pull request merges into the default branch (on by default) applies to repositories that already have documentation, on the default branch or in Ada's open pull request. It needs an Azure DevOps service hook: Web Hooks, event Pull request updated (and optionally Pull request merge attempted), on the project or the organization, to
POST /api/hooks/pull-requeston the backend with the headerx-ada-hook-secretset toPR_DOCS_HOOK_SECRET. The hook of the pull-request checks is the same one; a backend without those checks needs only the secret. The rules choose the model for a change: Up to so many changed code files (tests aren't counted), More files, Paths that hold certain words whatever the size, and the Effort. A model not chosen is the default shown. A run someone starts uses these rules too, for the code changes it takes up.Dev documents (off by default). Two documents in each enabled repository that has a dev branch, by pull request into that branch:
main-dev-differences.mdandunreleased.md. Folder (dev-docsby default, outside the documentation folder), Dev branches (the first a repository has; a repository with none is left out), Repositories with another dev branch, Every so many days (14 by default) Starting on a date (a day or two before a release is the useful moment; empty: one interval from now; a past day doesn't count, today does; besides the schedule, each merge into the dev branch or the default branch writes them too, through the service hook of the pull-request checks), the Model (the smallest the backend offers unless you choose) and their own Reviewers. With no team or group, nobody is asked to review and each pull request completes by itself once the dev branch's policies pass; a policy that wants a person still waits for one. This applies to the dev documents only: the documentation always asks a team or group. Check that the pipelines of the dev branch ignore this folder, like the documentation folder.
Without the hook nothing updates by itself; a merge that was missed is taken up by the next one, or by a run someone starts, because an update starts from the commit the documents follow. Updates are shown as started by Ada (automatic), and only administrators can stop, discard or retry them. They run on what the backend's identity can read, not a person's rights, like the rest: enable only repositories that everyone who uses Ada may read.
The backend reads the repositories with its own identity. It checks that the user who starts a run may read the chosen repositories, and the checks of findings against connected services read only enabled repositories that user may read; quotes from them can appear in the documentation of the repository being documented. Everyone with the Code tab sees every run: repository names, pull-request links and counts of findings by severity (limit that with Access). Only who started a run and administrators can stop, discard or retry it or download its files. Enable only repositories that everyone who uses Ada may read. The backend's identity needs Read, Contribute, Create branch and Contribute to pull requests on them, and if your repositories restrict branches to folders, in the ada/docs folder (ada/dev-docs for dev documents). A run asks Azure DevOps about this first, before anything is read or any AI request is made: a repository where something is missing ends as Failed, saying what, and the others go on; grant it and select Retry. If Azure DevOps still refuses the push or the pull request at the end, the repository ends as Needs permission, and its files can be downloaded instead.
Ratings
Organization settings > Ada AI > Monitoring shows, below the runs per quick action, what people said of Ada's results. Under an answer in Ask Ada and under a ticket's diagnosis, everyone can choose Right, Partly right or Wrong and add a note.
- By quick action and by AI model: how many results were right, partly right and wrong in the period, and the share that was right. Each rating is kept with the quick action, the model and the version of Ada's backend, so you can see whether another model or a new version made answers better.
- What people said: the newest ratings that carry a note or weren't "right".
- No names. A rating says nothing about who gave it.
- Keep what was asked and what Ada answered with each rating is off until you tick it. Off, you see how often Ada is right, not about what. On, each new rating keeps the question and the answer (up to 16,000 characters, with keys and passwords masked) in your backend's storage; people are told before they rate. Questions can name people and customers: decide with whoever answers for data protection in your organization. Turning it off removes what was kept.
Access
Shown under Organization settings > Ada AI > Access when the backend has its own Azure DevOps access (ADO_AUTH_MODE) and, in production, STATE_STORE=blob. Only administrators (Project Collection Administrators or the backend's admin role) see and change it.
- One rule per tab, for the whole organization: Wiki, Work Items, Documentation, Code, Support, Requirements, Others, and Ada on work items (the panel on work item forms). A tab without a rule is open to everyone.
- Add to
finds Azure DevOps teams (any part of "[Project]\Team") and groups (the beginning of their name). Once a tab has a team or group, only their members, also through nested groups, and administrators can use it. Administrators always have every tab. - Membership is Azure DevOps'. Add and remove people in the team as usual; Ada applies a change within about 5 minutes, and saved rules within a minute. Entra groups count when they were added to Azure DevOps.
- The backend enforces it. Ada hides the tabs a user doesn't have, and the backend refuses their requests. Ask Ada and Ask about Work Items share the chat: it's there with either tab, and searches only the sources of the user's tabs (Repos needs the Code tab). Project Templates and Create Sprint Schedule (Others) run in the browser with the user's own Azure DevOps permissions: limiting Others hides them.
- When Azure DevOps can't be reached, limited tabs stay closed until it can; open tabs keep working.
- One quick action (backend 1.64.0). Quick actions under a tab lists its actions; Add to
limits that one action to its teams and groups, while the rest of the tab stays open. A quick action also needs its tab. Ada hides it from everyone else and the backend refuses its requests, also through Ada's agent and coding tools. Ask Ada and Ask about Work Items share the chat: it's there with either. Code with Ada keeps its own rule, in its settings. - Read-only Ada (backend 1.64.0). Ada changes nothing in Azure DevOps, for everyone, administrators too: no Wiki pages, work items, sprints, templates, branches, pull requests or comments. Ada hides the quick actions that would change something and says why; the backend refuses every change, its own jobs included (Code with Ada, repository documentation, comments of pull-request checks and routines). Answers, reviews, diagnoses, mockups and reports keep working.
Outbound connections
Under Organization settings > Ada AI > Monitoring (backend 1.64.0), administrators see every outside address the backend called in the last 30 days: how many requests, on how many days, and the last day. Only addresses and counts are kept, never what was sent. localhost is a container beside the backend, such as web search; the search engines it asks in turn aren't counted. AdaDocs' Security and trust page lists what to expect.
Disable AI
- For the organization: clear Enable AI features and save.
- Backend side (all organizations it serves): set
AI_DISABLED=true, or remove the organization fromALLOWED_ORGANIZATIONS.
Administrator responsibilities
- Operate or select the backend. Its AI provider, subscription and costs belong to your organization.
- Set usage limits, allowed organizations and allowed users or groups on the backend (see DEPLOY.md).
- Confirm that your AI provider's data-processing terms cover the content listed in DATA-HANDLING.md.
- Rotate backend secrets (procedure in DEPLOY.md).
