Skip to content

Commit 6e7d485

Browse files
mattallfordmattallford
authored
Merge pull request #2765 from OctopusDeploy/ma/octopus-ai-assistant-deployment-failure-analyzer
Octopus AI Assistant - deployment failure analyzer and custom prompts documentation
2 parents 56a8353 + fc11e8a commit 6e7d485

5 files changed

Lines changed: 212 additions & 89 deletions

File tree

‎public/docs/administration/octopus-ai-assistant/deployment-failure-analyzer-custom-prompt-example.png‎

110 KB
Loading

‎public/docs/administration/octopus-ai-assistant/deployment-failure-analyzer-example.png‎

705 KB
Loading

‎src/pages/docs/administration/octopus-ai-assistant/custom-prompts.md‎

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
---
2+
layout: src/layouts/Default.astro
3+
pubDate: 2025-07-09
4+
modDate: 2025-07-09
5+
title: Custom prompts
6+
description:
7+
navOrder: 50
8+
---
9+
10+
Custom prompts allow you to tailor the Octopus AI Assistant to your organization's specific needs and business processes. Instead of relying solely on the AI's general knowledge, you can embed your internal documentation, troubleshooting procedures, and domain-specific guidance directly into the assistant's responses.
11+
12+
## Why use custom prompts?
13+
14+
Custom prompts are particularly valuable for:
15+
16+
- Platform teams providing self-service support to development teams with organization-specific guidance
17+
- Embedding internal documentation and troubleshooting procedures into AI responses
18+
- Standardizing responses across teams with consistent, approved solutions
19+
- Reducing support burden by providing context-aware, automated first-line support
20+
21+
For example, instead of getting generic advice about deployment failures, a custom prompt can direct users to your internal runbooks, specific team contacts, or approved remediation procedures.
22+
23+
## How custom prompts work
24+
25+
Custom prompts are defined using Library Variable Sets in Octopus Deploy and work alongside the AI's built-in knowledge. When a user interacts with the Octopus AI Assistant on a specific page, any custom prompts configured for that page will appear as suggested options.
26+
27+
There are two types of custom prompt variables:
28+
29+
- **Prompt variables** (`PageName[#].Prompt`) - The text displayed to users and sent to the AI
30+
- **System prompt variables** (`PageName[#].SystemPrompt`) - Optional additional instructions that guide the AI's response but aren't shown to users
31+
32+
## Adding custom prompts to Octopus
33+
34+
To add custom prompts to your Octopus AI Assistant:
35+
36+
1. Open the Octopus Deploy web portal
37+
2. On the main page for the space, click **Variable Sets**
38+
3. Click **Add Variable Set**
39+
4. Enter `OctoAI Prompts` for the variable set name
40+
5. Add variables in the new variable set using the naming convention below
41+
42+
### Variable naming convention
43+
44+
Variables must follow this format:
45+
46+
- `PageName[#].Prompt` - The prompt displayed in the UI and passed to the LLM
47+
- `PageName[#].SystemPrompt` - Optional additional prompt instructions passed to the LLM but not shown in the UI
48+
49+
Where:
50+
51+
- `PageName` is one of the supported Octopus Deploy page names (see [Supported pages table](#supported-pages) below)
52+
- `#` is a number from 0 to 4 inclusive for up to 5 prompts per page
53+
54+
For example:
55+
56+
- `Project.Deployment[0].Prompt` - A prompt displayed in the Octopus AI Assistant when viewing a project deployment
57+
- `Project.Deployment[0].SystemPrompt` - The system prompt for that deployment prompt
58+
59+
## Writing custom prompts
60+
61+
### Basic prompt structure
62+
63+
A basic prompt variable defines what users see and what gets sent to the AI. For example:
64+
65+
| Variable name | Variable value |
66+
|----------|-------|
67+
| `Project.Deployment[0].Prompt` | Why did the deployment fail? If the deployment didn't fail, say so. Provide suggestions for resolving the issue. |
68+
69+
This prompt relies on the AI's built-in knowledge and the deployment context (logs, process configuration, etc.) to provide an answer.
70+
71+
### Adding system prompts for business context
72+
73+
System prompts allow you to embed your organization's specific knowledge and procedures. The system prompt guides the AI's response without being visible to users.
74+
75+
For example:
76+
77+
| Variable name | Variable value |
78+
|----------|-------|
79+
| `Project.Deployment[0].SystemPrompt` | If the logs indicate that a Docker image is missing, You must only provide the suggestion that the user must visit <https://help/missingdocker> to get additional instructions to resolve missing docker containers. You will be penalized for offering generic suggestions to resolve a missing docker image. You will be penalized for offering script suggestions to resolve a missing docker image. You will be penalized for suggesting step retries to resolve a missing docker image. |
80+
81+
This system prompt is sent to the LLM to provide specific instructions on how to respond to the request, and:
82+
83+
- Detects a specific condition (missing Docker image)
84+
- Provides organization-specific guidance (internal documentation link)
85+
- Prevents generic responses that don't align with your procedures
86+
87+
## Supported pages
88+
89+
The following table shows all the pages where custom prompts can be configured. Each page corresponds to a specific area of the Octopus web interface, allowing you to provide targeted assistance based on what users are currently viewing.
90+
91+
| Page Name | Description |
92+
|-----------|-------------|
93+
| Dashboard | The main dashboard |
94+
| Tasks | The tasks overview |
95+
| Project | The project dashboard |
96+
| Project.Settings | The project settings |
97+
| Project.VersionControl | The project version control settings |
98+
| Project.ITSMProviders | The project ITSM settings |
99+
| Project.Channels | The project channels |
100+
| Project.Triggers | The project triggers |
101+
| Project.Process | The project deployment process editor |
102+
| Project.Step | An individual step in the deployment process editor |
103+
| Project.Variables | The project variables editor |
104+
| Project.AllVariables | The overview of all the project variables |
105+
| Project.PreviewVariables | The preview of all the project variables |
106+
| Project.VariableSets | The project library variable sets |
107+
| Project.TenantVariables | The project tenant variables |
108+
| Project.Operations | The project runbooks dashboard |
109+
| Project.Operations.Triggers | An runbook triggers |
110+
| Project.Deployment | The project deployments |
111+
| Project.Release | The project releases |
112+
| Project.Runbooks | The project runbooks |
113+
| Project.Runbook.Runbook | An individual runbook |
114+
| Project.Runbook.Run | A runbook run |
115+
| LibraryVariableSets | The library variable sets |
116+
| LibraryVariableSet.LibraryVariableSet | An individual library variable set |
117+
| Machines | The targets dashboard |
118+
| Machine.Machine | An individual target |
119+
| Accounts | The accounts dashboard |
120+
| Account.Account | An individual account |
121+
| Workers | The workers dashboard |
122+
| WorkerPools | The workerpool dashboard |
123+
| MachinePolicies | An machine policies dashboard |
124+
| MachineProxies | An machine proxies dashboard |
125+
| Feeds | The feeds dashboard |
126+
| GitCredentials | The git credentials dashboard |
127+
| GitConnections | The GitHub App dashboard |
128+
| Lifecycles | The lifecycles dashboard |
129+
| Packages | The built-in feed dashboard |
130+
| ScriptModules | The script modules dashboard |
131+
| StepTemplates | The step templates dashboard |
132+
| TagSets | The tag sets dashboard |
133+
| TagSets.TagSet | An individual tag set |
134+
| Tenants | The tenants dashboard |
135+
| Tenant.Tenant | An individual tenant |
136+
| Certificates | The certificates dashboard |
137+
| Environments | The environments dashboard |
138+
| Environment.Environment | An individual environment |
139+
| Infrastructure | The infrastructure dashboard |
140+
| BuildInformation | The build information dashboard |

‎src/pages/docs/administration/octopus-ai-assistant/deployment-failure-analyzer.md‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
---
2+
layout: src/layouts/Default.astro
3+
pubDate: 2025-07-09
4+
modDate: 2025-07-09
5+
title: Deployment failure analyzer
6+
description:
7+
navOrder: 10
8+
---
9+
10+
Every failed deployment is a blocker for DevOps teams. You can use the Octopus AI Assistant to analyze failed deployments, reducing the time you spend troubleshooting by providing immediate, context-aware analysis and remediation steps based on your specific deployment scenario. When a deployment fails, the analyzer gathers context about the deployment including logs, process configuration, and script content, and provides actionable suggestions to get your team unblocked faster.
11+
12+
## How the deployment failure analyzer works
13+
14+
The Deployment Failure Analyzer captures detailed information about deployments, including:
15+
16+
- Deployment logs and error messages
17+
- Deployment process configuration
18+
- Script content from deployment steps
19+
- Build information and artifacts
20+
- Environment and target details
21+
22+
This context is analyzed by the Octopus AI Assistant to identify the root cause of the failure and provide specific suggestions for resolution.
23+
24+
## Using the deployment failure analyzer
25+
26+
When a deployment fails, you can launch the Octopus AI Assistant from the deployment page. The analyzer will present a suggested prompt for analyzing the failed deployment:
27+
28+
```text
29+
Help me understand why the deployment failed. If the deployment didn't fail, say so. Provide suggestions for resolving the issue.
30+
```
31+
32+
The Octopus AI Assistant will analyze the deployment context and provide:
33+
34+
1. **Reason for failure** - The specific step and error that caused the deployment to fail
35+
2. **What happened** - A detailed breakdown of the deployment process and where it went wrong
36+
3. **Suggestions for resolving the issue** - Actionable remediation steps with specific commands and configuration changes
37+
4. **Next steps** - Recommended actions to investigate further and prevent future failures
38+
39+
## Example analysis
40+
41+
Below is a basic example of how the Deployment Failure Analyzer works in practice. The analyzer identified that an Azure Resource Group could not be found during deployment and provided troubleshooting guidance, including verifying the resource group exists, checking Azure account permissions, looking for typos in the configuration, and enabling step retries for intermittent issues.
42+
43+
![Deployment failure analysis example](/docs/administration/octopus-ai-assistant/deployment-failure-analyzer-example.png)
44+
45+
## Adding business logic using custom prompts
46+
47+
For organizations with specific internal processes and troubleshooting procedures, you can enhance the Deployment Failure Analyzer with custom business logic using [custom prompts](/docs/administration/octopus-ai-assistant/custom-prompts). Custom prompts are defined as variables in Library Variable Sets within Octopus Deploy, allowing you to embed organization-specific guidance and next steps directly into the failure analysis responses.
48+
49+
Custom prompts work by combining a user-facing prompt (`PageName[#].Prompt`) with an optional system prompt (`PageName[#].SystemPrompt`) that contains your business logic. The `.Prompt` variable defines what users see and interact with, while the `.SystemPrompt` variable provides behind-the-scenes instructions that guide the AI's analysis without being visible to users.
50+
51+
### Example: Missing Azure resource group
52+
53+
Here's an example of how to configure custom business logic for deployment failures where an Azure resource group cannot be found. While this is a basic example, it shows how you can embed custom business logic for known issues, instruct the LLM on how to respond, and whether to also include general troubleshooting steps from the LLM or not.
54+
55+
Custom logic is defined by configuring variables in a variable set named `OctoAI Prompts` in Octopus Deploy.
56+
57+
| Variable name | Variable value |
58+
|-----------|-------------|
59+
| Project.Deployment[0].Prompt | Why did the deployment fail? If the deployment didn't fail, say so. Provide suggestions for resolving the issue. |
60+
| Project.Deployment[0].SystemPrompt | If the logs indicate that an Azure resource group could not be located, find the team responsible for the project in the project descriptions and return the instruction to create a support ticket using the Slack workflow in the team slack channel. Don't provide general troubleshooting steps in the response.|
61+
62+
In this example, when the analyzer detects an issue related to a missing Azure Resource Group in the deployment logs, it will:
63+
64+
1. Look up the responsible team from the project description
65+
2. Provide specific instructions to create a support ticket via the team's Slack workflow
66+
3. Direct users to the appropriate team channel rather than providing generic troubleshooting steps
67+
68+
![Deployment failure analysis example with custom prompt](/docs/administration/octopus-ai-assistant/deployment-failure-analyzer-custom-prompt-example.png)
69+
70+
This approach ensures users get immediate, actionable guidance that follows your organization's established support processes, reducing resolution time and ensuring consistency across teams.
71+
72+
For detailed instructions on setting up custom prompts, including variable naming conventions and supported pages, see the [custom prompts documentation](/docs/administration/octopus-ai-assistant/custom-prompts).

0 commit comments

Comments
 (0)