Content
View differences
Updated by Jonas Jabari 26 days ago
<img class="op-uc-image op-uc-image_inline" src="/api/v3/attachments/923083/content">
As an OpenProject admin
I want to test an action's prompt against sample text directly in the admin UI
so that I can iterate on prompts before activating them for users.
## Acceptance criteria
* The action edit form contains a "Test" panel with:
* a textarea for sample input text
* for actions restricted to types (and for "Sort into template"): a work package type
<br>
selector, so the server can inject that type's template during the test
* a "Run test" button
* Running a test uses the **current form state** of the prompt (unsaved changes included), so
<br>
admins can iterate without saving.
* The result is displayed rendered as formatted text, with a toggle to view the raw markdown.
* Optionally (collapsible), the final assembled request is shown (system scaffold + action
<br>
prompt + injected template) — valuable for understanding why a prompt behaves as it does.
* Errors from the LLM connection (unreachable, timeout, invalid key) are shown inline in the
<br>
panel; they do not block saving the form.
* Nothing is persisted: no work package is touched, no log entry with content is stored.
* The panel is disabled with a hint when no LLM connection is configured or the global AI
<br>
toggle is off.
## Technical notes
* Reuses the server-side prompt assembly of the text transform API (same code path — the
<br>
sandbox is effectively the transform endpoint with a dry-run input) and the LLM client from
<br>
AI-3. Avoid a parallel implementation.
* Synchronous request with a strict timeout is acceptable here (admin-only, low volume),
<br>
regardless of the sync-vs-job decision for the end-user path.
## Permissions and visibility considerations
* Admins only (part of the admin form).
## Translation considerations
* UI strings translatable; sample text and results are not.
## Out of scope
* Automated prompt regression testing (openproject-llm-evals, see separate task)
* Test history / saved test cases
As an OpenProject admin
I want to test an action's prompt against sample text directly in the admin UI
so that I can iterate on prompts before activating them for users.
## Acceptance criteria
* The action edit form contains a "Test" panel with:
* a textarea for sample input text
* for actions restricted to types (and for "Sort into template"): a work package type
<br>
* a "Run test" button
* Running a test uses the **current form state** of the prompt (unsaved changes included), so
<br>
* The result is displayed rendered as formatted text, with a toggle to view the raw markdown.
* Optionally (collapsible), the final assembled request is shown (system scaffold + action
<br>
* Errors from the LLM connection (unreachable, timeout, invalid key) are shown inline in the
<br>
* Nothing is persisted: no work package is touched, no log entry with content is stored.
* The panel is disabled with a hint when no LLM connection is configured or the global AI
<br>
## Technical notes
* Reuses the server-side prompt assembly of the text transform API (same code path — the
<br>
<br>
* Synchronous request with a strict timeout is acceptable here (admin-only, low volume),
<br>
## Permissions and visibility considerations
* Admins only (part of the admin form).
## Translation considerations
* UI strings translatable; sample text and results are not.
## Out of scope
* Automated prompt regression testing (openproject-llm-evals, see separate task)
* Test history / saved test cases