# Feature ITSM-32 - Inbound email processing

Project: Stream IT service management (ITSM)
Status: in development
URL: https://community.openproject.org/work_packages/ITSM-32

## Planning

| Attribute | Value |
| --- | --- |
| Date |  |
| Priority | Medium |
| Target versions | 18.0.0 |
| Votes |  |
| List |  |
| Roadmap |  |
| Relevance |  |

## OPEN POINTS / FEEDBACK

_No work packages_

## Features and Chunks

| ID | Subject | Type | Status | Assignee | Target versions | Spent time |
| --- | --- | --- | --- | --- | --- | --- |
| ITSM-42 | Fetching email from mailboxes | Implementation | needs review | Yauheni Suhakou |  |  |
| ITSM-43 | Creating work packages from new emails | Implementation | needs review | Yauheni Suhakou |  |  |
| ITSM-44 | Replies to existing work packages | Implementation | new |  |  |  |
| ITSM-51 | Attachments of incoming emails | Implementation | new |  |  |  |

## Bugs and other work packages

_No work packages_

## Technical Details

| Attribute | Value |
| --- | --- |
| Module |  |
| Feature Flag |  |
| Requires doc change | No |
| Enterprise plan |  |

## Details

| Attribute | Value |
| --- | --- |
| Pull preview |  |
| Mockups |  |
| Build Number |  |
| Found in mobile app version |  |
| Follow-up feedback e-mail |  |
| Accessibility | No |

## People

| Attribute | Value |
| --- | --- |
| Assignee | Yauheni Suhakou |
| Accountable |  |
| Product manager |  |
| Developers |  |
| Designer |  |
| Ordered by |  |
| Signed off by |  |
| Interest by |  |

## Estimates and progress

| Attribute | Value |
| --- | --- |
| PD (min) |  |
| PD (max) |  |
| Work |  |
| Labor costs |  |

## Description

### Inbound email processing

> This is the processing workflow for this iteration. We will add improvements in further iterations.

Email processing should run every 15 minutes.

1.  Before ingesting emails, on every run this process should make sure that the helpdesk user is added to the project (see <mention class="mention" data-id="63585" data-type="work_package" data-text="##SISMI-22" data-display-id="SISMI-22">\#\#SISMI-22</mention>). If the email ingestion fails due to an authentication failure, it should apply an exponential back-off.

2.  The ingest process should scan the IMAP `INBOX` for unread emails (without `\Seen` flag). If an email fails processing, it should mark it as `\Seen`, so the mailbox admin can later deal with it manually.

3.  The process should then scan the existing work packages in the current project using the `Message-ID` header from the email to identify emails that are duplicates.

4.  The process should also filter out emails that are auto-generated, which can be identified by the `Auto-Submitted`, `X-Autoreply`, `X-Autorespond`, `List-Unsubscribe` headers (unless they contain the value `no`). If the email contains the headers `Precedence` or `X-Precedence` with the content of `auto\_reply`, `bulk`, or `junk`, they should also be treated as auto-generated. Auto-generated emails should be deleted from the INBOX without processing. This prevents two ticketing systems from emailing each other. *Since it also drops bounce messages, we need to resolve this in the next iteration.*

5.  The process should then look at the `Subject`.
    
      - If it contains a work package ID in the form of `[WORKPACKAGE-ID]` **and** the specified work package is in the current project **or** has originally been in the current work package, the email should be marked as a follow-up.
    
      - If the subject yields no result, the ingestion should inspect the `References` header to figure out if an email is a reply to an email OpenProject sent out. (The `Message-ID` headers of outbound emails should be stored in OpenProject.) If a work package is found, the email should be treated as a follow-up.
    
      - If none of these yield a result, the email should be treated as new *(next step)*.

6.  If the email is new, a new work package should be created. The title of the work package comes from the email subject, the description from the email text. For follow-ups, the email should be added as a comment *(including the sender email)*. In both cases the text should be extracted from HTML or multipart emails and escaped for inclusion in OpenProject. If possible, the process should also filter out any quoted text (`<blockquote>` or lines prefixed with `>`).

7.  Attachments (`Content-Disposition: attachment`) should be attached to the work package as files, deduplicating them, and linking to them in the respective description or comment.

8.  The ingest process should investigate the `From`, `To`, `CC`, `Delivered-To`, `X-Original-To` headers to figure out what email addresses should be added to the work package as recipients. These should be added to the work package to a newly-created `Recipients` field.

### Acceptance Criteria

  - The email processing incorporates all the described steps.

  - There is a `Recipients` field in each work package created by email that is used as a list of recipients for outbound communication.

  - Emails cannot produce formatted text or tag people in the OpenProject instance *(interpret only plaintext until a further iteration for security reasons)*.

### Testing

  - We need to ensure that an email work package or comment creation loop cannot happen via auto-generated replies.

  - Test functionality with at least 3 commonly used email clients.