Hi all, I am new to openproject and working on deploying to kubernetes (microk8s) using the helm chart. To start testing I have been deploying with this command:

microk8s helm upgrade --create-namespace --namespace openproject --install my-openproject openproject/openproject --set openproject.https=false --set ingress.host="op.192.168.1.227.nip.io" --set postgresql.auth.password="QWERTY123" --set postgres.auth.postgresPassword="qwerty123" --set persistence.storageClassName="microk8s-hostpath" --set ingress.tls.enabled=false

I am able to get most everything running, but I am not able to upload attachments or create budgets. In the logs after doing one of those actions I see:

2024-02-09 19:45:20 +0000 Rack app ("POST /projects/test-proj/budgets" - (192.168.1.224)): #<Errno::EROFS: Read-only file system @ rb_sysopen - /app/RackMultipart20240209-19-vte7rz>

I see a bug next to "Attachments directory writable" on Administration->Information, but I don't see where to set the directory as writable. I attached the information page.

Some environment variables from inside the container:

APP_PATH=/app
APP_DATA_PATH=/var/openproject/assets
OPENPROJECT_ATTACHMENTS__STORAGE__PATH=/var/openproject/assets/files

Some folder permissions from inside the container:

drwxr-xr-x 1 app app 4096 Jan 31 07:48 /app
drwxrwxrwx 2 root root 4096 Feb  9 19:41 /var/openproject/assets

notably /var/openproject/assets/files doesn't actually seem to exist

Did I miss an important step in my deployment to make the attachments directory writable?

I might have solved this.

There's an interaction (read: bug) in how Ruby handles temp directory creation and how Kubernetes handles ephemeral volumes. When the Pod is created, /tmp has permissions 777, which is world-writable without a sticky bit. That's sane in a container world, where only one user (or at least one security domain) will have access to that volume. Ruby, however, has a check that it won't use a world-writable path for tmpdir unless the sticky bit is set. That's sane in a non-container world.

To work around the issue, I added an extra init container and environment variable, in appropriate places in the Deployment:

spec:   
   containers:
   - command:
     - bash
     - /app/docker/prod/web
     env:
     - name: OPENPROJECT_DB_PASSWORD
       value: <REDACTED>
     - name: TMPDIR
       value: /tmp/openproject

   initContainers:
   - command:
     - bash
     - -c
     - mkdir /tmp/openproject
     image: docker.io/openproject/community:13-slim
     imagePullPolicy: Always
     name: mktmpdir
     resources: {}
     terminationMessagePath: /dev/termination-log
     terminationMessagePolicy: File
     volumeMounts:
     - mountPath: /tmp
       name: tmp

In my case, I was getting 500 errors when trying to create a Wiki page. I'm not too good with Ruby, so I haven't thoroughly traced the code, but I believe the root cause is that the Wiki pages (and perhaps file attachments) are written to a temp directory first, then moved to their final storage location. Since /tmp is world-writable and not sticky, and no better TMPDIR is set by default, Ruby crashes rather than permit something potentially insecure.

The extra init container simply creates a path /tmp/openproject, which is naturally not world-writable. That path is then used in the TMPDIR environment variable, allowing Ruby to confidently create temp directories.

Depending on the filesystem type used by your volumes, there could be word hurdle restrictions on write access.