Top Menu

Jump to content
Home
    Modules
      • Projects
      • Activity
      • Work packages
      • Gantt charts
      • Calendars
      • Team planners
      • Boards
      • News
    • Getting started
    • Introduction video
      Welcome to OpenProject Community
      Get a quick overview of project management and team collaboration with OpenProject. You can restart this video from the help menu.

    • Help and support
    • Upgrade to Enterprise edition
    • User guides
    • Videos
    • Shortcuts
    • Community forum
    • Enterprise support

    • Additional resources
    • Data privacy and security policy
    • Digital accessibility (DE)
    • OpenProject website
    • Security alerts / Newsletter
    • OpenProject blog
    • Release notes
    • Report a bug
    • Development roadmap
    • Add and edit translations
    • API documentation
  • Sign in
      Forgot your password?

      or sign in with your existing account

      Google

Side Menu

  • Overview
  • Activity
    Activity
  • Roadmap
  • Work packages
    Work packages
  • Gantt charts
    Gantt charts
  • Calendars
    Calendars
  • Team planners
    Team planners
  • Boards
    Boards
  • News
  • Forums

Content

Support Installation & Updates
  1. OpenProject
  2. Forums
  3. Support Installation & Updates
  4. Using S3 for storing attachments

Using S3 for storing attachments

Added by Alvaro Navas Peire about 9 years ago

Hi everyone,

I’m trying to set up an Openproject server for evaluation purposes and I’d like to be able to store all files and attachments in S3 in order to avoid filling up my VPS’ limited storage space, but I haven’t figured out yet how to do so.

I’m using the Docker installation method as explained in https://www.openproject.org/open-source/download/docker/ . Under Configuration it states that configuration must be done through environment variables. According to https://github.com/opf/openproject/blob/dev/doc/CONFIGURATION.md , if I understood correctly, I should change the variables under Attachments Storage to environment variables like so:

ATTACHMENTS_STORAGE=fog FOG_DIRECTORY=’my-bucket’ FOG_CREDENTIALS_PROVIDER=’AWS’ FOG_CREDENTIALS_AWS__ACCESS__KEY__ID='myKeyID' FOG_CREDENTIALS_AWS__SECRET__ACCESS__KEY__ID='mySecretKeyID' FOG_CREDENTIALS_REGION='region.goes.here'

but I can’t make it work. I’m trying different names for the variables but nothing yet. Am I understanding things correctly? What am I doing wrong?

Please excuse me if this is a stupid question; I’m new at Docker and OpenProject.

Thank you in advance for any help!


Replies (16)

RE: Using S3 for storing attachments - Added by Ethan Piliavin over 8 years ago

Did you ever have any success with this? I am having the same problem.

RE: Using S3 for storing attachments - Added by Oliver Günther over 8 years ago

If you’re overriding configuration values through ENV variables (apart from Rails SMTP settings), OpenProject requires you to prefix them with OPENPROJECT_

OPENPROJECT_ATTACHMENTS__STORAGE=fog
OPENPROJECT_FOG_CREDENTIALS_AWS__ACCESS__KEY__ID=<access key>
OPENPROJECT_FOG_CREDENTIALS_AWS__SECRET__ACCESS__KEY=<secret access key>
OPENPROJECT_FOG_CREDENTIALS_PROVIDER=AWS
OPENPROJECT_FOG_CREDENTIALS_REGION=<region e.g., eu-west-1>
OPENPROJECT_FOG_DIRECTORY=<directory name for storage>

I’ve added that to the config.rb documentation.

Best,
Oliver

RE: Using S3 for storing attachments - Added by Ethan Piliavin over 8 years ago

Thank you Oliver! That did the trick. I was confused because the other entries weren’t prefixed with OPENPROJECT_ and didn’t realize I needed it.

Anyway. for anyone else that is stuck with this, here it is spelled out for you:

My install is via RPM on Centos 7 (Linode).

Edit: /etc/openproject/conf.d/other

Comment out this line:

# export ATTACHMENTS_STORAGE_PATH="/var/db/openproject/files"

Then I added the following lines right after it:

export OPENPROJECT_ATTACHMENTS__STORAGE="fog"
export OPENPROJECT_FOG_DIRECTORY="----"
export OPENPROJECT_FOG_CREDENTIALS_PROVIDER="AWS"
export OPENPROJECT_FOG_CREDENTIALS_AWS__ACCESS__KEY__ID="----"
export OPENPROJECT_FOG_CREDENTIALS_AWS__SECRET__ACCESS__KEY="----"
export FOG_CREDENTIALS_REGION="us-east-1"

Then from the commandline

service openproject restart

RE: Using S3 for storing attachments - Added by Alvaro Navas Peire over 8 years ago

Thank you for your replies and the solution; we eventually switched to another solution, but this will come in handy for future reference.

RE: RE: Using S3 for storing attachments - Added by Jananath Banuka about 4 years ago

Hi @ethan can you please tell me how you did this? And can you tell me what is your s3 bucket policy you have set?

RE: Using S3 for storing attachments - Added by Dev User over 3 years ago

Hi all (@ethan @oliver), i'm also facing a similar issue when trying to configure an external storage (DigitalOcean Spaces), in my docker-compose based setup.

This is what the relevant part of my docker-compose.yml file looks like:

x-op-restart-policy: &restart_policy
  restart: unless-stopped
x-op-image: &image
  image: openproject/community:${TAG:-11}
x-op-app: &app
  <<: *image
  <<: *restart_policy
  environment:
    RAILS_CACHE_STORE: "memcache"
    OPENPROJECT_CACHE__MEMCACHE__SERVER: "cache:11211"
    OPENPROJECT_RAILS__RELATIVE__URL__ROOT: "${OPENPROJECT_RAILS__RELATIVE__URL__ROOT:-}"
    DATABASE_URL: "${DATABASE_URL:-postgres://postgres:pass@db/openproject?pool=20&encoding=unicode&reconnect=true}"
    RAILS_MIN_THREADS: 4
    RAILS_MAX_THREADS: 16
    USE_PUMA: "true"
    # set to true to enable the email receiving feature. See ./docker/cron for more options
    IMAP_ENABLED: "${IMAP_ENABLED:-false}"
    OPENPROJECT_ATTACHMENTS__STORAGE: fog
    OPENPROJECT_FOG_CREDENTIALS_PROVIDER: DIGITALOCEAN
    OPENPROJECT_FOG_CREDENTIALS_DIGITALOCEAN__ACCESS__KEY__ID: "key-id"
    OPENPROJECT_FOG_CREDENTIALS_DIGITALOCEAN__SECRET__ACCESS__KEY: "access-key"
    OPENPROJECT_FOG_CREDENTIALS_REGION: "region"
    OPENPROJECT_FOG_DIRECTORY: openproject

Kindly advise, how to resolve this.

RE: Using S3 for storing attachments - Added by Oliver Günther over 3 years ago

You need to specify the provider to AWS as that is what fog expects for S3 (or compatible APIs).

Please see https://github.com/fog/fog-digitalocean/issues/20#issuecomment-332009742 as an example for digitalocean.

Note that we officially support only the AWS flow of fog, so I'm not sure whether digitalocean is fully compatible.

Best

Oliver

RE: Using S3 for storing attachments - Added by Dev User over 3 years ago

@oliver Thanks for the link.

I tried it like this, in my docker-compose file:

x-op-restart-policy: &restart_policy
  restart: unless-stopped
x-op-image: &image
  image: openproject/community:${TAG:-11}
x-op-app: &app
  <<: *image
  <<: *restart_policy
  environment:
    RAILS_CACHE_STORE: "memcache"
    OPENPROJECT_CACHE__MEMCACHE__SERVER: "cache:11211"
    OPENPROJECT_RAILS__RELATIVE__URL__ROOT: "${OPENPROJECT_RAILS__RELATIVE__URL__ROOT:-}"
    DATABASE_URL: "${DATABASE_URL:-postgres://postgres:p4ssw0rd@db/openproject?pool=20&encoding=unicode&reconnect=true}"
    RAILS_MIN_THREADS: 4
    RAILS_MAX_THREADS: 16
    USE_PUMA: "true"
    # set to true to enable the email receiving feature. See ./docker/cron for more options
    IMAP_ENABLED: "${IMAP_ENABLED:-false}"
    OPENPROJECT_ATTACHMENTS__STORAGE: fog
    OPENPROJECT_FOG_CREDENTIALS_PROVIDER: AWS
    OPENPROJECT_FOG_CREDENTIALS_AWS__ACCESS__KEY__ID: "key-id"
    OPENPROJECT_FOG_CREDENTIALS_AWS__SECRET__ACCESS__KEY: "access-key"
    OPENPROJECT_FOG_CREDENTIALS_REGION: "region"
    OPENPROJECT_FOG_CREDENTIALS_HOST: "region.digitaloceanspaces.com"
    OPENPROJECT_FOG_CREDENTIALS_ENDPOINT: "https://region.digitaloceanspaces.com"
    OPENPROJECT_FOG_DIRECTORY: "directory"
    OPENPROJECT_FOG_ATTRIBUTES_CACHE__CONTROL: "PUBLIC"

This is still not working.

Error in browser console:

Refused to connect to 'https://region.digitaloceanspaces.com/directory/' because it violates the following Content Security Policy directive: "connect-src 'self' directory.s3.amazonaws.com directory.s3.region.amazonaws.com

RE: RE: Using S3 for storing attachments - Added by Dev User over 3 years ago

Any idea how to resolve this issue?

RE: Using S3 for storing attachments - Added by Oliver Günther over 3 years ago

This is due to our content-security policy restricting available hosts to contact. It doesn't look like you can immediately fix this on your own, however I added a pull request for the next release to allow custom hosts to be set. You could apply this patch and build the OpenProject container manually with it applied:

https://github.com/opf/openproject/pull/9668

Best

Oliver

RE: Using S3 for storing attachments - Added by Dev User over 3 years ago

@oliver Thank you for your prompt assistance!

Would this PR also allow me to use DigitalOcean Spaces easily?

Also, any idea regarding the timeline for the release of v11.4?

RE: Using S3 for storing attachments - Added by Oliver Günther over 3 years ago

Hi Dev,

as they claim to be S3 compatible, this should work and we'd be interested to support it. However please note we don't use DO ourselves so we have no option to test it.

The release of 11.4 is specified here and it's release date is scheduled for the 22nd of september.

Best

Oliver

RE: Using S3 for storing attachments - Added by Dev User over 3 years ago

Hi Oliver,

It does work in many other apps that i use.

I tried these environment settings in the docker-compose.yml

    OPENPROJECT_FOG_CREDENTIALS_AWS__ACCESS__KEY__ID: "key_id"
    OPENPROJECT_FOG_CREDENTIALS_AWS__SECRET__ACCESS__KEY: "access_key"
    OPENPROJECT_FOG_CREDENTIALS_REGION: "region"
    OPENPROJECT_FOG_CREDENTIALS_HOST: "region.digitaloceanspaces.com"
    OPENPROJECT_FOG_CREDENTIALS_ENDPOINT: "https://region.digitaloceanspaces.com"
    OPENPROJECT_FOG_DIRECTORY: "openproject"
    OPENPROJECT_FOG_ATTRIBUTES_CACHE__CONTROL: "PUBLIC"

Still receiving this error:

Refused to connect to 'https://region.digitaloceanspaces.com/openproject/' because it violates the following Content Security Policy directive: "connect-src 'self' openproject.s3.amazonaws.com openproject.s3.region.amazonaws.com

RE: Using S3 for storing attachments - Added by Oliver Günther over 3 years ago

The issue appears to be related to direct uploads (uploading files directly to S3 instead of OpenProject) which probably isn't supported by other hosts.

Please set OPENPROJECT_DIRECT__UPLOADS=false and try again.

Meanwhile, https://github.com/opf/openproject/pull/9705 is going to disable this feature for non-s3 hosts

Best

Oliver

RE: Using S3 for storing attachments - Added by Dev User over 3 years ago

Thanks for this info. I can finally upload files.

I am still facing another issue though: I can't upload any document greater than size 1 MB.

And, i couldn't find any Environment Variable/Setting to change this either.

My current setting is:

So, it doesn't seem to make sense why i'm seeing this error:

RE: Using S3 for storing attachments - Added by Oliver Günther over 3 years ago

You likely have a proxying server that restricts available upload size. The setting that you show is only for OpenProject internal validations and does not control any web server limits.

For nginx: https://www.cyberciti.biz/faq/linux-unix-bsd-nginx-413-request-entity-too-large/

For Apache https://www.cyberciti.biz/faq/increase-file-upload-size-limit-in-php-apache-app/

Best

Oliver

  • (1 - 16/16)
Loading...