Hi everyone!

For a while now, I have been busy developing an OpenProject plugin. For the most part this is going great. However, after trying for a long while now, I am still unable to run the full OpenProject test suite (even when just running straight from the OpenProject git without making any modifications, except the set-up of course).

I have tried many things like:

• In the most basic way I just followed the OpenProject development Setup on Debian/Ubuntu tutorial. Then I followed the Testing OpenProject tutorial to run the tests. I tried to just run specific groups of tests, the full suite. I also tried to do this in parallel.
• Because This didn't work I also tried to simulate what happens on Travis CI for OpenProject including the use of the same distribution, commands found in the CI scripts, environment variables, amount of CPUs, packages, installed locales, etc.
• I tried both Ubuntu 16.04 and 18.04
• I also tried many different versions and combinations of Ruby, Node, PostgreSQL, etc.
• Increasing time until timeout, number of retries, etc.
• Using a virtual machine as well as just running bare metal
• And many others

The problem is that whatever I seem to do, some specific test examples just keep failing (about 6 of them). An example is the ./spec/features/auth/omniauth_spec.rb spec of which two examples keep failing with the following output:

   1) Omniauth authentication error occurs with direct login and login required behaves like omniauth signin error should fail with generic error message
      Failure/Error: visit login_path
      OmniAuth::Error:
        invalid_credentials
      Shared Example Group: "omniauth signin error" called from ./spec/features/auth/omniauth_spec.rb:271
      # ./spec/features/auth/omniauth_spec.rb:251:in `block (4 levels) in <top (required)>'
      # ./spec/support/shared/with_mail.rb:15:in `block (2 levels) in <top (required)>'
   2) Omniauth authentication error occurs behaves like omniauth signin error should fail with generic error message
      Failure/Error: visit login_path
      OmniAuth::Error:
        invalid_credentials
      Shared Example Group: "omniauth signin error" called from ./spec/features/auth/omniauth_spec.rb:260
      # ./spec/features/auth/omniauth_spec.rb:251:in `block (4 levels) in <top (required)>'
      # ./spec/support/shared/with_mail.rb:15:in `block (2 levels) in <top (required)>'

However, when I just run rspec ./spec/features/auth/omniauth_spec.rb, there are no failures. The logs also provide no extra information on the failures. At this point I really can't think of anything else to try to get this working.

Therefore, I would like to ask if anyone has an idea what I could be missing here. Or if there's someone who's able to run the full suite of tests, an explanation of the steps taken to get this to work could really help me out (this doesn't have to be very extensive, just the steps taken outside of the documentation, if any, and the environment used, etc. would be enough).

Thanks in advance for any help provided!