Content
View differences
Updated by Christophe Bliard over 1 year ago
Our spec is flaky, specifically "Watcher tab within a **full screen** behaves like watchers tab when auto completing users with view\_user\_email permissions shows the email address of all users" in `spec/features/work_packages/tabs/watcher_tab_spec.rb`
Note that the full screen could also refer to a **split screen** or a **primerized split screen**. The auto completing specs are executed in all three shared examples.
See here for a failing CI run: [https://github.com/opf/openproject/actions/runs/11591676237/job/32273782259](https://github.com/opf/openproject/actions/runs/11591676237/job/32273782259) The error message is:
```text
1) Watcher tab within a full screen behaves like watchers tab when auto completing users with view_user_email permissions shows the email address of all users
Failure/Error: expect(target_dropdown).to have_css(".ng-option", text: other_user.name)
expected to find visible css ".ng-option" with text "Other Bobbit" within #<Capybara::Node::Element tag="ng-dropdown-panel" path="//HTML[1]/BODY[1]/NG-DROPDOWN-PANEL[1]"> but there were no matches. Also found "BB\n Bob Bobbit bobmail47.bobbit@bob.com", which matched the selector but not all filters.
Shared Example Group: "watchers tab" called from ./spec/features/work_packages/tabs/watcher_tab_spec.rb:172
# ./spec/features/work_packages/tabs/watcher_tab_spec.rb:148:in `block (5 levels) in <top (required)>'
# ./spec/support/capybara.rb:16:in `block (2 levels) in <top (required)>'
# /usr/local/bundle/gems/webmock-3.24.0/lib/webmock/rspec.rb:39:in `block (2 levels) in <top (required)>'
# ./spec/support/shared/with_env.rb:58:in `block (2 levels) in <top (required)>'
# ./spec/support/shared/with_direct_uploads.rb:204:in `block (2 levels) in <top (required)>'
# /usr/local/bundle/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:124:in `block in run'
# /usr/local/bundle/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:110:in `run'
# /usr/local/bundle/gems/rspec-retry-0.6.2/lib/rspec_ext/rspec_ext.rb:12:in `run_with_retry'
# ./spec/support/rspec_retry.rb:24:in `block (2 levels) in <top (required)>'
# ./spec/support/cuprite_setup.rb:126:in `block (2 levels) in <top (required)>'
# /usr/local/bundle/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:124:in `block in run'
# /usr/local/bundle/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:110:in `run'
# /usr/local/bundle/gems/rspec-retry-0.6.2/lib/rspec_ext/rspec_ext.rb:12:in `run_with_retry'
# /usr/local/bundle/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:37:in `block (2 levels) in setup'
```
What is expected by the spec is the following:
* when using the user auto completer to add watchers, you should see all selectable users.
* if you have the `view_user_email` permission (part of the standard global role), you can see the email address next to the user name in your search results
* if you don't have the permission, you will only see your own email address, but not the one of other users.
To test this, **two** **users** are added to the project and the expected result is checked when opening the auto completer.
For some reason, the second user is not visible at all in some cases. Since the user name is always expected (and the email address is expected when having the necessary rights), the spec fails.
<br>
It's not possible to reproduce a failing spec locally, it does only seem to happen on the CI. I suspect that the permissions of the individual users to the project are not loaded correctly in some cases due to side effects.
Any attempt of fixing this has been in vain. See [this PR](https://github.com/opf/openproject/pull/17063) for an ongoing attempt to fix it. Feel free to start your own investigation.
### Update 1
I managed to reproduce the flakiness locally. Here is how to reproduce it:
* Use headful mode. For some reason, the issue is not triggered locally in headless mode.
* in `spec/features/work_packages/tabs/watcher_tab_spec.rb`, in `context "when auto completing users"`, in the `before` block, add a `sleep 2` instruction at the beginning of the block
* run the example "with view\_user\_email permissions" `OPENPROJECT_TESTING_NO_HEADLESS=1 bundle exec rspec -e "with view_user_email permissions" ./spec/features/work_packages/tabs/watcher_tab_spec.rb`
* \=> 3 examples will run. 2 will fail: the fullscreen one and the primerized split screen one. The split screen one will pass.
**Analysis**
The flakiness comes from the watcher page being visited before the second user is created. The page is visited in the `before` block of the `"watchers tab"` context, and the `other_user` is created in the second block, in `"when auto completing users"` context.
When the page is visited, the autocomplete user dropdown selector is automatically focused. The autocompleter being focused triggers the request to the `/api/v3/work_packages/56/available_watchers` which returns the users. Depending on the timing, `other_user` may not be created yet, triggering the issue. That's why adding a `sleep 2` before the `other_user` creation forces the error to happen.
For some reason, the autofocus only happens in full screen and primerized split screen.
And what I do not understand is that locally it happens only in headful mode, never in headless. Yet on CI it happens on headless mode as well. Maybe I'm running another version of chrome which behaves differently (I'm on Mac (Darwin Mac.lan 24.0.0 Darwin Kernel Version 24.0.0: Tue Sep 24 23:35:10 PDT 2024; root:xnu-11215.1.12~1/RELEASE\_ARM64\_T6031 arm64), using "Google Chrome for Testing 130.0.6723.91"). I tried using [the new headless chrome (`--headless=new`)](https://stackoverflow.com/questions/45631715/downloading-with-chrome-headless-and-selenium/73840130#73840130) as I read that the legacy headless mode uses an alternate rendering engine but it does not change anything. If someone knows why, I would be highly interested
To fix the issue, either:
* Visit the page after creating the second user
* Reload the page after creating the second user with `page.refresh` (longer test time, ugly but works)
* Remove the autofocus on the user autocompleter (probably not a good idea, but would fix the issue)
## Acceptance Criteria
* Enable the spec again
* Fix it :)
Note that the full screen could also refer to a **split screen** or a **primerized split screen**. The auto completing specs are executed in all three shared examples.
See here for a failing CI run: [https://github.com/opf/openproject/actions/runs/11591676237/job/32273782259](https://github.com/opf/openproject/actions/runs/11591676237/job/32273782259) The error message is:
```text
1) Watcher tab within a full screen behaves like watchers tab when auto completing users with view_user_email permissions shows the email address of all users
Failure/Error: expect(target_dropdown).to have_css(".ng-option", text: other_user.name)
expected to find visible css ".ng-option" with text "Other Bobbit" within #<Capybara::Node::Element tag="ng-dropdown-panel" path="//HTML[1]/BODY[1]/NG-DROPDOWN-PANEL[1]"> but there were no matches. Also found "BB\n Bob Bobbit bobmail47.bobbit@bob.com", which matched the selector but not all filters.
Shared Example Group: "watchers tab" called from ./spec/features/work_packages/tabs/watcher_tab_spec.rb:172
# ./spec/features/work_packages/tabs/watcher_tab_spec.rb:148:in `block (5 levels) in <top (required)>'
# ./spec/support/capybara.rb:16:in `block (2 levels) in <top (required)>'
# /usr/local/bundle/gems/webmock-3.24.0/lib/webmock/rspec.rb:39:in `block (2 levels) in <top (required)>'
# ./spec/support/shared/with_env.rb:58:in `block (2 levels) in <top (required)>'
# ./spec/support/shared/with_direct_uploads.rb:204:in `block (2 levels) in <top (required)>'
# /usr/local/bundle/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:124:in `block in run'
# /usr/local/bundle/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:110:in `run'
# /usr/local/bundle/gems/rspec-retry-0.6.2/lib/rspec_ext/rspec_ext.rb:12:in `run_with_retry'
# ./spec/support/rspec_retry.rb:24:in `block (2 levels) in <top (required)>'
# ./spec/support/cuprite_setup.rb:126:in `block (2 levels) in <top (required)>'
# /usr/local/bundle/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:124:in `block in run'
# /usr/local/bundle/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:110:in `run'
# /usr/local/bundle/gems/rspec-retry-0.6.2/lib/rspec_ext/rspec_ext.rb:12:in `run_with_retry'
# /usr/local/bundle/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:37:in `block (2 levels) in setup'
```
What is expected by the spec is the following:
* when using the user auto completer to add watchers, you should see all selectable users.
* if you have the `view_user_email` permission (part of the standard global role), you can see the email address next to the user name in your search results
* if you don't have the permission, you will only see your own email address, but not the one of other users.
To test this, **two** **users** are added to the project and the expected result is checked when opening the auto completer.
For some reason, the second user is not visible at all in some cases. Since the user name is always expected (and the email address is expected when having the necessary rights), the spec fails.
<br>
It's not possible to reproduce a failing spec locally, it does only seem to happen on the CI. I suspect that the permissions of the individual users to the project are not loaded correctly in some cases due to side effects.
Any attempt of fixing this has been in vain. See [this PR](https://github.com/opf/openproject/pull/17063) for an ongoing attempt to fix it. Feel free to start your own investigation.
### Update 1
I managed to reproduce the flakiness locally. Here is how to reproduce it:
* Use headful mode. For some reason, the issue is not triggered locally in headless mode.
* in `spec/features/work_packages/tabs/watcher_tab_spec.rb`, in `context "when auto completing users"`, in the `before` block, add a `sleep 2` instruction at the beginning of the block
* run the example "with view\_user\_email permissions" `OPENPROJECT_TESTING_NO_HEADLESS=1 bundle exec rspec -e "with view_user_email permissions" ./spec/features/work_packages/tabs/watcher_tab_spec.rb`
* \=> 3 examples will run. 2 will fail: the fullscreen one and the primerized split screen one. The split screen one will pass.
**Analysis**
The flakiness comes from the watcher page being visited before the second user is created. The page is visited in the `before` block of the `"watchers tab"` context, and the `other_user` is created in the second block, in `"when auto completing users"` context.
When the page is visited, the autocomplete user dropdown selector is automatically focused. The autocompleter being focused triggers the request to the `/api/v3/work_packages/56/available_watchers` which returns the users. Depending on the timing, `other_user` may not be created yet, triggering the issue. That's why adding a `sleep 2` before the `other_user` creation forces the error to happen.
For some reason, the autofocus only happens in full screen and primerized split screen.
And what I do not understand is that locally it happens only in headful mode, never in headless. Yet on CI it happens on headless mode as well. Maybe I'm running another version of chrome which behaves differently (I'm on Mac (Darwin Mac.lan 24.0.0 Darwin Kernel Version 24.0.0: Tue Sep 24 23:35:10 PDT 2024; root:xnu-11215.1.12~1/RELEASE\_ARM64\_T6031 arm64), using "Google Chrome for Testing 130.0.6723.91"). I tried using [the new headless chrome (`--headless=new`)](https://stackoverflow.com/questions/45631715/downloading-with-chrome-headless-and-selenium/73840130#73840130) as I read that the legacy headless mode uses an alternate rendering engine but it does not change anything. If someone knows why, I would be highly interested
To fix the issue, either:
* Visit the page after creating the second user
* Reload the page after creating the second user with `page.refresh` (longer test time, ugly but works)
* Remove the autofocus on the user autocompleter (probably not a good idea, but would fix the issue)
## Acceptance Criteria
* Enable the spec again
* Fix it :)