Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More non-parallel test assumptions #6206

Open
joshmoore opened this issue Jan 24, 2020 · 0 comments
Open

More non-parallel test assumptions #6206

joshmoore opened this issue Jan 24, 2020 · 0 comments

Comments

@joshmoore
Copy link
Member

Following on from #6156, I tried running the integration tests as background jobs:

./build.py ... python &
./build.py ... java &
...
wait

but https://merge-ci.openmicroscopy.org/jenkins/job/OMERO-test-integration/287/#showFailuresLink failed with CLI assumptions:


Stacktrace

self = <test.integration.clitest.test_user.TestUser object at 0x7f52e4b75198>
capsys = <_pytest.capture.CaptureFixture object at 0x7f520acbbd68>
sort_key = None, group_format = None

    @pytest.mark.parametrize("sort_key", sort_keys)
    @pytest.mark.parametrize("group_format", [None, "count", "long"])
    def testList(self, capsys, sort_key, group_format):
        self.args += ["list"]
        if sort_key:
            self.args += ["--sort-by-%s" % sort_key]
        if group_format:
            self.args += ["--%s" % group_format]
        self.cli.invoke(self.args, strict=True)
    
        # Read from the stdout
        out, err = capsys.readouterr()
        ids = get_user_ids(out, sort_key=sort_key)
    
        # Check all users are listed
        if sort_key == 'login':
            sorted_list = sorted(self.users, key=lambda x: x.omeName.val)
        elif sort_key == 'first-name':
            sorted_list = sorted(self.users, key=lambda x: x.firstName.val)
        elif sort_key == 'last-name':
            sorted_list = sorted(self.users, key=lambda x: x.lastName.val)
        elif sort_key == 'email':
            sorted_list = sorted(self.users, key=lambda x: (
                x.email and x.email.val or ""))
        else:
            sorted_list = sorted(self.users, key=lambda x: x.id.val)
>       assert ids == [user.id.val for user in sorted_list]
E       assert [0, 1, 2, 3, 4, 5, ...] == [0, 1, 2, 3, 4, 5, ...]
E         Left contains 2 more items, first extra item: 3127
E         Full diff:
joshmoore added a commit to joshmoore/openmicroscopy that referenced this issue Jan 27, 2020
`omero user list` was compared to the output of iadmin.lookupExperimenters().
Depending on the interval, other users could have been added. Now we assume
monotonically increasing users.
snoopycrimecop added a commit to snoopycrimecop/openmicroscopy that referenced this issue Jan 27, 2020
snoopycrimecop added a commit to snoopycrimecop/openmicroscopy that referenced this issue Jan 27, 2020
Repository: openmicroscopy/openmicroscopy
Excluded PRs:
  - PR 6099 will-moore 'Use chosen plugin for script UI if large number of options' (label: exclude)
  - PR 5912 manics 'OMERO.web default log can be changed to stdout' (label: exclude)
  - PR 6096 will-moore 'Script gui file upload' (label: exclude)
  - PR 6094 manics 'omero.web.override_root_app to set custom web root app' (label: exclude)
  - PR 5946 manics 'Python gateway secure' (label: exclude)
  - PR 6081 will-moore 'Webclient performance fixes' (label: exclude)
  - PR 5966 joshmoore 'Allow testing multiple builds in travis' (status: failure)
  - PR 5884 manics 'Ignore owner when checking omero.data.dir is accessible' (label: exclude)
  - PR 5910 manics 'Allow cli_login to be used without closing session' (label: exclude)
  - PR 6086 manics 'Alternative JSON configuration system for OMERO.web' (label: exclude)
  - PR 5914 kennethgillen 'Adding --as-default to CLI omero user joingroup' (label: exclude)
  - PR 6093 manics 'admin diagnostics: show BF (and optionally other) jar versions' (label: exclude)
  - PR 6101 dominikl 'Set namespace of fileannoation of omero.table - test' (status: failure)
  - PR 6205 chris-allan 'Test cases for expanded user session creation abilities' (status: failure)
  - PR 6083 manics 'web: omero.web.http_response_headers to set custom headers' (label: exclude)
Already up-to-date.

Merged PRs:
  - PR 6060 rgozim 'Added gradle support to omerojava component'
  - PR 6071 joshmoore 'Tests for 5.5.2 PRs'
  - PR 6176 sbesson 'Restore CLI DB integration tests'
  - PR 6207 jburel 'Review training example setup'
  - PR 6208 joshmoore 'test_user: don't assume single-threaded (see ome#6206)'

Generated by OMERO-push#224 (https://idr-ci.openmicroscopy.org/jenkins/job/OMERO-push/224/)
snoopycrimecop added a commit to snoopycrimecop/openmicroscopy that referenced this issue Jan 27, 2020
Repository: openmicroscopy/openmicroscopy
Excluded PRs:
  - PR 6081 will-moore 'Webclient performance fixes' (user: will-moore)
  - PR 5966 joshmoore 'Allow testing multiple builds in travis' (user: joshmoore)
  - PR 6205 chris-allan 'Test cases for expanded user session creation abilities' (user: chris-allan)
  - PR 5914 kennethgillen 'Adding --as-default to CLI omero user joingroup' (user: kennethgillen)
  - PR 6071 joshmoore 'Tests for 5.5.2 PRs' (user: joshmoore)
  - PR 5910 manics 'Allow cli_login to be used without closing session' (user: manics)
  - PR 6096 will-moore 'Script gui file upload' (user: will-moore)
  - PR 6093 manics 'admin diagnostics: show BF (and optionally other) jar versions' (user: manics)
  - PR 6086 manics 'Alternative JSON configuration system for OMERO.web' (user: manics)
  - PR 5884 manics 'Ignore owner when checking omero.data.dir is accessible' (user: manics)
  - PR 6207 jburel 'Review training example setup' (user: jburel)
  - PR 6208 joshmoore 'test_user: don't assume single-threaded (see ome#6206)' (user: joshmoore)
  - PR 5946 manics 'Python gateway secure' (user: manics)
  - PR 6083 manics 'web: omero.web.http_response_headers to set custom headers' (user: manics)
  - PR 5912 manics 'OMERO.web default log can be changed to stdout' (user: manics)
  - PR 6094 manics 'omero.web.override_root_app to set custom web root app' (user: manics)
  - PR 6099 will-moore 'Use chosen plugin for script UI if large number of options' (user: will-moore)
  - PR 6060 rgozim 'Added gradle support to omerojava component' (user: rgozim)
  - PR 6101 dominikl 'Set namespace of fileannoation of omero.table - test' (user: dominikl)
  - PR 6176 sbesson 'Restore CLI DB integration tests' (user: sbesson)
Already up-to-date.

Generated by OMERO-push#277 (https://latest-ci.openmicroscopy.org/jenkins/job/OMERO-push/277/)
snoopycrimecop added a commit to snoopycrimecop/openmicroscopy that referenced this issue Jan 28, 2020
snoopycrimecop added a commit to snoopycrimecop/openmicroscopy that referenced this issue Jan 28, 2020
Repository: openmicroscopy/openmicroscopy
Excluded PRs:
  - PR 6205 chris-allan 'Test cases for expanded user session creation abilities' (status: failure)
  - PR 6101 dominikl 'Set namespace of fileannoation of omero.table - test' (status: failure)
  - PR 6099 will-moore 'Use chosen plugin for script UI if large number of options' (label: exclude)
  - PR 6096 will-moore 'Script gui file upload' (label: exclude)
  - PR 6094 manics 'omero.web.override_root_app to set custom web root app' (label: exclude)
  - PR 6093 manics 'admin diagnostics: show BF (and optionally other) jar versions' (label: exclude)
  - PR 6086 manics 'Alternative JSON configuration system for OMERO.web' (label: exclude)
  - PR 6083 manics 'web: omero.web.http_response_headers to set custom headers' (label: exclude)
  - PR 6081 will-moore 'Webclient performance fixes' (label: exclude)
  - PR 5966 joshmoore 'Allow testing multiple builds in travis' (status: failure)
  - PR 5946 manics 'Python gateway secure' (label: exclude)
  - PR 5914 kennethgillen 'Adding --as-default to CLI omero user joingroup' (label: exclude)
  - PR 5912 manics 'OMERO.web default log can be changed to stdout' (label: exclude)
  - PR 5910 manics 'Allow cli_login to be used without closing session' (label: exclude)
  - PR 5884 manics 'Ignore owner when checking omero.data.dir is accessible' (label: exclude)
Already up-to-date.

Merged PRs:
  - PR 6060 rgozim 'Added gradle support to omerojava component'
  - PR 6071 joshmoore 'Tests for 5.5.2 PRs'
  - PR 6176 sbesson 'Restore CLI DB integration tests'
  - PR 6207 jburel 'Review training example setup'
  - PR 6208 joshmoore 'test_user: don't assume single-threaded (see ome#6206)'

Generated by OMERO-push#208 (https://py3-ci.openmicroscopy.org/jenkins/job/OMERO-push/208/)
snoopycrimecop added a commit to snoopycrimecop/openmicroscopy that referenced this issue Jan 28, 2020
snoopycrimecop added a commit to snoopycrimecop/openmicroscopy that referenced this issue Jan 28, 2020
Repository: openmicroscopy/openmicroscopy
Excluded PRs:
  - PR 6205 chris-allan 'Test cases for expanded user session creation abilities' (status: failure)
  - PR 6101 dominikl 'Set namespace of fileannoation of omero.table - test' (status: failure)
  - PR 6099 will-moore 'Use chosen plugin for script UI if large number of options' (label: exclude)
  - PR 6096 will-moore 'Script gui file upload' (label: exclude)
  - PR 6094 manics 'omero.web.override_root_app to set custom web root app' (label: exclude)
  - PR 6093 manics 'admin diagnostics: show BF (and optionally other) jar versions' (label: exclude)
  - PR 6086 manics 'Alternative JSON configuration system for OMERO.web' (label: exclude)
  - PR 6083 manics 'web: omero.web.http_response_headers to set custom headers' (label: exclude)
  - PR 6081 will-moore 'Webclient performance fixes' (label: exclude)
  - PR 5966 joshmoore 'Allow testing multiple builds in travis' (status: failure)
  - PR 5946 manics 'Python gateway secure' (label: exclude)
  - PR 5914 kennethgillen 'Adding --as-default to CLI omero user joingroup' (label: exclude)
  - PR 5912 manics 'OMERO.web default log can be changed to stdout' (label: exclude)
  - PR 5910 manics 'Allow cli_login to be used without closing session' (label: exclude)
  - PR 5884 manics 'Ignore owner when checking omero.data.dir is accessible' (label: exclude)
Already up-to-date.

Merged PRs:
  - PR 6060 rgozim 'Added gradle support to omerojava component'
  - PR 6071 joshmoore 'Tests for 5.5.2 PRs'
  - PR 6176 sbesson 'Restore CLI DB integration tests'
  - PR 6207 jburel 'Review training example setup'
  - PR 6208 joshmoore 'test_user: don't assume single-threaded (see ome#6206)'

Generated by OMERO-push#340 (https://merge-ci.openmicroscopy.org/jenkins/job/OMERO-push/340/)
snoopycrimecop added a commit to snoopycrimecop/openmicroscopy that referenced this issue Jan 28, 2020
snoopycrimecop added a commit to snoopycrimecop/openmicroscopy that referenced this issue Jan 28, 2020
Repository: openmicroscopy/openmicroscopy
Excluded PRs:
  - PR 6093 manics 'admin diagnostics: show BF (and optionally other) jar versions' (label: exclude)
  - PR 6099 will-moore 'Use chosen plugin for script UI if large number of options' (label: exclude)
  - PR 6101 dominikl 'Set namespace of fileannoation of omero.table - test' (status: failure)
  - PR 5912 manics 'OMERO.web default log can be changed to stdout' (label: exclude)
  - PR 5966 joshmoore 'Allow testing multiple builds in travis' (status: failure)
  - PR 6096 will-moore 'Script gui file upload' (label: exclude)
  - PR 6083 manics 'web: omero.web.http_response_headers to set custom headers' (label: exclude)
  - PR 6205 chris-allan 'Test cases for expanded user session creation abilities' (status: error)
  - PR 6081 will-moore 'Webclient performance fixes' (label: exclude)
  - PR 5946 manics 'Python gateway secure' (label: exclude)
  - PR 6086 manics 'Alternative JSON configuration system for OMERO.web' (label: exclude)
  - PR 5914 kennethgillen 'Adding --as-default to CLI omero user joingroup' (label: exclude)
  - PR 5910 manics 'Allow cli_login to be used without closing session' (label: exclude)
  - PR 5884 manics 'Ignore owner when checking omero.data.dir is accessible' (label: exclude)
  - PR 6094 manics 'omero.web.override_root_app to set custom web root app' (label: exclude)
Already up-to-date.

Merged PRs:
  - PR 6060 rgozim 'Added gradle support to omerojava component'
  - PR 6071 joshmoore 'Tests for 5.5.2 PRs'
  - PR 6176 sbesson 'Restore CLI DB integration tests'
  - PR 6207 jburel 'Review training example setup'
  - PR 6208 joshmoore 'test_user: don't assume single-threaded (see ome#6206)'

Generated by OMERO-push#225 (https://idr-ci.openmicroscopy.org/jenkins/job/OMERO-push/225/)
snoopycrimecop added a commit to snoopycrimecop/openmicroscopy that referenced this issue Jan 28, 2020
Repository: openmicroscopy/openmicroscopy
Excluded PRs:
  - PR 6099 will-moore 'Use chosen plugin for script UI if large number of options' (user: will-moore)
  - PR 6071 joshmoore 'Tests for 5.5.2 PRs' (user: joshmoore)
  - PR 5914 kennethgillen 'Adding --as-default to CLI omero user joingroup' (user: kennethgillen)
  - PR 6208 joshmoore 'test_user: don't assume single-threaded (see ome#6206)' (user: joshmoore)
  - PR 5910 manics 'Allow cli_login to be used without closing session' (user: manics)
  - PR 6093 manics 'admin diagnostics: show BF (and optionally other) jar versions' (user: manics)
  - PR 6096 will-moore 'Script gui file upload' (user: will-moore)
  - PR 5884 manics 'Ignore owner when checking omero.data.dir is accessible' (user: manics)
  - PR 6081 will-moore 'Webclient performance fixes' (user: will-moore)
  - PR 6205 chris-allan 'Test cases for expanded user session creation abilities' (user: chris-allan)
  - PR 5966 joshmoore 'Allow testing multiple builds in travis' (user: joshmoore)
  - PR 5912 manics 'OMERO.web default log can be changed to stdout' (user: manics)
  - PR 6101 dominikl 'Set namespace of fileannoation of omero.table - test' (user: dominikl)
  - PR 5946 manics 'Python gateway secure' (user: manics)
  - PR 6094 manics 'omero.web.override_root_app to set custom web root app' (user: manics)
  - PR 6176 sbesson 'Restore CLI DB integration tests' (user: sbesson)
  - PR 6083 manics 'web: omero.web.http_response_headers to set custom headers' (user: manics)
  - PR 6086 manics 'Alternative JSON configuration system for OMERO.web' (user: manics)
  - PR 6207 jburel 'Review training example setup' (user: jburel)
  - PR 6060 rgozim 'Added gradle support to omerojava component' (user: rgozim)
Already up-to-date.

Generated by OMERO-push#278 (https://latest-ci.openmicroscopy.org/jenkins/job/OMERO-push/278/)
snoopycrimecop added a commit to snoopycrimecop/openmicroscopy that referenced this issue Jan 29, 2020
snoopycrimecop added a commit to snoopycrimecop/openmicroscopy that referenced this issue Jan 29, 2020
Repository: openmicroscopy/openmicroscopy
Excluded PRs:
  - PR 6205 chris-allan 'Test cases for expanded user session creation abilities' (status: error)
  - PR 6101 dominikl 'Set namespace of fileannoation of omero.table - test' (status: failure)
  - PR 6099 will-moore 'Use chosen plugin for script UI if large number of options' (label: exclude)
  - PR 6096 will-moore 'Script gui file upload' (label: exclude)
  - PR 6094 manics 'omero.web.override_root_app to set custom web root app' (label: exclude)
  - PR 6093 manics 'admin diagnostics: show BF (and optionally other) jar versions' (label: exclude)
  - PR 6086 manics 'Alternative JSON configuration system for OMERO.web' (label: exclude)
  - PR 6083 manics 'web: omero.web.http_response_headers to set custom headers' (label: exclude)
  - PR 6081 will-moore 'Webclient performance fixes' (label: exclude)
  - PR 5966 joshmoore 'Allow testing multiple builds in travis' (status: failure)
  - PR 5946 manics 'Python gateway secure' (label: exclude)
  - PR 5914 kennethgillen 'Adding --as-default to CLI omero user joingroup' (label: exclude)
  - PR 5912 manics 'OMERO.web default log can be changed to stdout' (label: exclude)
  - PR 5910 manics 'Allow cli_login to be used without closing session' (label: exclude)
  - PR 5884 manics 'Ignore owner when checking omero.data.dir is accessible' (label: exclude)
Already up-to-date.

Merged PRs:
  - PR 6060 rgozim 'Added gradle support to omerojava component'
  - PR 6071 joshmoore 'Tests for 5.5.2 PRs'
  - PR 6176 sbesson 'Restore CLI DB integration tests'
  - PR 6207 jburel 'Review training example setup'
  - PR 6208 joshmoore 'test_user: don't assume single-threaded (see ome#6206)'

Generated by OMERO-push#341 (https://merge-ci.openmicroscopy.org/jenkins/job/OMERO-push/341/)
joshmoore added a commit that referenced this issue Jan 29, 2020
test_user: don't assume single-threaded (see #6206)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant