Get integration tests running in CI

[jaqx0r]

@3c2b2ff5 contrinubted a samba integration test script but it needs a bit of work to run in Travis CI. I'd also like to turn my (not submitted) OpenLDAP test script into an integration test.

So two goals:

• make the integration tests runnable manually
• incorporate the tests on CI. We don't have to keep Travis if, say, spinning up a docker container on Circle to run the source server is significantly easier.

[3c2b2ff5]

I have setup a new branch CircleCi without travis.
I noticed that when ahving python3-ldap in requirements.txt the build will fail, with python-ldap the build succeeds, even when running the tests with Python 3.
Next step will be to spinn up a docker container with samba 4 AD and another one with openldap or slapd.
I still need to figure out how to test against the source server.

[jaqx0r]

Awesome. The python package's name is `python-ldap`, the Debian package calls it `python-ldap` or `python3-ldap`. So don't put python3-ldap in requirements.txt. How are you testing your Circle config?

…

On Sat, 2 Nov 2019 at 05:43, 3c2b2ff5 ***@***.***> wrote: I have setup a new branch CircleCi <https://github.com/3c2b2ff5/nsscache/tree/CircleCi> without travis. I noticed that when ahving python3-ldap in requirements.txt the build will fail, with python-ldap the build succeeds, even when running the tests with Python 3. Next step will be to spinn up a docker container with samba 4 AD and another one with openldap or slapd. I still need to figure out how to test against the source server. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#98?email_source=notifications&email_token=AAXFX62JPQJAV5OQYCIXCPDQRR2G3A5CNFSM4JHUUQD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEC32GAA#issuecomment-548905728>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAXFX65XFGRPJIVANO4YWQLQRR2G3ANCNFSM4JHUUQDQ> .

[3c2b2ff5]

currently I created a circleci account and it runs successfully, with the difference to travisci that you cannot choose a branch, at least I couldn't finde it.
It seems samba cannot be installed because of the file system circleci is using:

ERROR(<class 'samba.provision.ProvisioningError'>): Provision failed - ProvisioningError: Your filesystem or build does not support posix ACLs, which s3fs requires.  Try the mounting the filesystem with the 'acl' option.

But it is really fast. The CI runs in less than a minute.
There is a possibility to run circleci local ci, look here.

[3c2b2ff5]

Now the build in CircleCi runs successful, samba is installed and the domain provisioned, the users, the groups and the group members are are successfully added, there are still some issues to be solved.

Samba is not being run as an AD Domain Controller: Masking samba-ad-dc.service

ERROR(<class 'samba.provision.ProvisioningError'>): Provision failed - ProvisioningError: Your filesystem or build does not support posix ACLs, which s3fs requires.  Try the mounting the filesystem with the 'acl' option.

And a lot of:

Could not find machine account in secrets database: Failed to fetch machine account password for LOCAL from both secrets.ldb (Could not find entry to match filter: '(&(flatname=LOCAL)(objectclass=primaryDomain))' base: 'cn=Primary Domains': No such object: dsdb_search at ../source4/dsdb/common/util.c:4705) and from /var/lib/samba/private/secrets.tdb: NT_STATUS_CANT_ACCESS_DOMAIN_INFO

[3c2b2ff5]

I am tending now to run CircleCi loacally or on CircleCi just to test the config and to run python3 runtests.py. Then spinn up two docker containers locally, one with nsscache and the other one with the source server.
The nsscache container should run nsscache installation with a modified nssccache.conf to contact openldap | AD running on the other container.
I think by creating a docker network and connecting both containers to it, they should be able to communicate. Of course we can run nsscache and openldap | AD on the same container.
An ldap query to AD must be encrypted. For the test the client just needs the default certificate form the AD, which is created with the installation:

# echo -n | openssl s_client -connect ad.host:636 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > /usr/local/share/ca-certificates/ad.crt
# update-ca-certificates

Do you have further ideas?

[3c2b2ff5]

I have managed to install samba4 AD with Travis. nsscache -d update -f fails because of permissions, it cannot lock /var/run/nsscache. nsscache -d verify has always 3 warnings, but no errors.
Running both commands with sudo didn't work neither, I think because of the environment.
The new branch is here.
I hope you can fix it.

[3c2b2ff5]

I fixed the verify command by adding a nsswitch.conf file. The update still fails because of the lacking permissions to lock /var/run/nsscache.

[3c2b2ff5]

I fixed it in the last commit. I had to switch the verify and the update commands, otherwise when running the verify at the end, it gives the error that the cache entries in passwd/group/shadow are not available via NSS.

[3c2b2ff5]

Any ideas why running verify after update result in the mentioned error?

[jaqx0r]

I haven't had a chance to test yet, but I'll investigate layer today or tomorrow.

…

On Fri, 8 Nov 2019, 13:32 3c2b2ff5, ***@***.***> wrote: Any ideas why running verify after update result in the mentioned error? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#98?email_source=notifications&email_token=AAXFX6ZI7HPHDFVM3HQ2RBDQSVL7RA5CNFSM4JHUUQD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDRH57I#issuecomment-551714557>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAXFX6YCAAO24QDTDSBPYRDQSVL7RANCNFSM4JHUUQDQ> .

[3c2b2ff5]

cool. Thanks

[jaqx0r]

The CircleCI build now runs the OpenLDAP regression test as par tof the release. I discovered that the sambaSID changes broke this, so that's good that it caught them. I think making sambaSID an essential attribute only if use_rid is configured is the right thing to do.

I tried to get the samba4 script running, but I got stuck trying to figure out how to run samba as a non-root user.

[3c2b2ff5]

Indeed! Running openldap without samba3 schema will break the tests and nsscache. Normally you have the smaba3 schema integration within openldap. Using openldap without any extensions for authentication or as a directory service we need to modify the code at this point to satisfy a plain openldap installation.

To install and run samba4 AD you need to run the script samba.sh with sudo. Unfortunately there is no other way to do it. All the operations (installation/provisioning/user and group creation etc.) need root privileges.

[3c2b2ff5]

I cannot test now with CircleCi. I keep get the following error at the coveralls stage:

#!/bin/bash -eo pipefail
coveralls
/bin/bash: coveralls: command not found
Exited with code 127

I already created an account on coveralls. I have no prior experience with coveralls.

[jaqx0r]

You can instal lthe python package 'python-coveralls' to get the command. All it's doing isuplodaing the coverage information to http://coveralls.io. If you want you can ignore that, comment it out on your own branch, but also I don't think you need to run your own circleci now. If you can get a shell script doing the full setup locally we can work from there.

…

On Mon, 11 Nov 2019 at 18:22, 3c2b2ff5 ***@***.***> wrote: I cannot test now with CircleCi. I keep get the following error at the coveralls stage: #!/bin/bash -eo pipefail coveralls /bin/bash: coveralls: command not found Exited with code 127 I already created an account on coveralls. I have no prior experience with coveralls. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#98?email_source=notifications&email_token=AAXFX632YZEJDKJGOENI3MDQTGPFNA5CNFSM4JHUUQD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDXVPPA#issuecomment-552556476>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAXFX66DBMP7IKEMDEPQUOLQTGPFNANCNFSM4JHUUQDQ> .

[3c2b2ff5]

Well I could get the samba script running on travisci but not on circleci, circleci seems to use filesystem not supporting posix acl's, which is required for the domain provision.
I am trying to find an option how to enable posix acl's on circleci build, but no success. Another possibility would be remount the filesystem with acl, but I don't know if this is possible at all, maybe using a build that supports them would make the script run successful.

[3c2b2ff5]

I have a solution for this issue, we need to compile samba with non default options to fake acl's. I'll be working on it today.

[3c2b2ff5]

So compiling samba takes 20 - 25 minutes. I don't think this a good idea to compile in CI.

[jaqx0r]

Maybe there is a docker image we can reuse?

…

On Fri, 15 Nov 2019, 17:41 3c2b2ff5, ***@***.***> wrote: So compiling samba takes 20 - 25 minutes. I don't think this a good idea to compile in CI. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#98?email_source=notifications&email_token=AAXFX67XHWD7QGEMW4NB5EDQT3GMRA5CNFSM4JHUUQD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEGAHSA#issuecomment-554435528>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAXFX63T2TASJWSPMROBKALQT3GMRANCNFSM4JHUUQDQ> .

[3c2b2ff5]

Indeed, there are several images out there, but can they be used with circleci? or are we stuck with what circleci provides?

[3c2b2ff5]

we still have the possibility to compile samba4 from source and create a Debian package locally and provide it to the CI, but this is also not the best idea.

[3c2b2ff5]

after some tests, the image doesn't provide neither sudo nor apt-get, so I cannot install any requirements within a container. It seems it is a standard behavior of docker images.
If any circleci image supports posix acl's, the issue will be solved, or if there is a possibility to remount the filesystem with acl, some thing like mount -o remount,acl /.
Or may be get back to travis. It is your call.

[3c2b2ff5]

or may be create a custom nsscache docker image?

[3c2b2ff5]

I tested further images from docker hub, unfortunately we still have the same issue with filesytem. The problem seems that circleci mounts the filesystem from whatever image we use without posix ACLs support.
I can't find a way to bypass this. I asked few days a go on circleci discuss how to get around this, but still no answer.

[3c2b2ff5]

I got it to work, but with the machine executor, non of the docker images or orbs support posix ACLs. If we want to provision a Samba4 AD, then we have to use the machine executor.

[3c2b2ff5]

If you decided to provision a Samba4 AD, there are some challenges to achieve, since the machine executor is not a Python build.
If so, please let me know, I'll try to find out how to make the tests run successfully.

[3c2b2ff5]

I am stuck with the with the machine executor. Please take a look to the config.yml.
I can't install nsscache and run the ldap regtest.

[3c2b2ff5]

The last commit works.
The regtest of openldap doesn't work, there is an error Can't contact LDAP server (-1).
Further more the Samba4 AD test is not implemented yet.

[jaqx0r]

I don't understand what the problem is with the POSIX acls on the filesystem, can you explain what errors you got?

…

On Sun, 17 Nov 2019 at 22:54, 3c2b2ff5 ***@***.***> wrote: The last commit <https://github.com/3c2b2ff5/nsscache/blob/CircleCi/.circleci/config.yml> works. The regtest of openldap doesn't work, there is an error Can't contact LDAP server (-1). Further more the Samba4 AD test is not implemented yet. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#98?email_source=notifications&email_token=AAXFX6YUUJ7RWOWNPVBP24LQUHDRDA5CNFSM4JHUUQD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEIYRDI#issuecomment-554797197>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAXFX64COQBCX3JNQXMOI5DQUHDRDANCNFSM4JHUUQDQ> .

[3c2b2ff5]

to provision a Samba4 domain, the filesystem must support posix ACLs, otherwise you get the following error:

 ERROR(<class 'samba.provision.ProvisioningError'>): Provision failed - ProvisioningError: Your filesystem or build does not support posix ACLs, which s3fs requires.  Try the mounting the filesystem with the 'acl' option.
  File "/usr/lib/python2.7/dist-packages/samba/netcmd/domain.py", line 538, in run
    backend_store=backend_store)
  File "/usr/lib/python2.7/dist-packages/samba/provision/__init__.py", line 2316, in provision
    backend_store=backend_store)
  File "/usr/lib/python2.7/dist-packages/samba/provision/__init__.py", line 1921, in provision_fill
    names.domaindn, lp, use_ntvfs)
  File "/usr/lib/python2.7/dist-packages/samba/provision/__init__.py", line 1649, in setsysvolacl
    raise ProvisioningError("Your filesystem or build does not support posix ACLs, which s3fs requires.  "
Exited with code 255

Either the filesystem supports posix ACLs, or we remount the filesystem with something like: mount -o remount,acl /, which is not possible in a CI environment, or we need to compile Samba4 from source with the option --with-ntvfs-fileserver and provision the domain withe the --use-ntvfs option.
Posix ACLs are not supported in Docker, therefore a machine executor must be used if we want to test against Samba4 AD. I got this answer on discuss CircleCi.
If you would like to reproduce the error, just add - run: sudo tests/samba.sh to .circleci/config.yml on master branch, where orb is in use.

It is your call if you want test against Samba4 AD.

The previous error is related to dns resolver for my understanding. Since example.com can be resolved and there is no LDAP server running there, I guess we need to resolve example.com to local host for openldap as well as for Samba4 AD.

[3c2b2ff5]

I opened an issue on CircleCI-Public/python-orb, maybe they can add the posix ACLs support.

[jaqx0r]

Thanks for the detailed explanation, I really appreciate you taking the time. I don't know what to do about it yet, but for now will leave this bug open.

…

On Mon, 18 Nov 2019, 13:45 3c2b2ff5, ***@***.***> wrote: I opend an issue on CircleCI-Public/python-orb <CircleCI-Public/python-orb#25>, maybe they can add the posix ACLs support. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#98?email_source=notifications&email_token=AAXFX66O25VAUO5IINGQGJTQUKL6LA5CNFSM4JHUUQD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEKPGVI#issuecomment-555021141>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAXFX663B2Q5P6QCXL6Y7WTQUKL6LANCNFSM4JHUUQDQ> .

[3c2b2ff5]

Thanks for the detailed explanation, I really appreciate you taking the time. I don't know what to do about it yet, but for now will leave this bug open.
…
On Mon, 18 Nov 2019, 13:45 3c2b2ff5, @.***> wrote: I opend an issue on CircleCI-Public/python-orb <CircleCI-Public/python-orb#25>, maybe they can add the posix ACLs support. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#98?email_source=notifications&email_token=AAXFX66O25VAUO5IINGQGJTQUKL6LA5CNFSM4JHUUQD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEKPGVI#issuecomment-555021141>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAXFX663B2Q5P6QCXL6Y7WTQUKL6LANCNFSM4JHUUQDQ .

No problem at all. I feel committed as well, and I would like to make things perfect as long as it is possible. I don't like half solutions.
Let's wait and see what the developer of CircleCI-Public/python-orb will say about it. Once we have an official answer, further steps will be clear.
With the machine executor, I just wanted to demonstrate that it could work if we get to a dead end regarding Samba4 AD and python-orb, so that we still have an option, which can be definitely much better implemented than my current solution.

[3c2b2ff5]

Hi,

we would like to roll out nsscache on Debian buster in production, any ideas when the new Debian package will be available?

Thanks

[3c2b2ff5]

Hi Jamie,
any news about the Debian package?

[jaqx0r]

https://tracker.debian.org/pkg/nsscache says that nsscache has been 0.39-2 since november in testing. Sorry I didn't mention that sooner.

…

On Wed, 15 Jan 2020 at 19:00, 3c2b2ff5 ***@***.***> wrote: Hi Jamie, any news about the Debian package? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#98?email_source=notifications&email_token=AAXFX62JBS3MPJZQVVECCT3Q527C7A5CNFSM4JHUUQD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEI7M6IQ#issuecomment-574541602>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAXFX6ZAEPGPWH22ARW4XKLQ527C7ANCNFSM4JHUUQDQ> .

[3c2b2ff5]

Thanks a lot.