Real time config support

[benlangfeld]

WIP

• Integration tests covering the module's functionality
• Switch to seeding cn=config from slapd.conf as an interim measure
• Configure cn=config directly via LDAP, dropping slapd.conf
• Integrate support for managing directory via custom resources
• Add support for loading schemas from .schema or .ldif files (custom resource type)

[benlangfeld]

/cc @bklang

[benlangfeld]

This obviates all of #31, #28, #19 and #18.

[CtrlC-Root]

@benlangfeld how close is this to being ready to merge? I'm really looking forward to having this in the module and I was trying to determine if it's worth running from a fork for the time being or if I should just wait a day or two for this to be merged.

[CtrlC-Root]

@benlangfeld also I've found a bug with setting up a server from scratch. If I apply this manifest

class {'ldap::server::master':
  suffix => 'dc=example,dc=com',
  rootpw => 'password',
}

on a clean install of CentOS 6.5 x64 then puppet fails with:

Info: Applying configuration version '1401827444'
Notice: /Stage[main]/Ldap/File[/etc/openldap]/ensure: created
Notice: /Stage[main]/Ldap::Server::Convertschema/File[/etc/openldap/convertschema.sh]/ensure: defined content as '{md5}88f3f59a5bfeda5c8586a01d77bc8928'
Notice: /Stage[main]/Ldap::Server::Redhat/File[/etc/sysconfig/ldap]/ensure: created
Info: /Stage[main]/Ldap::Server::Redhat/File[/etc/sysconfig/ldap]: Scheduling refresh of Service[slapd]
Notice: /Stage[main]/Ldap::Server::Generic/Package[openldap-servers]/ensure: created
Notice: /Stage[main]/Ldap::Server::Master/Ldap::Server::Database[{2}bdb]/File[/var/lib/ldap/{2}bdb]/ensure: created
Notice: /Stage[main]/Ldap::Server::Generic/Ldap::Server::Builtin_schema[nis]/Exec[load_schema_nis]/returns: ldap_sasl_interactive_bind_s: Can't contact LDAP server (-1)
Error: /usr/bin/ldapadd -QY EXTERNAL -H ldapi:/// < nis.ldif returned 255 instead of one of [0]
Error: /Stage[main]/Ldap::Server::Generic/Ldap::Server::Builtin_schema[nis]/Exec[load_schema_nis]/returns: change from notrun to 0 failed: /usr/bin/ldapadd -QY EXTERNAL -H ldapi:/// < nis.ldif returned 255 inste
ad of one of [0]
Notice: /Stage[main]/Ldap::Server::Generic/Ldap::Server::Builtin_schema[core]/Exec[load_schema_core]/returns: ldap_sasl_interactive_bind_s: Can't contact LDAP server (-1)
Error: /usr/bin/ldapadd -QY EXTERNAL -H ldapi:/// < core.ldif returned 255 instead of one of [0]
Error: /Stage[main]/Ldap::Server::Generic/Ldap::Server::Builtin_schema[core]/Exec[load_schema_core]/returns: change from notrun to 0 failed: /usr/bin/ldapadd -QY EXTERNAL -H ldapi:/// < core.ldif returned 255 in
stead of one of [0]
Notice: /Stage[main]/Ldap::Server::Generic/Ldap::Server::Builtin_schema[inetorgperson]/Exec[load_schema_inetorgperson]/returns: ldap_sasl_interactive_bind_s: Can't contact LDAP server (-1)
Error: /usr/bin/ldapadd -QY EXTERNAL -H ldapi:/// < inetorgperson.ldif returned 255 instead of one of [0]
Error: /Stage[main]/Ldap::Server::Generic/Ldap::Server::Builtin_schema[inetorgperson]/Exec[load_schema_inetorgperson]/returns: change from notrun to 0 failed: /usr/bin/ldapadd -QY EXTERNAL -H ldapi:/// < inetorg
person.ldif returned 255 instead of one of [0]
Notice: /Stage[main]/Ldap::Server::Generic/Service[slapd]/ensure: ensure changed 'stopped' to 'running'
Info: /Stage[main]/Ldap::Server::Generic/Service[slapd]: Unscheduling refresh on Service[slapd]
Notice: /Stage[main]/Ldap::Server::Generic/Ldapdn[global confg]/ensure: created
Notice: /Stage[main]/Ldap::Server::Generic/Ldapdn[module config]/ensure: created
Notice: /Stage[main]/Ldap::Server::Master/Ldap::Server::Database[{2}bdb]/Ldapdn[{2}bdb database config]/ensure: created
Notice: /Stage[main]/Ldap::Server::Generic/Ldap::Server::Module[back_bdb]/Ldapdn[back_bdb module config]/ensure: created
Notice: /Stage[main]/Ldap::Server::Master/Ldap::Server::Database[{2}bdb]/Ldapdn[{2}bdb indices]/ensure: created
Notice: Finished catalog run in 20.71 seconds

I suspect this is because the service hasn't been started when you try to run ldapadd. I can work around this issue by doing this instead (once again on a fresh install):

$ yum install openldap-servers
$ service slapd start
$ puppet agent --test

and that completes successfully.

[benlangfeld]

This is indeed the service dependency issue I was talking about, and it is the only bug I am aware of. Everything works fine on Debian/Ubuntu, and CentOS builds on a second run. You can see this easily using kitchen converge centos. If you have suggestions for resolving this issue, that would be the last step before this could be merged, IMO. I'm using this PR in dev and staging, and preparing to roll it out to production this week.

[CtrlC-Root]

Excellent. I'm about one week away from putting this in production so I will definitely look into it and see if I can figure it out. I would prefer to use a puppet forge published module instead of a github fork so I have every interest to find this bug.

[benlangfeld]

Could I get a review of this please, @torian?

[torian]

@benlangfeld First of all, sorry for the delay, and thanks for the code.
It's a lot to look at, and I'm really short in time. I'd like to merge most of it, but not without a little discussion first. Are you on IRC ?

[benlangfeld]

Sure, benlangfeld on Freenode.

I will note ahead of time that I'm not particularly happy with the ldapdn implementation, though it works right now. Where it doesn't work is replacement of multi-value attributes where some values match those asserted, and no comparison is possible in OpenLDAP, for example olcDbConfig. Long-term I think it would be best to replace the ldapdn provider with an alternative implementation on ActiveLDAP.

[torian]

@benlangfeld, my main concern is that I don't want to remove support for slapd.conf, as there are many places where it is still in use. The other issue that rings a bell, is the fact that 547f7af makes the code dependent on preseeding the debian package on a generic manifest, and breaks it all with other distros (haven't tested it though), or at least gets stuff into other distros that shouldn't be there (/var/cache/local/preseeding on rhel as an example).
That being said, your contribution is huge and greatly appreciated.

[CtrlC-Root]

@torian I'd like to throw my $0.02 in here with regards to the use of slapd.conf. It's been deprecated by the upstream project for a while now and most install guides will not even mention it anymore. I don't think any new users of OpenLDAP would expect or even know about it. If you are concerned about existing users of this module, then I believe an appropriate version change (see: semver.org) would be enough to notify them of the incompatible behavior. As long as slapd.conf continues to be in use we can't implement some of the more advanced features this pull request provides. Features that I think would be a deciding factor when considering which module to use for managing OpenLDAP.

[benlangfeld]

As for the preseed issue, sure, we could make that conditional on Debian such that the file isn't created where it won't be used (CentOS). The preseed stuff doesn't break CentOS as far as I can tell.

I will note that there is an issue I described above regarding CentOS. I think this is a Puppet problem rather than an OpenLDAP problem, so to speak, but I'm new to Puppet and I don't run OpenLDAP on CentOS, so I would appreciate some help.

As for maintaining slapd.conf,as @CtrlC-Root says, it's long-deprecated and shouldn't be in use. I would support this becoming v2.0 of this module as per SemVer.

[CtrlC-Root]

@benlangfeld I'm going to pull your most recent changes into my fork (only until this is merged ;) and test it again tonight. I'll let you know if the problem is still present.

[CtrlC-Root]

@benlangfeld the issue is still there. I haven't had a chance to look into it yet though. All you need to do is run puppet once, "service slapd start", then run puppet again. It's definitely something we need to fix but it's more of a minor inconvenience than a show-stopping bug. I only need to start slapd once on production machines. I'll let you know if I figure it out.