Koozali.org: home of the SME Server

systemd <-> init conflict for 3rd party service (wazuh-agent)

Offline mmccarn

  • *
  • 2,626
  • +10/-0
Scenario:
I am exploring wazuh for vulnerability monitoring on my home network.

Wazuh includes a lightweight agent that gets installed on each device.

The wazuh-agent install command for Centos7 looks like this:
Code: [Select]
sudo WAZUH_MANAGER='[IP or DNS of wazuh server]' WAZUH_AGENT_GROUP='default' yum install https://packages.wazuh.com/4.x/yum/wazuh-agent-4.3.5-1.x86_64.rpm
Of course on SME I also needed to define and enable the service:
Code: [Select]
config set wazuh-agent service status enabled
The problem
After installing as above I could start the service after each reboot using systemctl start wazuh-agent, and I could enable the service using systemctl enable wazuh-agent, but the service was always disabled after again after a reboot.

I could solve the problem by creating a crontab entry @reboot sleep 10; /sbin/e-smith/systemctl start wazuh-agent - but that seemed... "unusual"...

The koozali wiki docs for service control say to run expand-template /etc/systemd/system-preset/49-koozali.preset -- however wazuh-agent still showed "disabled" in /etc/systemd/system-preset/49-koozali.preset.

Looking at the contents of /etc/e-smith/templates/etc/systemd/system-preset/49-koozali.preset/20services, it looks like services with an init script in /etc/rc.d/init.d are to 'disabled' in 49-koozali.preset. I found that there was indeed a script for wazuh-agent at /etc/rc.d/init.d/wazuh-agent.

Renaming that script (mv /etc/rc.d/init.d/wazuh-agent /etc/rc.d/init.d/wazuh-agent.huh) and re-expanding /etc/systemd/system-preset/49-koozali.preset solved the problem (temporarily?).

Conclusion
I suspect that the folks at wazuh are including both systemd and init startup scripts for wazuh-agent as a convenience.

I also suspect that the conflicting init script will come back with the next wazuh-agent update.

I have no idea how many packages are likely to include both systemd and init startup scripts.

Is this a bug that should be addressed in /etc/e-smith/templates/etc/systemd/system-preset/49-koozali.preset/20services, or an update step I should document on my network?

Offline Jean-Philippe Pialasse

  • *
  • 2,747
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: systemd <-> init conflict for 3rd party service (wazuh-agent)
« Reply #1 on: July 02, 2022, 01:07:27 AM »
expected behaviour.

you need to create a dropin file to make your service wanted for sme-server.target


[Install]
WantedBy=sme-server.target

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: systemd <-> init conflict for 3rd party service (wazuh-agent)
« Reply #2 on: July 02, 2022, 02:57:31 PM »
Find the name of your unit file in /usr/lib/systemd/system and override with something like this:

/etc/e-smith/templates/usr/lib/systemd/system/wazuh-service.d/50koozali.conf/80install

Containing:

Code: [Select]
[Install]
WantedBy=sme-server.target

You can of course add other overrides.

You can look in some of the other .d directories in there for more examples or various contribs eg wsdd,

Code: [Select]
or an update step I should document on my network?
Yup :-)

I think the wiki page could do with a slightly clarified/simplified method - the page does tell you, and once you know, you know. But it isn't easy straight off the bat. Reading the source on a few contribs will help enormously.

Note - if you don't want a reboot after install make sure you have a createlink for an event with the same name as your service.

This would be an ideal candidate for a small contrib.....
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline mmccarn

  • *
  • 2,626
  • +10/-0
Re: systemd <-> init conflict for 3rd party service (wazuh-agent)
« Reply #3 on: July 02, 2022, 08:25:49 PM »
[edit]
Don't use the instructions at the end of this post to customize systemd.

See this later post for a simpler solution:
https://forums.koozali.org/index.php/topic,54838.msg288340.html#msg288340
[/edit]


Thanks.  Creating a drop-in file didn't get the service enabled after a reboot.

I could not get wazuh-agent enabled unless I did one of these:
- rename or remove /etc/rc.d/init.d/wazuh-agent
OR
- customize 49koozali.preset to add wazuh-agent to '@list' *and* create a drop-in file with WantedBy=sme-server.targe

More details than anyone wants:

I created this file:
Quote from: /usr/lib/systemd/system/wazuh-agent.service.d/50-koozali.conf
[Install]
WantedBy=sme-server.target

However, even with that file, if /etc/rc.d/init.d/wazuh-agent exists:
Code: [Select]
# ls /etc/rc.d/init.d/wazuh*
/etc/rc.d/init.d/wazuh-agent
# expand-template /etc/systemd/system-preset/49-koozali.preset
# grep wazuh- /etc/systemd/system-preset/49-koozali.preset
disable wazuh-agent.service

... and after rebooting...
Code: [Select]
# systemctl status wazuh-agent
● wazuh-agent.service - Wazuh agent
   Loaded: loaded (/usr/lib/systemd/system/wazuh-agent.service; disabled; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/wazuh-agent.service.d
           └─50-koozali.conf
   Active: inactive (dead)

By comparison:
Code: [Select]
# mv /etc/rc.d/init.d/wazuh-agent /etc/rc.d/init.d/wazuh-agent.huh
# signal-event console-save
# grep wazuh /etc/systemd/system-preset/49-koozali.preset
enable wazuh-agent.service
# signal-event reboot
...
<reconnect>
...
# systemctl status wazuh-agent
● wazuh-agent.service - Wazuh agent
   Loaded: loaded (/usr/lib/systemd/system/wazuh-agent.service; enabled; vendor preset: enabled)
  Drop-In: /usr/lib/systemd/system/wazuh-agent.service.d
           └─50-koozali.conf
   Active: active (running) since Sat 2022-07-02 07:26:12 EDT; 5min ago
  Process: 1431 ExecStart=/usr/bin/env /var/ossec/bin/wazuh-control start (code=exited, status=0/SUCCESS)
   Memory: 481.5M
   CGroup: /system.slice/wazuh-agent.service
           ├─1529 /var/ossec/bin/wazuh-execd
           ├─1839 /var/ossec/bin/wazuh-agentd
           ├─2358 /var/ossec/bin/wazuh-syscheckd
           ├─2791 /var/ossec/bin/wazuh-logcollector
           └─2856 /var/ossec/bin/wazuh-modulesd

Jul 02 07:26:04 office.smedomain.us systemd[1]: Starting Wazuh agent...
Jul 02 07:26:04 office.smedomain.us env[1431]: Starting Wazuh v4.3.5...
Jul 02 07:26:05 office.smedomain.us env[1431]: Started wazuh-execd...
Jul 02 07:26:06 office.smedomain.us env[1431]: Started wazuh-agentd...
Jul 02 07:26:07 office.smedomain.us env[1431]: Started wazuh-syscheckd...
Jul 02 07:26:08 office.smedomain.us env[1431]: Started wazuh-logcollector...
Jul 02 07:26:09 office.smedomain.us env[1431]: Started wazuh-modulesd...
Jul 02 07:26:11 office.smedomain.us env[1431]: Completed.
Jul 02 07:26:12 office.smedomain.us systemd[1]: Started Wazuh agent.

Specifically, I think this bit from /etc/e-smith/templates/etc/systemd/system-preset/49-koozali.preset/20services disables any systemd service  that has a matching init script:
Quote
  unless ( $service->key ~~ @list ) {
    $status = "disable" if  -e "/etc/rc.d/init.d/".$service->key || -e "/etc/rc.d/init.d/supervise/".$service->key;
    }


With your hint about a drop-in file, I've implemented a persistent solution by:
* customizing /etc/e-smith/templates/etc/systemd/system-preset/49-koozali.preset/05config
* and creating /usr/lib/systemd/system/wazuh-agent.service.d/50-koozali.conf
Code: [Select]
mkdir -p /etc/e-smith/templates-custom/etc/systemd/system-preset/49-koozali.preset
cd /etc/e-smith/templates-custom/etc/systemd/system-preset/49-koozali.preset
sed 's/ldap.init/ldap.init wazuh-agent/' /etc/e-smith/templates/etc/systemd/system-preset/49-koozali.preset/05config >05config

mkdir -p /usr/lib/systemd/system/wazuh-agent.service.d
printf "[Install]\nWantedBy=sme-server.target\n" > /usr/lib/systemd/system/wazuh-agent.service.d/50-koozali.conf
signal-event console-save

With these two changes, wazuh-agent starts successfully after a reboot.
« Last Edit: July 03, 2022, 02:06:51 PM by mmccarn »

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: systemd <-> init conflict for 3rd party service (wazuh-agent)
« Reply #4 on: July 02, 2022, 08:58:44 PM »
KK.

JP will know but he's busy in ER for a few days - probably around after the weekend.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline Jean-Philippe Pialasse

  • *
  • 2,747
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: systemd <-> init conflict for 3rd party service (wazuh-agent)
« Reply #5 on: July 02, 2022, 10:44:27 PM »
you need dropin file AND config entry as a service with status enabled

please do not mess up with preset file and template

Offline Jean-Philippe Pialasse

  • *
  • 2,747
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: systemd <-> init conflict for 3rd party service (wazuh-agent)
« Reply #6 on: July 02, 2022, 11:03:24 PM »
ok forgot one point

you have an init.d file AND a .service file provided by the wahzu rpms. both are rhe same rpm? no way to have only one?

if yes, then indeed you will need a custom fragment template for preset but specific for your service.

99wahzu

wazhu-agent.service= enable

if you do allow any init.d files you will create conflict for network AND firewall at least. 
« Last Edit: July 02, 2022, 11:05:34 PM by Jean-Philippe Pialasse »

Offline mmccarn

  • *
  • 2,626
  • +10/-0
Re: systemd <-> init conflict for 3rd party service (wazuh-agent)
« Reply #7 on: July 03, 2022, 01:56:21 PM »
you have an init.d file AND a .service file provided by the wahzu rpms. both are rhe same rpm? no way to have only one?

Short of compiling the agent from scratch, no - the only rpm available installs both the systemd service and an init script.

Quote
if yes, then indeed you will need a custom fragment template for preset but specific for your service.

99wazuh

That does it; thanks!

Summary:
To get wazuh-agent running on SME10:

* Install wazuh-agent using the command provided from your wazuh-server

* Create a SME config database entry
Code: [Select]
config set wazuh-agent service status enabled
* Create a custom systemd template to override the wazuh dual systemd/init options:
Code: [Select]
mkdir -p /etc/e-smith/templates-custom/etc/systemd/system-preset/49-koozali.preset
cd /etc/e-smith/templates-custom/etc/systemd/system-preset/49-koozali.preset
echo 'enable wazuh-agent.service'

* Update settings and reboot
Code: [Select]
signal-event console-save; signal-event reboot

* After rebooting, re-connect and make sure wazuh-agent starts automatically:
Code: [Select]
systemctl status wazuh-agent

Offline Jean-Philippe Pialasse

  • *
  • 2,747
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: systemd <-> init conflict for 3rd party service (wazuh-agent)
« Reply #8 on: July 03, 2022, 07:22:07 PM »
good to know


final touch:

ideally the template should be with a little code so you can disable using the config

something like this


{
$OUT="";
$OUT.="enable wahzu-agent.service" if ${'wahzu-agent'}{'status'} eq "enabled";
}

Offline mmccarn

  • *
  • 2,626
  • +10/-0
Re: systemd <-> init conflict for 3rd party service (wazuh-agent)
« Reply #9 on: July 04, 2022, 02:17:38 PM »
Thanks.

I deleted the original template (/etc/e-smith/templates-custom/etc/systemd/system-preset/49-koozali.preset) and created a new one named "99-wazuh-agent":
Code: [Select]
{
$customservice='wazuh-agent';
$OUT .= "enable                $customservice.service\n" if ${$customservice}{'status'} eq "enabled";
}


I tested the results with status:enabled, status:disabled and missing config entry.

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: systemd <-> init conflict for 3rd party service (wazuh-agent)
« Reply #10 on: July 04, 2022, 03:30:10 PM »
Can you add a wiki page for this?
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation