Koozali.org: home of the SME Server
Contribs.org Forums => Koozali SME Server 10.x Contribs => Topic started by: ReetP on September 22, 2021, 11:01:19 AM
-
So, I have started working on this again.
Affa is a huge piece of code and pretty complicated. I have cleaned up a lot of it, tried to refine some of it, and fix some of it. And I am no coder.
It probably needs breaking into some separate libraries, and I am pretty sure there is a lot of duplication in there.
Nonetheless there is some code to test. It is a LONG way from prime time.
I will make this quite clear, right here, and right now.
DO NOT USE THIS IN PRODUCTION.
DO NOT USE THIS WITH v9.
Get two v10 VMs and test (see the note below about Windows testing)
I think there are probably some issues in there that may totally destroy your backup, or during a restore.
So, for those who have claimed they "can't code but want to help test" now is the time to put your money where your mouth is.
The main import bug is https://bugs.koozali.org/show_bug.cgi?id=11024
The wiki page, which lists associated bugs at the bottom is here.
https://wiki.koozali.org/Affa
It is badly out of date and will need a lot of rewriting. Volunteers to assist are required. If you don't have wiki editing rights then ask us.
You will need to do something like this to install affa - do NOT use 'update' or 'upgrade' with smedev. It will break your server every time.
You will need the openfusion repo.
yum --enablerepo=smedev,openfusion install smeserver-affa smeserver-systemd-control
It has several linked bugs. Each needs testing and verifying. Use the bug tracker to make practical comments and notes - bugzilla is not a place for idle chatter. If you want to waffle, do it here.
Make sure you get logs of any errors. /var/log/messages and /var/log/affa/*
Make sure you comment on the correct bug. Try and stay focused and on topic.
Note that this is NOT a 'how do I use affa' session. This is real testing.
If you are verifying, use a verification template.
See this page for a template to use:
https://wiki.koozali.org/SME_Server:Documentation:QA:Verification
I am in the process of modifying the code to use RSA keys instead of DSA. I hope that I will push that to CVS for testing later today.
http://bugs.contribs.org/show_bug.cgi?id=10783
Please go though each affa option carefully and see what works, and what does not work. Be patient and methodical and MAKE NOTES so you, and I, can repeat the issue.
If you find a potential issue don't open another bug and until we have had a look.
There is an option to run rsyncd on Windows - I have nothing to test this on so will need assistance.
You can ask questions here, or ask for if you really want to get involved in helping properly then DM me and ask for a Rocket.Chat account on my server and you can chat to us real time.
Please read these guides to help us:
https://www.chiark.greenend.org.uk/~sgtatham/bugs.html
http://www.catb.org/esr/faqs/smart-questions.html
http://xyproblem.info/
Remember, this is open source. Don't sit there and complain. Get involved. Learn. Those who help tend to get helped......
-
Newer version 3.3.1-4 in smedev.
Updates keys from DSA to RSA 4096. (No it does not try to migrate old ones - do a new send-key)
* Wed Sep 22 2021 John Crisp <jcrisp@safeandsoundit.co.uk> 3.3.1-4
- Change ssh keys from DSA to RSA 4096 [SME: 10783]
- Fix missing /var/affa store dir
Tagged with: smeserver-affa-3_3_1-4_el7_sme
/usr/bin/plague-client build smeserver-affa smeserver-affa-3_3_1-4_el7_sme contribs10
Package smeserver-affa enqueued. Job ID: 3329.
Looking forward to the testing and verifications.
-
I used epel repo as well :-)
# yum install smeserver-affa --enablerepo=smedev,epel,openfusion
-
So that broke things. This should fix it for now.
All the SSH & Rsync commands need refactoring so I have untidied the SSH commands for now.
* Wed Sep 22 2021 John Crisp <jcrisp@safeandsoundit.co.uk> 3.3.1-5
- Modify ssh key links [SME: 10783]
- Untidy the ssh commands so I can see them and tidy them more easily
-
'Looking forward to the testing and verifications'
'If you want to waffle, do it here'
so far so good!
# yum --enablerepo=smedev update smeserver-affa
---> Package smeserver-affa.noarch 0:3.3.1-1.el7.sme will be updated
---> Package smeserver-affa.noarch 0:3.3.1-5.el7.sme will be an update
---> Package smeserver-systemd-control.noarch 0:0.1-2 will be installed
---> Package python3.x86_64 0:3.6.8-18.el7 will be installed
---> Package python3-libs.x86_64 0:3.6.8-18.el7 will be installed
---> Package python3-pip.noarch 0:9.0.3-8.el7 will be installed
---> Package python3-setuptools.noarch 0:39.2.0-10.el7 will be installed
# signal-event post-upgrade; signal-event reboot
SUCCESS sshPort=4567
SUCCESS chattyOnSuccess=9
SUCCESS SMEServer=yes
SUCCESS Watchdog=yes
SUCCESS RPMCheck=yes
Other properties not checked (yet)
BandwidthLimit=0
dedup=no
DiskSpaceWarn=risky
SUCCESS Delete folder from targetsvr user /home directory, check folder deleted on backup server
SUCCESS Restore folder to targetsvr using export RDIR=/path/ rsync $RDIR method
SUCCESS Reduce scheduled archives, check archives reduced and deleted
SUCCESS Increase scheduled archives, check scheduled archives
SUCCESS bkupsvr missing RPM's compared to targetsvr notified by email
SUCCESS affa --run JOB
SUCCESS affa --configcheck
SUCCESS affa --make-cronjobs
SUCCESS affa --send-key [JOB]
SUCCESS affa --check-connections [JOB JOB ...]
affa --resume-interrupted NOT TESTED
SUCCESS affa --full-restore [--preserve-newer=no] [--delete=yes] JOB [ARCHIVE]
SUCCESS affa --list-archives [--csv] [JOB JOB ...]
SUCCESS affa --status [--csv]
SUCCESS affa --show-config-pathes [--csv] [JOB JOB ...]
SUCCESS affa --show-default-config
SUCCESS affa --show-schedule [-15]
SUCCESS affa --show-property PROPERTY
SUCCESS affa --log-tail [JOB]
SUCCESS affa --send-status
SUCCESS affa --disk-usage [--csv]
SUCCESS affa --cleanup JOB
SUCCESS affa --rename-job JOB NEWNAME
SUCCESS affa --move-archive JOB NEWROOTDIR
SUCCESS affa --delete-job JOB
SUCCESS affa --revoke-key JOB
SUCCESS affa --kill JOB
affa --killall NOT TESTED
SUCCESS affa --mailtest JOB
affa --nrpe [JOB JOB ...] NOT TESTED
SUCCESS affa --version
SUCCESS affa --warranty
SUCCESS affa --license
SUCCESS affa --help
-----------------------
SUCCESS affa --rise --all
pre-rise on targetsvr;
systemctl stop qpsmtpd.service
systemctl stop sqpsmtpd.service
systemctl stop crond.service
systemctl stop dovecot.service
systemctl stop httpd-e-smith.service
systemctl stop smb.service
systemctl stop qmail.service
systemctl stop pop3.service
systemctl stop pop3s.service
systemctl stop ftp.service
SUCCESS affa --undo-rise
SUCCESS affa --full-restore (after deleting a few user folders on targetsvr)
gives; Error (1766): Executing script SME/signal-post-upgrade-reboot on targetsvr failed; but targetsvr rebooted and folders restored.
options [--preserve-newer=no] [--delete=yes] NOT TESTED
-----------------------
FAIL affa --init-nrpe (never used this option but do use nrpe)
bkupsvr:/etc/nagios# affa --init-nrpe
sh: /etc/init.d/nrpe: No such file or directory
bkupsvr:/etc/nagios# systemctl status nrpe.service
nrpe.service - Nagios Remote Program Executor
-----------------------
-
There is an option to run rsyncd on Windows - I have nothing to test this on so will need assistance.
I have some win clients with rsyncd running (for backups with BackupPC), I can give it a try.
Any documentation available on how the rsync option should work?
-
so far so good!
Nice!! And thank you for stepping up to the plate.
Sorts the men from the boys.
SUCCESS affa --full-restore (after deleting a few user folders on targetsvr)
gives; Error (1766): Executing script SME/signal-post-upgrade-reboot on targetsvr failed; but targetsvr rebooted and folders restored.
options [--preserve-newer=no] [--delete=yes] NOT TESTED
OK - I'll take a look. Interesting as it must have run pre-restore correctly, but failed after. Maybe it did execute but doesn't correctly understand the response.
Let me know the conf settings please so we can vaguely replicate it.
dedup=no
I have a test binary build for dedup which I will import to contribs but my lappy here is messing me about so will do it when I get home next week.
https://www.reetspetit.com/smetest/7/repoview/freedup.html
Feel free to download and test.
FAIL affa --init-nrpe (never used this option but do use nrpe)
bkupsvr:/etc/nagios# affa --init-nrpe
sh: /etc/init.d/nrpe: No such file or directory
bkupsvr:/etc/nagios# systemctl status nrpe.service
nrpe.service - Nagios Remote Program Executor
--init-nrpe will need NRPE installed (part of Nagios)
I have no idea on the situation with that on v10 at the minute.
A couple of other options that cold be tested.
1. A normal Linux box.
Set conf
SMEServer=no
2. A windows box - I think you can do that with Rsycnd as there is some mentions in the code.
Set conf
SMEServer=no
rsyncdMode=yes
There are some Samba options in there too:
SambaShare
setupSamba
But the confs seem to go here:
my $affasmb = "/etc/samba/Affa-Job-$jobname.conf";
Any other variables in the Affa page.
This is the code but I haven't really figure it out yet - this gets called after setting up the watchdog. I think the Samba settings need to be put in the Global Conf but it doesn't say that in the Wiki page!
This is the setup section - if I have time I'll step through the code and try to figure it out.
sub setupSamba() {
lg("Configuring Samba");
if ( $smbdStatus ne 'enabled' ) {
lg("Samba service is not installed or not properly configured.");
return;
}
my %inc;
foreach my $jobname ( $cfg->Sections() ) {
next if $jobname eq 'GlobalAffaConfig';
my %job = getJobConfig($jobname);
if ( not $job{'_SambaValidUser'} and $job{'SambaShare'} eq 'yes' ) {
lg( "Job $jobname: No Valid Users defined. Samba has been access disabled."
);
}
my $affasmb = "/etc/samba/Affa-Job-$jobname.conf";
if ( $job{'SambaShare'} eq 'yes' and $job{'_SambaValidUser'} ) {
open( FD, ">$affasmb" )
or affaErrorExit("Could not create $affasmb");
print FD "[$jobname]\n";
print FD "path = $job{'RootDir'}/$jobname\n";
print FD "comment = Affa archive: $job{'Description'}\n";
print FD "valid users = $job{'_SambaValidUser'}\n";
print FD "force user = root\n";
print FD "read only = yes\n";
print FD "writable = no\n";
print FD
"veto files = /.$jobname-setup.ini/,/.doneDates/,/.AFFA3-REPORT/,/.AFFA-REPORT/.AFFA-TRASH/\n\n";
close(FD);
$inc{"include = $affasmb"} = 1;
All comments welcome!
-
I have some win clients with rsyncd running (for backups with BackupPC), I can give it a try.
Any documentation available on how the rsync option should work?
Ooohhhh - might be fun then.
Defo set 'SMEServer no' .
See the wiki page and this for a bit of guidance, but it is a bit of trial and error!
http://affa.sourceforge.net/AffaPdfMan.pdf
I'll try and figure a bit more out next week but I am trying to get some other stuff fixed too!!
-
And all, please document stuff so we can repeat/test.
It is VITALLY important.
Thanks.
-
I can confirm that I've been able to backup a directory of a windows machine via affa through rsyncd without particular issues.
-
Cool. Restore?
Can you detail that for us for the wiki?
-
For restore you'd need also a sshd server running on the windows machine, which I don't have.
I tried with Freesshd but I could not get the exchange of keys to work...
Of course I can share my notes on the setup used for rsyncd backup.
-
For restore you'd need also a sshd server running on the windows machine, which I don't have.
I tried with Freesshd but I could not get the exchange of keys to work...
OK - be nice to see if someone can get the restore to work somehow!!
Of course I can share my notes on the setup used for rsyncd backup.
If you can list them here - without passwords/server names - that would be good. You can also drop any notes in the SME Doc channel on Rocket and James will pick them up - he's cleaning up the page currently.
A note of your conf file etc too.
-
I sent my notes on the Rocket chat.
-
Haveing a go at getting a restore going..thank you for your work