Koozali.org: home of the SME Server

Affa for SME 10 and Nextcloud

Offline monoman

  • **
  • 45
  • +0/-0
Affa for SME 10 and Nextcloud
« on: June 24, 2022, 06:32:11 AM »
First up, big thank you to everyone involved in bringing affa to SME-10.  I've been watching contribs for an annopuncment almost daily.  I've been running affa for many years across many organisations and it has always served me well.

After a fair amount of frustration getting remote SSH keys setup, I now have my first backup done.

However, after an affa --rise --all --jobname I am unable to access nextcloud.  Can someone assist with this please.

Thanks 

Offline TerryF

  • grumpy old man
  • *
  • 1,821
  • +6/-0
Re: Affa for SME 10 and Nextcloud
« Reply #1 on: June 24, 2022, 06:39:45 AM »
Does affa backup the Nextcloud contrib? not sure see the wiki

https://wiki.koozali.org/Affa#SME_Specific_usage_and_command_line_options
--
qui scribit bis legit

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: Affa for SME 10 and Nextcloud
« Reply #2 on: June 24, 2022, 11:56:19 AM »
Does affa backup the Nextcloud contrib? not sure see the wiki

https://wiki.koozali.org/Affa#SME_Specific_usage_and_command_line_options

By default it won't.

At the least you would need /home/e-smith/files/nextcloud but it may require other stuff too - config entries, DB, passwords etc. That all needs careful checking and testing.

I'm not sure which DBs Affa backs up currently. If you are migrating to a newer Nextcloud + Maria 10.5 then it might not - I need to check and look at adding that if it is missing.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline monoman

  • **
  • 45
  • +0/-0
Re: Affa for SME 10 and Nextcloud
« Reply #3 on: June 24, 2022, 12:12:58 PM »
By default it won't.

At the least you would need /home/e-smith/files/nextcloud but it may require other stuff too - config entries, DB, passwords etc. That all needs careful checking and testing.

I'm not sure which DBs Affa backs up currently. If you are migrating to a newer Nextcloud + Maria 10.5 then it might not - I need to check and look at adding that if it is missing.

/home/e-smith/files/nextcloud is backed up by default, as is /home/e-smith/db/mysql/nextcloud.dump




« Last Edit: June 24, 2022, 12:16:21 PM by monoman »

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: Affa for SME 10 and Nextcloud
« Reply #4 on: June 24, 2022, 01:14:27 PM »
/home/e-smith/files/nextcloud is backed up by default, as is /home/e-smith/db/mysql/nextcloud.dump

Hmmm ok - must be backing up the whole /home/e-smith hierachrchy. But that's only the nextcloud data dir, and probably the default Maria DB, but may not include Maria 10.x if installed for newer versions of Nextcloud (see wiki). Jean Philippe will know.

There may be other bits of nextcloud - eg install/session dirs - that are required for a full rise.

Sorry - semi holiday today so on phone!!
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: Affa for SME 10 and Nextcloud
« Reply #5 on: June 24, 2022, 01:19:36 PM »
Hmmm - not sure if you'd need to reinstall the nextcloud contribs (probably) but then how that might conflict with the files in /usr/share/nextcloud which might be an upgrade from the original install rpms.

Definitely need to JP to have a think abut this one as it is quite complex due to the possible variations on installed versions.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline monoman

  • **
  • 45
  • +0/-0
Re: Affa for SME 10 and Nextcloud
« Reply #6 on: June 24, 2022, 01:59:58 PM »

Definitely need to JP to have a think abut this one as it is quite complex due to the possible variations on installed versions.

Prod server is running Nextcloud 21.0.9 whereas the affabox is on the default Install, 20.0.1

CORRECTION:  Our production server is running 20.1.14 and I have now updated to the same version on affabox
« Last Edit: June 25, 2022, 12:59:39 AM by monoman »

Offline Jean-Philippe Pialasse

  • *
  • 2,746
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: Affa for SME 10 and Nextcloud
« Reply #7 on: June 24, 2022, 03:39:05 PM »
you need to reinstall rpms. and it needs to be done ideally at the same moment on both servers.

issue would have been the same for wordpress on sme9 about rpm version being inferior to the manually updated. that’s why you should install the rpms in the first place not at the moment of an unexpected failure or the rise function loose all its purpose

as stated on the page at the very precise point Terry showed you YOU HAVE TO manage the rpm separated from the backup.


Affa should be able to backup any db making use of pre-backupmand post-restore events

You can not simply rsync the whole server and expecting it to work, notably with a situation where you might have 2 different server versions.

one question i have, is it aware of the sme backup list with its backup.d extended list?

Offline Gary Douglas

  • *
  • 68
  • +1/-0
Re: Affa for SME 10 and Nextcloud
« Reply #8 on: June 24, 2022, 06:39:50 PM »
I have done several SME9 -> SME10 AFFA --rise --all with nextcloud installed. First I made sure both the prodbox and affabox nextcloud versions were the same. In the affa config I only added; Include=/usr/share/nextcloud/config/config.php
Most went smoothly. Those that didn't were my mistake. Required an undo-rise then scrape nextcloud off the affabox, reinstall nextcloud up to the same version.
With SME10 I am using Mariadb105 and newer versions of nextcloud, and the affabox only has; Include=/usr/share/nextcloud/config/config.php  The nextcloud database is backed up during the pre-backup event. Can't recall if I have done an affa --rise --all from SME10 -> SME10 with nextcloud installed, it should work smoothly.


Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: Affa for SME 10 and Nextcloud
« Reply #9 on: June 24, 2022, 07:04:16 PM »
Code: [Select]
preJobCommand
preJobCommandRemote

postJobCommand
postJobCommandRemote

However, I'm not sure how that works in relation to 'rise' as the DBs like Maria 10.5 would likely need manual intervention.

A nicely documented 'how to' would be appreciated - we can add to the wiki.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: Affa for SME 10 and Nextcloud
« Reply #10 on: June 24, 2022, 08:26:03 PM »
With SME10 I am using Mariadb105 and newer versions of nextcloud, and the affabox only has; Include=/usr/share/nextcloud/config/config.php  The nextcloud database is backed up during the pre-backup event.

Do you add a specific code to backup the DB with Affa or is that done elsewhere by the system?

That's a key thing with newer versions of the DB and Nextcloud.

...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline monoman

  • **
  • 45
  • +0/-0
Re: Affa for SME 10 and Nextcloud
« Reply #11 on: June 25, 2022, 12:58:23 AM »
you need to reinstall rpms. and it needs to be done ideally at the same moment on both servers.

In the past, I have maintained my affa servers concurrently with my production servers to ensure all RPMS matched.  The exception being when setting up a new server and using affa to migrate to new hardware - which has been my preferred method by far.

With affa for sme-10 only just being released, concurrency was not an option when I migrated to sme-10 last December.

We use nextcloud, in part,  as a customer portal which currently has over a 1,000 unique shares that we can't afford to lose.  Would reinstalling rpms put these shares at risk?


 
« Last Edit: June 25, 2022, 01:32:49 AM by monoman »

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: Affa for SME 10 and Nextcloud
« Reply #12 on: June 25, 2022, 01:33:51 AM »
Numerous things could.

You really need a documented and *tested* recovery plan. The more imporyant the data the more time you should spend on your planning & testing. Same goes for any system, not just SME.

Likely that your data itself on the ibays would be ok. Non ibay data shpuld be in the nextcloud/data which is also backed up.

Nextcloud itself might be a bit recalcitrant with the newer Maria 10.x dbs. It will probably be fine, but you need the right series of steps to recover. You'll get that from testing.

NB I run automysqlbackup for DBs and they then get backed up with Affa as well.

But I haven't tested with a latest nextcloud/Maria 10.x myself.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline monoman

  • **
  • 45
  • +0/-0
Re: Affa for SME 10 and Nextcloud
« Reply #13 on: June 25, 2022, 01:48:53 AM »
Numerous things could.

...

Likely that your data itself on the ibays would be ok. Non ibay data shpuld be in the nextcloud/data which is also backed up.


I'm not concerned with losing any the customer portal data, just losing or corrupting the share links.  1,000 plus share links are way too many to recreate, share and record them in our db manually. 

Offline Gary Douglas

  • *
  • 68
  • +1/-0
Re: Affa for SME 10 and Nextcloud
« Reply #14 on: June 26, 2022, 09:17:38 AM »
Quote
Do you add a specific code to backup the DB with Affa or is that done elsewhere by the system?

No extra code added. The sme-server pre-backup event dumps the database/s, and Affa syncs to the archives.

SMEServer=yes
Watchdog=yes
RPMCheck=yes
RootDir=/var/affa
Include=/etc/dehydrated
Include=/etc/nagios/nrpe.cfg
Include=/usr/share/nextcloud/config/config.php
Include=/var/log/qmail
Include=/var/log/qpsmtpd
Include=/var/log/sqpsmtpd


affabox:/var/affa/prodsvr/scheduled.0/home/e-smith/db# ls -al
drwxr-x---  2 root  root             4096 Jun 26 07:15 mariadb105
drwxr-x---  3 root  root             4096 Jun 26 07:15 mysql

affabox:/var/affa/prodsvr/scheduled.0/home/e-smith/db/mariadb105# ls -al
-rw-r--r-- 1 root root   1386835 Jun 26 07:15 mysql.dump
-rw-r--r-- 1 root root 285408813 Jun 26 07:15 nextcloud.dump