Topic: Nextcloud updates - NC24 and mariadb migration for existing installs

[Jean-Philippe Pialasse]

Nextcloud, do you use it ?

If not, please give it a try.

We have been improving your experience of it with SME. It allow you to work simultanously on one ibay from samba and from Nextcloud.

Your groups and users are synced using the SME integrated LDAP directory.

Last updates :
- migrate NC 24.0.2 for initial install
- fix gencache again [SME: 12096]
- fix ldap users/groups not availables on first run [SME: 12095]
- migrate property CloudDomain to VirtualHost [SME: 11918]
- use mariadb105 on new installs [SME: 11343]
- update rewriterules [SME: 11972]
- update to NC24 [SME: 12094]
  use a nextcloud-src rpm and avoid conflicts between web and rpm updates
- add to backup [SME: 12016]


If you use already nextcloud on SME 10, you might have found yourself stuck on NC20 because of the database, this has been solved, update to last smeserver-nextcloud version in smecontribs and then follow this procedure to migrate your db to mariadb 10.5 :  https://wiki.koozali.org/Nextcloud#Change_Database

We suggest you to update then up to NC 23, which will be supported until late 2022. Wait a little to upgrade to NC 24 as few extensions are still not ready.

[bclayton]

Thanks for this procedure!  I use Nextcloud and followed your instructions and everything worked well.

The only issue that my Nextcloud installation shows is this: SQLite is currently being used as the backend database. For larger installations we recommend that you switch to a different database backend. This is particularly recommended when using the desktop client for file synchronisation.

Can you tell me the best way to do that?

Thank you!

[Jean-Philippe Pialasse]

sqlite is neither the default we use in the contribs, neither a procedure we propsoed on the wiki,
if you really add a sqlite contrib at a time for your NC, then you will have to dive into NC forum to see how to migrate. This should be pretty similar to what is migrating from maria55 to maria105

what gives

Code: [Select]

occ config:system:get dbtype

[bclayton]

Thanks.

[root@provue-server ~]# occ config:system:get dbtype
sqlite3
[root@provue-server ~]#

Not sure this matters, but this server did have Owncloud installed before I upgraded to V10.

[Jean-Philippe Pialasse]

you need to migrate from sqlite to mariadb

https://docs.nextcloud.com/server/18/admin_manual/configuration_database/db_conversion.html


just not sure need before to create db and user with access

php occ db:convert-type --all-apps mysql nextcloud 127.0.0.1:/var/lib/mysql/mariadb105.sock nextcloud

[bclayton]

In looking at the Nextcloud documentation, it looks like a new db is needed:

"Run the conversion

First set up the new database, here called “new_db_name”. In Nextcloud root folder call

php occ db:convert-type [options] type username hostname database"

I don't want to bomb my system, so would that be the safe bet?  I do have reliable backups, just in case.

[Jean-Philippe Pialasse]

the command i post it in last message.

you have confirmation that you need to create the db. you can use the part of the migration to mariadb105 in the wiki page to create user and db. 

you need the option all app.

make a backup of your sql db and your config.php before proceeding.   

this is indeed always a risk that something get wrong

if you do not inow where is your sql db they search for the occ command to get the db path. 

[bclayton]

Thanks, JP.

I'll give this a shot.

[monoman]

Are there any gotchas I need to look our for upgrading a largish Nextcloud install of 300Gb of data in hundreds of shared customer folders?  I am somewhat concerned I may stuff this up.

Just to be clear I am going about this the right way, these are the steps as I understand them.

1. Backup SME server
2. Upgrade to smeserver-nextcloud.noarch  1.2.0-17.el7.sme  via yum
3. Follow the CHANGE DATABASE steps in https://wiki.koozali.org/Nextcloud#Change_Database
4. Incrementally upgrade NC from 20.0.14 to 21.0.9 to 22.2.10 to 23.0.8
5. Test and be happy

Thanks

One last thing.  Why is this called NC24 when we are only updating to 23?

[Jean-Philippe Pialasse]

correct for the steps

the most important thing is to backup your db and your /usr/share/nextcloud

user data is in /home/e-smith/files/nextcloud and is safe from the changes.

the two things you could mess up are really db and software dir.
use a
cp -a /usr/share/nextcloud
or a
tar -czf nextcloud.tgz /usr/share/nextcloud


for nc24, the new rpm nextcloud-src has been built against nc24 sources before i saw that a lot of modules were not yet available. too much work to undo, while people with nc already installed will have to manually update and could stop before (at nc23). the new rpm will install nc24 only on new installs. and even if you are doing a new install you could still manually put a tgz of nc23 in nextcloud-src folder

the new rpm will also prevent conflict with rpm overwriting the software folder on update. i had to conflict with error in pre scriptlet to avoid this before.
here the nextcloud-src will only rewrite the app folder if it is empty. so on migration one must took care to backup its config file out and only import it after nextcloud is running. at least for the moment.

[monoman]

It took around an hour to run.  The entire upgrade process ran without issue.

Thank you JP for helping clarify this for me.

[Jean-Philippe Pialasse]

Great news to read !

all the best to you !

[pat]

And for me it's the same !
All work fine and collabora is Ok
Thank you JP for this great job !

[monoman]

We suggest you to update then up to NC 23, which will be supported until late 2022. Wait a little to upgrade to NC 24 as few extensions are still not ready.

Is it safe to upgrade to 24.0.5 if nextcloud Security & setup warnings report All checks passed?

[TerryF]

Can only report my experinces, two prod systems now updated without issue, both operating with 24.0.5

[Jean-Philippe Pialasse]

Is it safe to upgrade to 24.0.5 if nextcloud Security & setup warnings report All checks passed?

security no issue.
only issue, is to first check that all needed extension are available for you before proceeding !
if you only use core just go, if you heavily rely on a lot of modules check the pre update messages about plugins...

[Randall]

Last night I applied SME updates via UI, and this morning I discovered that NextCloud (still on 20.0.1-1.el7.sme) is now broken - <my server>/nextcloud now appears to point to default web page (Under Construction). Could that be related to this, or is this some new issue ? Thanks

[Jean-Philippe Pialasse]

broken? wait a little before this conclusion.

i would rather say not displaying what you expect. thus not accessible would be more appropriate.


what returns

Code: [Select]

/sbin/e-smith/audittools/templates | grep http

Code: [Select]

config show nextcloud |egrep -vi 'pass|secre'
have you configured a dedicated domain?

have tried

Code: [Select]

signal-event nextcloud-update

[Randall]

Thanks for the reply. Broken being the technical term to say it's not functioning correctly

No, I haven't set up a dedicated domain, just the stock install. I have been having issues with the www group being removed from the shared group (https://forums.koozali.org/index.php/topic,54851.15.html), so not sure if that's related.

Here's the output of the suggested commands:
[root@e-smith ~]# /sbin/e-smith/audittools/templates | grep http
[root@e-smith ~]# config show nextcloud |egrep -vi 'pass|secre'
nextcloud=service
    AdminUser=nextcloudadmin
    DbName=nextcloud
    DbUser=nextcloud
    TrustedDomains=
    access=public
    status=enabled
[root@e-smith ~]# signal-event nextcloud-update

The signal event didn't seem to help (tried reboot after as well, no change).

Somehow nextcloud appears to be directed to default doc. Not familiar with how this is supposed to work ...

[monoman]

broken? wait a little before this conclusion.

i would rather say not displaying what you expect. thus not accessible would be more appropriate.


what returns

Code: [Select]

/sbin/e-smith/audittools/templates | grep http

Code: [Select]

config show nextcloud |egrep -vi 'pass|secre'
have you configured a dedicated domain?

have tried

Code: [Select]

signal-event nextcloud-update

Randall's issues are almost an exact replica of my issue with the exception that I am running a dedicated domain.  It was at the stage Randall is at that I was getting a 404.

Code: [Select]

[root@mimime ~]# config show nextcloud |egrep -vi 'pass|secre'
nextcloud=service
    AdminUser=nextcloudadmin
    DbName=nextcloud
    DbUser=nextcloud
    IncludeIbay=distribution,distrofiles,jono-artwork,orders,printing,sol-shared
    IncludeIbays=distribution,distrofiles,jono-artwork,orders,printing,sol-shared
    MemoryLimit=4G
    TrustedDomains=www.solportal.com.au,solportal.com.au
    access=public
    status=enabled
I then tried upgrading the nextcloud contrib via signal-event nextcloud-update.  After that, I am getting the update required nextcloud screen, which I am being told is a nextcloud bug, not something caused by the SME upgrades

[Randall]

Thanks for that. So it seems likely that upgrading NextCloud will fix it ...

[monoman]

Thanks for that. So it seems likely that upgrading NextCloud will fix it ...

That has not been my experience.

[Jean-Philippe Pialasse]

@monoman those are likely two different situations.

On yours you had a 404, then a 403 error, which were one kind of error against httpd, and I am still waiting for you to show the logs so I can find a way to reproduce this step.
Then you are now on a state were you have a db that prevent you to update from one version to another because some field are missing on some table and some other table are present while they should not.
Please keep on your initial post.



@Randall,
you also have an issue right now with httpd access, this is not nextcloud issue, this is SME side. So do not try to fix nextcloud you just do not have access to it but your ibay instead.

I would need two things to help you from there :
content of /var/log/messages* during the signal-event temp right after the update of smeserver-nextcloud.

current /etc/httpd/conf/httpd.conf

you could send it to  security@lists.contribs.org


you could also report here

Code: [Select]

occ status
looking forward

[Randall]

I have just forwarded log, extending from just before reconfig to present, and config as requested (I got a notice that it Is being held until the list moderator can review it for approval). I notice there a some jumps in the log time - I think I may have a hardware clock that needs to be updated. You'll also see numerous reboots and reconfigs, and I've tried various things (including fixing the shared group to include www).

I should also mention that I have made no effort to ugrade nextcloud itself - it is still at 20.0.1.

[Jean-Philippe Pialasse]

I see in messages

Code: [Select]

Sep 30 23:38:51 e-smith esmith::event[3798]: Processing event: temp#no nextcloud update !

Code: [Select]

Sep 30 23:41:43 e-smith esmith::event[5562]: Processing event: post-upgrade
Sep 30 23:43:09 e-smith esmith::event[8669]: Processing event: reboot
Sep 30 23:44:12 e-smith esmith::event[686]: Processing event: bootstrap-console-save
Oct  1 00:01:05 e-smith esmith::event[3844]: Processing event: reboot
Oct  1 10:10:19 e-smith esmith::event[8572]: Processing event: post-upgrade
Oct  1 10:12:22 e-smith esmith::event[11852]: Processing event: reboot
Oct  1 10:14:49 e-smith esmith::event[681]: Processing event: bootstrap-console-save
Oct  1 14:42:59 e-smith esmith::event[3205]: Processing event: post-upgrade
Oct  1 14:44:54 e-smith esmith::event[6485]: Processing event: reboot
Oct  1 14:47:17 e-smith esmith::event[682]: Processing event: bootstrap-console-save
Oct  1 14:56:00 e-smith esmith::event[3783]: Processing event: reboot


in term of warning /errors, only issue I see not related is
Oct  1 10:14:48 e-smith initializedate: Exiting, name server cannot be used: Temporary failure in name resolution (-3) 1 Oct 10:14:48 ntpdate[688]: name server cannot be used: Temporary failure in name resolution (-3)

I never saw any smeserver-nextcloud.
I do not see any contribs update.  There is only one temp event in the whole log
I would say you might not have the smecontribs enabled, and you still have a very old version not compatible with the new  apache.
See alert about it on the wiki release page and in the SME 10 forum.

Code: [Select]

rpm -q smeserver-nextcloud
should show 1.2.0-19

looking at your httpd, I can confirm you do not use the compatible smeserver-nextcloud version.


you should keep enabled smecontribs.

[Randall]

You were correct. Contribs updates were disabled (I have never touched these settings, so maybe a remnant of upgrade from some old SME version). I enabled it, and then issued
yum update --enablerepo=smecontribs
signal-event post-upgrade;  signal-event reboot
(needed to fix shared group membership, because www is always removed after reconfig for some unknown reason, and reboot again)

It's a bit slow to start, but NextCloud now appears to be working. Eureka !
Once again, thanks again for your prompt help. I'm a bit of a dummy when it comes to administering some of this stuff, because during probably 15 years or more of using it, I rarely have to go beyond UI point and click - a real testament to the usability and stability of SME !

[Jean-Philippe Pialasse]

great news!

can you list your contribs to see which one might play on the www/shared thing?

[Randall]

Sure. Here's the full list:

Code: [Select]

GeoIP.x86_64                           1.6.12-9.el7.sme        @smecontribs     
GeoIP-GeoLite-data.noarch              2018.06-7.el7.sme       @smecontribs     
GeoIP-GeoLite-data-extra.noarch        2018.06-7.el7.sme       @smecontribs     
ddclient.noarch                        3.9.1-2.el7.sme         @smecontribs     
jq.x86_64                              1.6-2.el7               @smecontribs     
nextcloud.noarch                       20.0.1-1.el7.sme        @smecontribs     
nextcloud-src.noarch                   24.0.2-3.el7.sme        @smecontribs     
oniguruma.x86_64                       6.8.2-2.el7             @smecontribs     
perl-Array-Compare.noarch              3.0.0-1.of.el7          @smecontribs     
perl-B-Hooks-OP-Check.x86_64           0.22-1.of.el7           @smecontribs     
perl-Class-Method-Modifiers.noarch     2.13-1.of.el7           @smecontribs     
perl-Class-XSAccessor.x86_64           1.19-2.el7              @smecontribs     
perl-Data-Validate-IP.noarch           0.27-13.el7             @smecontribs     
perl-Devel-GlobalDestruction.noarch    0.14-1.of.el7           @smecontribs     
perl-Lexical-SealRequireHints.x86_64   0.011-1.of.el7          @smecontribs     
perl-Module-Runtime.noarch             0.016-1.of.el7          @smecontribs     
perl-Moo.noarch                        2.004004-2.of.el7       @smecontribs     
perl-Params-Classify.x86_64            0.013-7.el7             @smecontribs     
perl-Role-Tiny.noarch                  2.001004-1.of.el7       @smecontribs     
perl-Sub-Exporter-Progressive.noarch   0.001013-1.of.el7       @smecontribs     
perl-Sub-Name.x86_64                   0.26-1.of.el7           @smecontribs     
perl-Sub-Quote.noarch                  2.006006-1.of.el7       @smecontribs     
perl-bareword-filehandles.x86_64       0.007-1.of.el7          @smecontribs     
perl-indirect.x86_64                   0.39-1.of.el7           @smecontribs     
perl-multidimensional.x86_64           0.014-1.of.el7          @smecontribs     
perl-strictures.noarch                 2.000006-1.of.el7       @smecontribs     
rh-mariadb105.x86_64                   3.7-1.el7               @smecontribs     
rh-mariadb105-mariadb.x86_64           3:10.5.9-2.el7          @smecontribs     
rh-mariadb105-mariadb-common.x86_64    3:10.5.9-2.el7          @smecontribs     
rh-mariadb105-mariadb-config.x86_64    3:10.5.9-2.el7          @smecontribs     
rh-mariadb105-mariadb-errmsg.x86_64    3:10.5.9-2.el7          @smecontribs     
rh-mariadb105-mariadb-libs.x86_64      3:10.5.9-2.el7          @smecontribs     
rh-mariadb105-mariadb-server.x86_64    3:10.5.9-2.el7          @smecontribs     
rh-mariadb105-mariadb-server-utils.x86_64
                                       3:10.5.9-2.el7          @smecontribs     
rh-mariadb105-runtime.x86_64           3.7-1.el7               @smecontribs     
smeserver-ddclient.noarch              1.3.0-23.el7.sme        @smecontribs     
smeserver-dhcp-dns.noarch              1.2.0-5.el7.sme         @smecontribs     
smeserver-mariadb105.noarch            2.7.0-16.el7.sme        @smecontribs     
smeserver-nextcloud.noarch             1.2.0-19.el7.sme        @smecontribs     
smeserver-wsdd.noarch                  0.2-5.el7.sme           @smecontribs     
wsdd.noarch                            0.7.0-1.el7             @smecontribs