Koozali.org: home of the SME Server
Obsolete Releases => SME 8.x Contribs => Topic started by: adlaw on July 19, 2013, 12:13:01 PM
-
Hi,
I'm backuping one SME with affa (smeserver-affa-2.0.0-rc5) on a NFS share on a Linux Mint station (Mint 13 Maya, based on ubuntu 12.04 Precise Pangolin). But the problem is that affa doen't makes hardlinks, and the backup disk is quicly full as everyday all files are copied. And as I have almost 100 G of data, the backup takes a very long time !
I'm surprised, because I have another SME where affa is backuping on FreeNAS (NFS share also), and it's working well...
Here is the affa job config :
# db affa show
AffaGlobalDisable=no
DefaultAffaConfig=default
sendStatus=weekly
status=enabled
baksme=job
AutoUnmount=yes
AutomountDevice=192.168.1.53:/mnt/BAKSME/affa
AutomountOptions=-t nfs -o nolock
AutomountPoint=/var/affadevice
ConnectionCheckTimeout=120
Debug=yes
Description=SME backup on local station
DiskSpaceWarn=strict
EmailAddresses=admin@mydomain.local
Exclude[0]=
Include[0]=
RPMCheck=no
RetryAfter=600
RetryAttempts=3
RetryNotification=yes
RootDir=/var/affadevice
SMEServer=yes
SambaShare=no
TimeSchedule=2030
Watchdog=yes
chattyOnSuccess=0
dailyKeep=1
doneDaily=2013199
doneMonthly=201304
doneWeekly=201328
doneYearly=2013
localNice=0
monthlyKeep=0
postJobCommand=
preJobCommand=
remoteHostName=localhost
remoteNice=0
rsync--inplace=yes
rsync--modify-window=3600
rsyncCompress=no
rsyncTimeout=900
scheduledKeep=1
sshPort=22
status=enabled
weeklyKeep=1
yearlyKeep=0
Where could be the problem ?
-
maybe a different NFS settings on the target machine?
-
Hi Stefano,
Thank you for this so fast reply :-)
Yes, maybe some NFS settings. I'll re-read the man, but I think that I already did and I don't see which setting could prevent affa to make the harlinks... Any idea ?
-
yes.. take a look in the affa's logs ;-)
-
Ooops ! I thought it was a problem with NFS only, as I recently changed the backup station. Affa seems to make correctly the backups if we except the hardlinks... But in fact, in the logs I have failed chown on each file ! (why affa is not sending a mail to warn about this ?)
The newly created folder with job name is owned by nobody:nogroup, with 755 rights. But it has been created by affa, and the rights of the parent folder are 777...
How must I do so that affa creates the folder with the good rights and owner ? Is it a problem with affa settings, or NFS settings ?
-
I would start from here (https://www.google.it/search?client=ubuntu&channel=fs&q=nfs+can%27t+chown&ie=utf-8&oe=utf-8&gws_rd=cr&redir_esc=&ei=4lbpUYv3L9HxhQff5oAY)
HTH
-
Hi,
Thanks for your help, Stephano !
Seems that the solution was to add no_root_squash option in /etc/exports on the station. I also added the sync option. I followed this tuto (https://www.digitalocean.com/community/articles/how-to-set-up-an-nfs-mount-on-ubuntu-12-04).
Before that, anything created in /var/affadevice had (seen from SME) the nfsnobody:nfsnogroup owner, and chown was impossible (operation not permitted).
After, a new file or directory has root:root owner :-)
Now, I'll see what will happen with affa. SHould I do something to the already created archives, or just let affa do and replace them when it will be time (I have one scheduled and one daily archive) ?
-
Hi,
Now, affa runs for 25 minutes, and stops with an error. Here is a part of the log :
Sun Jul 21 20:30:07[21999]: Running rsync...
Sun Jul 21 20:30:07[21999]: Exec Cmd: /usr/bin/rsync --archive --stats --delete-during --ignore-errors --delete-excluded --relative --partial --modify-window=3600 --inplace --timeout=900 --numeric-ids --rsync-path='/usr/bin/rsync' --link-dest='/var/affadevice/svgsme/daily.0' "/home/e-smith" "/etc/e-smith/templates-custom" "/etc/e-smith/templates-user-custom" "/etc/ssh" "/root" "/etc/sudoers" "/etc/passwd" "/etc/shadow" "/etc/group" "/etc/gshadow" "/etc/samba/secrets.tdb" "/etc/samba/smbpasswd" /var/affadevice/svgsme/scheduled.running/
Sun Jul 21 20:55:01[21999]: Exec Out: rsync: writefd_unbuffered failed to write 7 bytes to socket [sender]: Broken pipe (32)
Sun Jul 21 20:55:01[21999]: Exec Out: [sender] io timeout after 30 seconds -- exiting
Sun Jul 21 20:55:01[21999]: Exec Out: rsync error: timeout in data send/receive (code 30) at io.c(140) [sender=3.0.9]
Sun Jul 21 20:55:01[21999]: Exec Out: 30
Sun Jul 21 20:55:01[21999]: Exec Out: rsync: writefd_unbuffered failed to write 316 bytes to socket [generator]: Broken pipe (32)
Sun Jul 21 20:55:01[21999]: Exec Out: exitstatus=rsync error: received SIGUSR1 (code 19) at main.c(1298) [generator=3.0.9]
Sun Jul 21 20:55:01[21999]: signaling post-backup event on localhost
Looks like a problem with the connection, but
- When I make other tests, the SME and the station communicate very well,
- Affa was not stopping on an error before I made the change in /etc/exports, and nothing else changed in the network
- Affa stops always at the same time, 25 mn after the begining
Did I do something wrong or missed something ? What can I do to solve that ?
Thanks.
-
Hi,
I tried to add also the no_subtree_check option in /etc/exports, but the result is the same... The timeout occurs after a variable time, between 20 to 65 minutes, and seems not to depend on what is to be backed up.
Any idea on how I could solve that, or at least what I could do or try to locate the origin of the problem ?
Thanks.
-
I don't think you can create hard links on an NFS share
-
adlaw
The timeout occurs after a variable time, between 20 to 65 minutes, and seems not to depend on what is to be backed up.
One possibility.
Similar issues with copying large files across network connections has been attributed to "flaky" NICs. Try a good quality NIC eg Intel which are usually well supported by Linux. Disable your onboard NIC & try a Intel plug in card.
-
Hi,
I don't think you can create hard links on an NFS share
Yes, it is possible. It works with my FreeNAS !
Disable your onboard NIC & try a Intel plug in card.
Thanks for this advice. But how do you explain that the timeout happens only with the no_root_squash option in /etc/exports ? I never had any timeout before I put this option, and the timeout disapears when I remove it...
-
adlaw
Thanks for this advice. But how do you explain that the timeout happens only with the no_root_squash option in /etc/exports ? I never had any timeout before I put this option, and the timeout disapears when I remove it...
I was not trying to explain or answer your problem directly. You asked:
Any idea on how I could solve that, or at least what I could do or try to locate the origin of the problem ?
so I suggested a troubleshooting approach, if you swap the NIC to a known good plug in card, then you may find whether the original problem is resolved or not, ie by process of elimination.
This is based on previous examples where large file copying operations stopped for no apparent reason, which was eventually traced to a "flaky" NIC.
-
Hi,
I suggested a troubleshooting approach [...] by process of elimination.
Yes, thanks :-)
But :
1 - Put and remove the no_root_squash option in /etc/exports makes the "broken pipe" appear/desappear,
2 - The installation is far, and I have only ssh access.
So, do you think that it's really interresting to go there just to change the NIC ? I'm afraid that I will go there just to confirm that the problem is not the NIC... I should prefer to have others tries that I can do through ssh before, and go there only if really necessary.
-
Hi,
Yes, it is possible. It works with my FreeNAS !
freenas is NOT linux and so you may likely have different versions/implementation of NFS with different startup parameters..
in any case I would say that the issue is not with SME.. it works with FreeNAS..
you should search on mint/ubuntu side
-
Hi,
freenas is NOT linux and so you may likely have different versions/implementation of NFS with different startup parameters..
Sure ! But probably, there is a way to do the same things on two POSIX systems... More especially hard links certainly work on NFS/Linux, as well as on LINUX and NFS/BSD...
in any case I would say that the issue is not with SME.. it works with FreeNAS..
you should search on mint/ubuntu side
It's true. I was just asking here because I know that some people are backuping their SME on a linux station, and was hoping they could help me, even if the issue is certainly not affa or SME.
Thanks for your help anyway :-)
-
Using the same NFS storage server, when moving from AFFA ver2 on SME8 to AFFA ver3 on SME9, the archives began using the full GB per archive.
Backup jobs would complete in the usual time, but storage was diminishing quickly.
When mounting the NFS share, SME9 server defaulted to nfsvers=4, whereas SME8 was using NFSVER 3.
I mounted the NFS server from SME9 using nfsvers=3, which has solved the issue.
# vim /etc/fstab
192.168.100.2:/affa /affa nfs rsize=32768,wsize=32768,nfsvers=3,timeo=600,intr,soft,bg 0 0
I have now migrated from SME8 / AFFAv2 to SME9 / AFFAv3, and retained all previous archives.
Using du -sh * from the SME9 (or SME8) shows the correct byte count per archive, confirming hardlinks are working.
-
Using the same NFS storage server, when moving from AFFA ver2 on SME8 to AFFA ver3 on SME9, the archives began using the full GB per archive.
Backup jobs would complete in the usual time, but storage was diminishing quickly.
When mounting the NFS share, SME9 server defaulted to nfsvers=4, whereas SME8 was using NFSVER 3.
I mounted the NFS server from SME9 using nfsvers=3, which has solved the issue.
# vim /etc/fstab
192.168.100.2:/affa /affa nfs rsize=32768,wsize=32768,nfsvers=3,timeo=600,intr,soft,bg 0 0
I have now migrated from SME8 / AFFAv2 to SME9 / AFFAv3, and retained all previous archives.
Using du -sh * from the SME9 (or SME8) shows the correct byte count per archive, confirming hardlinks are working.
Can that go in the Wiki, or possibly a a bug against the affa contrib ?
-
Can that go in the Wiki, or possibly a a bug against the affa contrib ?
in bugzilla, definitely.. it's a bug (unexpected behaviour) IMO
@gary: could you please open it? thank you
-
When mounting the NFS share, SME9 server defaulted to nfsvers=4, whereas SME8 was using NFSVER 3.
I mounted the NFS server from SME9 using nfsvers=3, which has solved the issue.
So it seems that you have found an NFS4 implementation issue in Linux Mint.
-
Charlie, there's no evidence that Gary Douglas is storing his data on a mint nfs storage, AFAICT
in any case, he said:
Using the same NFS storage server, when moving from AFFA ver2 on SME8 to AFFA ver3 on SME9, the archives began using the full GB per archive.
that means, IIUC, that affa3 changed its behaviour, giving unexpected results
-
Stefano
Charlie, there's no evidence that Gary Douglas is storing his data on a mint nfs storage....
I would say that Charlie got confused with the OP adlaw, who was using Linux Mint
-
Stefano, I filed a bug with the following detail, per your request, but it is not a bug.
This thread seemed to have gone unanswered, thought this may help others using AFFA to NFS storage.
I have completed my cutover today, final gotcha was excluding directories with spaces. Seems it requires a * instead of a space.
i.e; Exclude=/home/e-smith/files/users/rob/home/My*Documents/My*Music
-------------------------------------------------------------------------------------------
Bug 9186
Changing from SME8 AFFAv2 to SME9 AFFAv3 using the same NFS storage and taking over the AFFAv2 archives, caused all new archives to become the full archive size. i.e. hardlinks were not working.
I have run AFFAv2 on SME7/8 to NFS shared storage for many years without issues, originally from a SME7/8 server to FreeNAS. Since a few years ago my NFS Server is now a Proxmox/Debian with additional HP410/RAID5 storage shared using debian nfs-kernel-server. SME8/AFFAv2 and SME9/AFFAv3 are virtual on another proxmox box.
By default SME8 attaches to the NFS share using nfsvers=3 and hardlinks have never been a problem. SME9 by default connects to the same NFS share using nfsvers=4. The issue of hardlinks failing appears to be when using nfsvers=4. By setting SME9 to use nfsvers=3, hardlinks are maintained on the AFFA archives.
There is no bug with SME or AFFA (nor using NFS if configured correctly), bug only raised on request of Stefano.
-
Gary Douglas
I think this would have worked
Exclude="/home/e-smith/files/users/rob/home/My Documents/My Music"
(or with single inverted commas)
Exclude='/home/e-smith/files/users/rob/home/My Documents/My Music'
-
Stefano, I filed a bug with the following detail, per your request, but it is not a bug.
-------------------------------------------------------------------------------------------
Bug 9186
Changing from SME8 AFFAv2 to SME9 AFFAv3 using the same NFS storage and taking over the AFFAv2 archives, caused all new archives to become the full archive size. i.e. hardlinks were not working.
I have run AFFAv2 on SME7/8 to NFS shared storage for many years without issues, originally from a SME7/8 server to FreeNAS. Since a few years ago my NFS Server is now a Proxmox/Debian with additional HP410/RAID5 storage shared using debian nfs-kernel-server. SME8/AFFAv2 and SME9/AFFAv3 are virtual on another proxmox box.
By default SME8 attaches to the NFS share using nfsvers=3 and hardlinks have never been a problem. SME9 by default connects to the same NFS share using nfsvers=4. The issue of hardlinks failing appears to be when using nfsvers=4. By setting SME9 to use nfsvers=3, hardlinks are maintained on the AFFA archives.
There is no bug with SME or AFFA (nor using NFS if configured correctly), bug only raised on request of Stefano.
one of the main feature of Affa is the use of hardlinks.. maybe is not a "real" bug but just a documentation one
anyway, since the main rule here is "if something doesn't work as expected out of the box is likely a bug", a bug in bugzilla is needed
than you for your time
-
Janet, I have tried
Exclude="/home/e-smith/files/users/rob/home/My Documents/My Music"
and
Exclude='/home/e-smith/files/users/rob/home/My Documents/My Music'
but both started fetching files + folders from /My Music
AFFAv2 worked with DB command, and stored Exclusions correctly if you said; db affa setprop htsvr Exclude[0] /home/e-smith/files/users/rob/home/"My Documents"/"My Music"
but that fails also on my AFFAv3.
I can confirm Exclude=/home/e-smith/files/users/rob/home/My*Documents/My*Music works for Excluding folders with spaces
-
spaces in paths are never a good thing....
-
spaces in paths are never a good thing....
So sayeth The Oracle :lol: :lol:
Beers are on you mate.... FOSDEM here we come !
-
AFFAv2 worked with DB command, and stored Exclusions correctly if you said; db affa setprop htsvr Exclude[0] /home/e-smith/files/users/rob/home/"My Documents"/"My Music"
but that fails also on my AFFAv3.
I can confirm Exclude=/home/e-smith/files/users/rob/home/My*Documents/My*Music works for Excluding folders with spaces
Have you filed a bug so the AFFA developers can take a look?