Koozali.org: home of the SME Server

Affa

Offline sti

  • *
  • 29
  • +0/-0
Affa
« on: December 13, 2023, 01:43:48 PM »
I have a problem with an allmost full Backup Disk.

My fault...but it happend !

My question is whether there is a possibility tho move parts of a job to another Volume
or computer  without loosing backup history.

...lets say split a backup on a directory level. Just copying is not possible because of symbolic links.

thanks a lot for help
 

Offline Jean-Philippe Pialasse

  • *
  • 2,765
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: Affa
« Reply #1 on: December 13, 2023, 02:04:48 PM »
as long as /etc/fstab is not part of backup you could mount your new disk at an important folder level after rsync its content

eg
mkdir /mnt/sdd1
mount /dev/sdd1 /mnt/sdd1
rsync -arvH /home/e-smith/files/ibays/ /mnt/sdd1
# check all is good for you
/usr/bin/systemctl stop smbd httpd-e-smith
rm -rf /home/e-smith/files/ibays/*
mount /dev/sdd1 /home/e-smith/files/ibays/

/usr/bin/systemctl start smbd  httpd-e-smith

then add the needed line in fstab to mount on reboot. try using the drive partition  uuid  instead of sd* to avoid surprise latter. 




if you have an additional security layer with raid or lvm over raid  you must adapt what is described above. also example uses sdd, but you need to change it with you new drive.

Offline sti

  • *
  • 29
  • +0/-0
Re: Affa
« Reply #2 on: December 13, 2023, 04:07:12 PM »
Thanks Jean-Philippe,

there is bit of misunderstanding

It is not question of how to mount an additional disk..
As I said I want to split a backup.

Example:   14TB Disk allmost full                      new 14TB Disk

   <JOB> daily.0    Directory1...      move  to     <JOBnew> daily.0  Directory1...       
                           Directory2...      leave   
                           Directory3...      leave
              daily.1    Directory1...      move to                      daily.1  Directory1...
                           Directory2...      leave
                           Directory3...      leave
              daily.2
              etc.
              weekly.0 Directory1...      move to                      weekly.0 Directory1...                               
                           Directory2...      leave
                           Direktory3...      leave
              weekly.1 Directory1...      move to                      weekly.1 Directory1...
                           Directory2...
                           Direktory3...
              etc.

on the "old" disk I will get space back.

with simpy copying directories symbolic links will be lost or show to the wrong place I think.
Maybe I am wrong.










Offline Gary Douglas

  • *
  • 73
  • +1/-0
Re: Affa
« Reply #3 on: December 13, 2023, 06:49:26 PM »
John is the expert, but as far as I know affa job only has one RootDir
RootDir=/mnt/14TB2/affa
You can copy more quickly with rsync than with affa --move-archive, but 14TB will take a while
rsync -avHWhe "ssh -p 22222" --progress /mnt/10TB1/affa/realmsvr /mnt/14TB1/affa
rsync -avHWh /mnt/14TB2/affa/nnasvr /mnt/16TB1/affa
« Last Edit: December 14, 2023, 06:27:16 AM by Gary Douglas »

Offline Jean-Philippe Pialasse

  • *
  • 2,765
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: Affa
« Reply #4 on: December 14, 2023, 03:17:49 AM »
affa has multiple modes.  if you do not explain whiche mode you use, we just assume and i assumed you were with a rise mode, so one server is the 1:1 backup of the other. 


is you backup disk lvm? if yes you can simply add your new disk to the lvm and extend your partition. 

Offline ReetP

  • *
  • 3,740
  • +5/-0
Re: Affa
« Reply #5 on: December 14, 2023, 01:57:34 PM »
As far as hardlinking goes, I think this is your answer (as I am not a guru on that bit, or anything really!)

https://forums.koozali.org/index.php?topic=49341.0

Effectively files are deduplicated to save space.

The only way you are going to realistically do this as far as I can see is:

1. Remove some of your backups to reduce space
2. Create a LVM with multiple disks to create one volume but no redundancy
3. Get more/bigger storage - possibly run a RAID array that you can enlarge by adding disks?

I'd be tempted by a RAID array so you are not dependent on one huge disk - if that fails what is your strategy at that point?? Several smaller ones in an array are always more fault tolerant.

I have my main affa backup on a RAID 1 array but it is tiny compared to yours. If I was to increase capacity I'd get myself a storage server with several disks and create a decent array - Raid 6 or better. It will have some inbuilt protection for you. If you really have that much data to protect then you really need to start getting spendy to protect it properly.

Note you do not need to run LVM if you use a Hardware RAID card. Just format as say ext4 and you can then add drives and expand the 'drive' as required.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline mmccarn

  • *
  • 2,627
  • +10/-0
Re: Affa
« Reply #6 on: December 14, 2023, 02:02:57 PM »
What if you label the "old" disk "Archive" with the dates covered, put it on the shelf, and leave it?

Then switch to the new drive.

The first backup takes a long time, but after that you have recent backups on the new drive and older backups on the old drive...

You would want to do some testing w/ Affa to make sure you could restore from either drive.

Offline ReetP

  • *
  • 3,740
  • +5/-0
Re: Affa
« Reply #7 on: December 15, 2023, 10:16:07 AM »
What if you label the "old" disk "Archive" with the dates covered, put it on the shelf, and leave it?

Then switch to the new drive.

The first backup takes a long time, but after that you have recent backups on the new drive and older backups on the old drive...

You could do that which indeed means starting afresh.

You would have to mount the old drive and do some hacking about to recover old files but it should be possible.

Quote
You would want to do some testing w/ Affa to make sure you could restore from either drive.

Note that Affa 3 archives will not work easily with Affa 4 (which is still testing and the Alpha rpm in smedev has a bug which I need to fix - time is my worst enemy) because of the issues with the latest versions of rsync.

As discussed elsewhere, the old versions of rsync will work with relative URLs which Affa 3 uses, but the new version 4 will not. It needs complete URLs.

I believe there is a 'compatibility' mode that someone mentioned somewhere but can't remember where. It should not be relied on.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline Gary Douglas

  • *
  • 73
  • +1/-0
Re: Affa
« Reply #8 on: December 15, 2023, 10:57:49 AM »
link was here; https://bugs.koozali.org/show_bug.cgi?id=12165#c34

rsyncOptions=--no-human-readable --old-args

Quote
It should not be relied on.
« Last Edit: December 15, 2023, 11:30:38 AM by Gary Douglas »