Koozali.org: home of the SME Server

Dirty Tools

Offline kryptos

  • ****
  • 245
  • +0/-0
Dirty Tools
« on: June 04, 2021, 08:23:01 AM »
Hello, does some here tried dirty tools dt-data-copy? I want to synchronize between two servers in realtime. Since the location of the two servers are miles away and I want sync the data between them. Were server A files will be copied to server B and vice versa when Server B modifies the same file it will copied also to serve A.  Is this possible? 

Online ReetP

  • *
  • 3,722
  • +5/-0
Re: Dirty Tools
« Reply #1 on: June 04, 2021, 10:10:28 AM »
Never even knew this was there.

https://wiki.koozali.org/Dirty_Tools

Worth a look Brian/JP?

Not sure this what you need though.

So, real time sync is not easy. It is an issue some of the finest IT brains on the planet have struggled with!! File/record locking to prevent overwriting is difficult.

Think about this.

User at location A edits File A

At the same time:

User at location B edits File A

Now we have a conflict. Which one is right?

I have run 2 sites for 14 years using Unison. (I ought to do a Howto).

It is a clever wrapper around rsync.

https://www.cis.upenn.edu/~bcpierce/unison/

It can't solve the above problem, but for most use it is fine. You just have to check the conflicts.

My ibays run on a cron every 20 minutes via a script to make sure a job is not already running.

If you are going to have lots of conflicts you need to look at something else.

If you want a full real time sync you need to look for a different solution entirely...

Have a search online for "linux real time file sync"

https://duckduckgo.com/?q=linux+real+time+file+sync&t=fpas&ia=qa
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline kryptos

  • ****
  • 245
  • +0/-0
Re: Dirty Tools
« Reply #2 on: June 04, 2021, 10:49:27 AM »
Never even knew this was there.

https://wiki.koozali.org/Dirty_Tools

Worth a look Brian/JP?

Not sure this what you need though.

So, real time sync is not easy. It is an issue some of the finest IT brains on the planet have struggled with!! File/record locking to prevent overwriting is difficult.

Think about this.

User at location A edits File A

At the same time:

User at location B edits File A

Now we have a conflict. Which one is right?

I have run 2 sites for 14 years using Unison. (I ought to do a Howto).

It is a clever wrapper around rsync.

https://www.cis.upenn.edu/~bcpierce/unison/

It can't solve the above problem, but for most use it is fine. You just have to check the conflicts.

My ibays run on a cron every 20 minutes via a script to make sure a job is not already running.

If you are going to have lots of conflicts you need to look at something else.

If you want a full real time sync you need to look for a different solution entirely...

Have a search online for "linux real time file sync"

https://duckduckgo.com/?q=linux+real+time+file+sync&t=fpas&ia=qa

Yes  I was referring to that wiki I was wondering someone has tried it also. Seems what I need for my need just want some feedback from others who tried it. I don't have high transaction volume activities on my file server so I wont worry much about oplocks or conflicts in realtime.We are only few users. Just want same copies on both server

I look into Unison also looks great. Thanks!

Offline Jean-Philippe Pialasse

  • *
  • 2,743
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: Dirty Tools
« Reply #3 on: June 04, 2021, 09:31:52 PM »
dirty tool like unisson is kind of a rsync wrapper. 


i have imported it to smedev for sme10 if you want to give a try and report.  SME 9 is deprecated so will not build for it now. be careful it has not been maintained for 12 years.  it is probable that the part for user and db sync is not working with recent changes in sme core.  For just the data sync i think it should be ok.

Have you think about nextcloud?

you could use the command line client on the less populate site.

you could also install the sync client on the client desktop.
Nextcloud handle the conflict similarly to unisson by creating another file. it handles versioning and trashbin restauration.

Offline TerryF

  • grumpy old man
  • *
  • 1,821
  • +6/-0
Re: Dirty Tools
« Reply #4 on: June 05, 2021, 01:36:27 AM »
Nextcloud, thinking outside of the square there JP..nice
--
qui scribit bis legit

Online ReetP

  • *
  • 3,722
  • +5/-0
Re: Dirty Tools
« Reply #5 on: June 05, 2021, 09:52:46 AM »
Nextcloud, thinking outside of the square there JP..nice

It's good, but doesn't work in all scenarios.

We needed fast access to a lot of big files.

When I started there was no nextcloud/sync and I didn't want all the files individually on everyones desktops all trying to cloud sync (over horrendously slow adsl).

So two servers + unison did the job.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation