Koozali.org: home of the SME Server

I'm setting up a mirror in Denmark!

Offline Denbert

  • *
  • 156
  • +0/-0
    • hegnstoften.net
I'm setting up a mirror in Denmark!
« on: May 08, 2009, 11:51:25 PM »
Hi there,

I'm setting up a mirror om my server with rsync.

Do we really need all the obsolete versions?

I'm running this command as suggested here http://wiki.contribs.org/Mirrors:

Code: [Select]
rsync -aHhimOSz --partial --timeout 300 --delay-updates --delete-after --delete --max-delete 1000 ibiblio.org::Linux/distributions/smeserver/ /home/smeserver/



/ Denbert
"Success is not final, failure is not fatal: it is the courage to continue that counts" - Sir Winston Churchill

Offline Denbert

  • *
  • 156
  • +0/-0
    • hegnstoften.net
Re: I'm setting up a mirror in Denmark!
« Reply #1 on: May 09, 2009, 10:21:31 AM »
Anyone got a solution to change timeout settings on ssh?

I'm using putty, and are trying to rsync the mirror on a remote mashine in my hosting center.

the rsync times out, and it's a bit strange, as a putty with "htop" can run for days whitout timeout.

I have tried to change timeout in my command line to this:

Code: [Select]
rsync -aHhimOSz --partial --timeout 500 --delay-updates --delete-after --delete --max-delete 1000 ibiblio.org::Linux/distributions/smeserver/ /home/smeserver/
/ Denbert
"Success is not final, failure is not fatal: it is the courage to continue that counts" - Sir Winston Churchill

Offline cactus

  • *
  • 4,880
  • +3/-0
    • http://www.snetram.nl
Re: I'm setting up a mirror in Denmark!
« Reply #2 on: May 09, 2009, 07:04:19 PM »
I'm setting up a mirror om my server with rsync.
Very nice, please report a bug in the bugtracker against the contribs.org website for it as well and please put a onter to it in this thread for future refference.

Do we really need all the obsolete versions?
Please ask this in the bug as well as I do not know how the routing is configured and if we can assign you for certain folders only.

It would be nice, if you could spare the ~12Gb, if not you should be able to exclude it by adding something like the following line to your rsync command:
Code: [Select]
--exclude=obsoletes/*
Be careful whose advice you buy, but be patient with those who supply it. Advice is a form of nostalgia, dispensing it is a way of fishing the past from the disposal, wiping it off, painting over the ugly parts and recycling it for more than its worth ~ Baz Luhrmann - Everybody's Free (To Wear Sunscreen)

Offline cactus

  • *
  • 4,880
  • +3/-0
    • http://www.snetram.nl
Re: I'm setting up a mirror in Denmark!
« Reply #3 on: May 09, 2009, 07:07:04 PM »
Anyone got a solution to change timeout settings on ssh?
I think the easiest solution is to configure your download job as a cronjob or if you really want to control it manually use screen. With this application you can start a terminal, execute a command, detach and log out and attach again after login. You should be able to install screen like this:
Code: [Select]
yum install screen
For more details on screen take a look here: http://www.gnu.org/software/screen/
Be careful whose advice you buy, but be patient with those who supply it. Advice is a form of nostalgia, dispensing it is a way of fishing the past from the disposal, wiping it off, painting over the ugly parts and recycling it for more than its worth ~ Baz Luhrmann - Everybody's Free (To Wear Sunscreen)

Offline Denbert

  • *
  • 156
  • +0/-0
    • hegnstoften.net
Re: I'm setting up a mirror in Denmark!
« Reply #4 on: May 09, 2009, 08:12:14 PM »
Thanks Cactus,

I'll try to skip the obsolete by adding the suggested syntax to the command line.

I'll look at the screen suggestion, and hopefully a mirror will be up and running within a week or so.

How often should the cron job be running, is once a week OK?

/ Denbert
"Success is not final, failure is not fatal: it is the courage to continue that counts" - Sir Winston Churchill

Offline cactus

  • *
  • 4,880
  • +3/-0
    • http://www.snetram.nl
Re: I'm setting up a mirror in Denmark!
« Reply #5 on: May 09, 2009, 08:14:48 PM »
How often should the cron job be running, is once a week OK?
No, first sync to ibiblio, in the mean time raise a bug. You will (most likely) receive instructions on what to do next in that bug. Normal sync frequencies are around very 3 hours IIRC.
Be careful whose advice you buy, but be patient with those who supply it. Advice is a form of nostalgia, dispensing it is a way of fishing the past from the disposal, wiping it off, painting over the ugly parts and recycling it for more than its worth ~ Baz Luhrmann - Everybody's Free (To Wear Sunscreen)

Offline Denbert

  • *
  • 156
  • +0/-0
    • hegnstoften.net
Re: I'm setting up a mirror in Denmark!
« Reply #6 on: May 09, 2009, 08:39:30 PM »
Very nice, please report a bug in the bugtracker against the contribs.org website for it as well and please put a onter to it in this thread for future refference.

Done = http://bugs.contribs.org/show_bug.cgi?id=5233

/ Denbert
"Success is not final, failure is not fatal: it is the courage to continue that counts" - Sir Winston Churchill

Offline Denbert

  • *
  • 156
  • +0/-0
    • hegnstoften.net
Re: I'm setting up a mirror in Denmark!
« Reply #7 on: May 09, 2009, 09:28:09 PM »
Sorry Cactus, thanks for the corrected bug report  :grin:
/ Denbert
"Success is not final, failure is not fatal: it is the courage to continue that counts" - Sir Winston Churchill

Offline cactus

  • *
  • 4,880
  • +3/-0
    • http://www.snetram.nl
Re: I'm setting up a mirror in Denmark!
« Reply #8 on: May 09, 2009, 09:29:26 PM »
Sorry Cactus, thanks for the corrected bug report  :grin:
No problem. Thanks for providing a mirror.
Be careful whose advice you buy, but be patient with those who supply it. Advice is a form of nostalgia, dispensing it is a way of fishing the past from the disposal, wiping it off, painting over the ugly parts and recycling it for more than its worth ~ Baz Luhrmann - Everybody's Free (To Wear Sunscreen)

Offline Denbert

  • *
  • 156
  • +0/-0
    • hegnstoften.net
Re: I'm setting up a mirror in Denmark!
« Reply #9 on: May 09, 2009, 09:40:24 PM »
How do i know when the mirror is in sync?

anyone got a suggestion for a cronjob?

/ Denbert
"Success is not final, failure is not fatal: it is the courage to continue that counts" - Sir Winston Churchill

Offline cactus

  • *
  • 4,880
  • +3/-0
    • http://www.snetram.nl
Re: I'm setting up a mirror in Denmark!
« Reply #10 on: May 09, 2009, 11:52:09 PM »
How do i know when the mirror is in sync?
Rsync would not download any files anymore.

anyone got a suggestion for a cronjob?
What is wrong with using the same as you used to download? Did you look at screen for your initial download? You can use cron to sync once you are in sync.
Be careful whose advice you buy, but be patient with those who supply it. Advice is a form of nostalgia, dispensing it is a way of fishing the past from the disposal, wiping it off, painting over the ugly parts and recycling it for more than its worth ~ Baz Luhrmann - Everybody's Free (To Wear Sunscreen)