Topic: [Announce] Addons.sh InstallScript for SME 7.1

[cool34000]

Hi everyone,


It's been over 1 hard year of work to do this script...
Back at the begining, I started using Linux with SME a year ago, using SME 6.01 release with smeplus.sh
That was the easiest and better way to learn for me!
A few months later, SME 7.0 was finally released and there weren't a lot of contribs, and no equivalent of smeplus.sh under SME 7.0
So I started learning bash and gathered rpms and howtos to make my own script for SME 7.0
I played with forums and searched a lot to finally release this script.

So here is Addons.sh, a 4000 lines script which will do for you:
- YUM Update your SME 7.1 to current version in 2 simple steps
- Download ~ 185 MB of contribs and dependencies
- Install contribs and setup everything for you
- Setup SME default configuration
- Send a complete mail to explain what's needed to start over
It supports 2 languages: EN and FR but it can be launched in any other SME native languages (EN will be default)

Here's the list of what's installed by default:

Code: [Select]

- auto file system check
- awstats
- dar
- mondo rescue
- crontab manager
- display config
- disk usage
- fuzzy ocr
- isoqlog
- spamassassin features
- fetchmail
- mxbackup
- vacation message
- mail sorting
- qmHandle
- password policy
- phpldapadmin
- phpmyadmin multi-user
- phpsysinfo
- sarg
- loginscript
- shell panel in server-manager
- samba status
- spamstats
- squid panel
- Lazy Admin tools
- rar support
- rpm install panel
- pdf printer
- lprng-monitor
- modindex support
- apache error customisation
- eaccelerator
- userpanel, userpanels and remote user access
Here's a list of optionnal installed componants:

Code: [Select]

- dansguardian
- DynDNS support
- egroupware
- phpbb forum
- groupoffice
- ocs reports and glpi
- mediawiki
- oscommerce
- sugar crm
- tomcat
- opensi (only available for FR)
- ntop
- phpwebsite
- popfile
- sme7admin
- snort
- teamspeak server
- torrentflux
- p2p blocking (only in server+gateway mode)
- thinclient + tftp server
Install should be painless with 4 easy steps.
Of course you don't need to install all the contribs (unless you need them all or want to test them on a test server)

Please read the mail that will be sent to you ('root') after the 3rd step.
All informations you need will be in this mail (and probably also here at contribs.org)
I will not answer to questions about how to use what's installed: you have forums, wikis, contributors websites, google, etc.
I will only try to answer to installation issues/errors.
If you have suggestions or improvements, you're also welcome

I will do my best to keep this script up to date, but the download section is under constant changes : if you experience errors during download you will need to search a bit and resolve the problem by yourself...


A special thanks to those who helped me:
- Berdie
- Gaston
- Jader
- Jibe
- MasterSleepy
- Sibsib
- VIP-Ire



Best regards,
Sylvain


[EDIT]As asked by the dev-team, the link to the script as been removed and the script is no more available.

[raem]

cool34000

Wow !
....But I'd hate to troubleshoot that server if it didn't run properly after all those upgrades at once !


> Here's the list of what's installed by default:

You left out 2 that I consider essential

qmHandle mail queue manager
smeserver-saco-qmHandle-1.3.1-1.noarch.rpm

Print queue manager
smeserver-lprng-monitor-0.0.7-1.noarch.rpm

Both from saco

Regards

[raem]

cool34000

Here's another essential

HTBWondershaper, although it's a script rather than a rpm.

[cool34000]

Thanks for your comments Ray.

You left out 2 that I consider essential

qmHandle mail queue manager
smeserver-saco-qmHandle-1.3.1-1.noarch.rpm

Print queue manager
smeserver-lprng-monitor-0.0.7-1.noarch.rpm

Both from saco

I consider them as essential too... But unfortunally, I wasn't able to download them using WGET from saco's website (is the website protected from direct download?)
So I skipped them.
I will look if these packages are now in smedev or somewhere else at contribs...

About HTBWondershaper, I didn't heard about that... Thanks for pointing me to it, I will give it a shot ASAP!

[bpivk]

But unfortunally, I wasn't able to download them using WGET from saco's website

http://www.saco-support.de/index.php?dload=Download&_m=downloads&_a=downloadfile&downloaditemid=3
http://www.saco-support.de/index.php?dload=Download&_m=downloads&_a=downloadfile&downloaditemid=2

Will this work (note: i can't access my server ATM so i can't confirm that it works with wget).

[raem]

cool34000

>...unfortunally, I wasn't able to download them using WGET from saco's website

You can request a contribs folder and upload the rpms to your own area using ftp, which will be permanent and then wget from there.
See mine at
http://mirror.contribs.org/smeserver/contribs/rmitchell/smeserver/contribs/

but I didn't create any of the rpms, just copied them there for a stable place to download from.

> About HTBWondershaper, I didn't heard about that... Thanks for pointing me to it

hanscees published great script a year or two ago for sme7. It really tames sme in regard to bandwidth management. You can still send big emails & surf and use VOIP without slowdown/interference, qmail is a big bandwidth hog when sending mail, so sme really needs this or Wondershaper only if you prefer that.

http://forums.contribs.org/index.php?topic=29495.0

http://forums.contribs.org/index.php?topic=29546.0

http://lartc.org/wondershaper/

[cool34000]

Hi,

No, these links don't work with WGET
Same problem as before, that's what I was talking about: the downloaded file is named "index.php?dload=Download&_m=downloads&_a=downloadfile&downloaditemid=3"

So, I'm gonna put these in my contribs area as suggested by Ray and update the script.
Thanks!


[Edit] Fixed, I added saco's RPMs (let mirrors synchronize!)
I also updated the original post.

[slords]

BIG WORD OF WARNING!!!

As with previous update scripts.  This will more then likely put your system in a state that will not be upgradable to 7.2 or 8.0.  We (the developers) went through a bunch of crap trying to ensure systems with update scripts would upgrade correctly.  We won't be doing this again!  You have been warned.

[CharlieBrady]

BIG WORD OF WARNING!!!

Let's make that:


BIG WORD OF WARNING!!!

As with previous update scripts.  This will more then likely put your system in a state that will not be upgradable to 7.2 or 8.0.

I'll add to that. This script will install many extra components, some of which may expose your system to remote compromise. The many extra components may also introduce bugs which the SME server developers do not wish to debug, and may not be able to identify and fix. Do not use this script, unless you are capable of debugging any problems in it, or are confident that you will obtain support from the developers of the script.

[cool34000]

Shad and Charlie,


With all due respect:
- The script is not for power admins, devellopers and SME gurus which should be able to do the same without a script (and a lot of time, but that's none of my business)
- The script is already compatible with SME 7.2 (you need to change 2 variables... But thank you!)
- Updates on a production server should ALWAYS be checked on a test server BEFORE applying them to a production server...

I would be curious to know what you think about people who installs manually a contrib and don't get a yum update after that because of the new RPM installed... Will you provide help to them?
Just curious because it happened to me twice (in one year wow, big deal!) with Dungog's RPMs. Hopefully there were help and fixes here on the forum and also on the bugtracker... That made me feel to be not alone... But thanks again!

OK, adding a contrib that was not tested by dev-team is dangerous...
Should we update the Wiki and write big warnings to each howto and RPM available there? Just tell me, I will love to update it!


All others problems I had were on YUM's fault with a default SME 7.1 install... Should I warn people not to use SME? Or should I blame the dev-team for that?

Thanks for the BIG WARNING anyway, that makes me feel that I've spent one fuc*ing year on it just for nothing and makes me quite disappointed.



For what it's worth...
Sylvain

[psoren]

Sylvian,

I wish someone had warned me when i installed loads and loads of great contribs to a bunch of servers running SME 6 !!!!!!

I have spend hours upgrading those to SME 7 because some contribs broke stuff or was not compatible with SME 7, and i was not smart enough to fix it myself. I have manually moved loads of mail and data, played around with permissons and ownership and so on, NOT funny. Having to explain to people  why it took so long and so much trouble when i told them it was a great software..... Not good.

Now i still use a lot of the great contribs i need, but only what i need. and i put everything i install into a directory so i know what i installed (if possible).

So does your script keep track of what's installed? could there be an uninstallscript as well? because that is probably the only chance to be able to upgrade later without issues.

So, big thanks to the guys for the warning and for the great work they do to have this unique server availible for all to use in the way we want. With or without contribs.

Thanks
Per

[raem]

Sylvain

> - The script is not for power admins, devellopers and SME gurus...

Part of the issue is that you have developed a script which makes it easy for less experienced people to easily install many different contribs, without really understanding/considering the potential ramifications of doing so.
eg They think "it must be OK to do so as it is a proper upgrade script & hey it's real easy to use too".

Some of these contribs may be compatible with sme and some less compatible.
The sme server concept is minimalism, don't install something if you don't really need it as it only introduces more possibilities of problems.

Theoretically, any compatible contribs should cause no problems, but bugs can still exist even in well supported compatible contribs.
It's mathematics, the more you have installed, the more possibility of introducing problems & bugs and as Charlie suggests, security issues.

Part of the direction of contribs.org was to vet contribs to see that they are fully compatible with sme server v7.x, and many of those on your list certainly have not been vetted, so therefore they are "unknown quantities".

Part of the issue is how well these various contribs are coded & tested ie they may work but have they undergone a proper security audit ?
The packages in the base server & approved updates are tested & well supported & compatible with other base packages, so upgrade paths are supported, and careful programming avoids creating security holes.

When a lot of "unknown" packages are installed the possibilities are endless, maybe nothing goes wrong, but maybe something does go wrong.

Part of the problem with the sme6 script was that it updated lots of individual package versions eg php mysql etc etc which did conflict with other base packages at upgrade time due to dependencies & template conflicts etc. It doesn't look like your script is specifically doing that  although I haven't fully reviewed it.

I think Charlies responses were tempered, there were a good few "may's" in there, and he said don't use your script unless you know how to troubleshoot potential problems, all quite sound advice....

Unless you intimately know all the coding in all those contribs, the only sensible approach is to be cautious when installing contribs, and if there is the slightest doubt, don't. Charlies advice has always been to review the code in any contrib before using it, and even then be careful.

Many packages & contribs have caused grief for end users & allowed thousands of servers to get hacked due to security issues, and that's big name products eg php, phpBB, awstats etc etc.

If I understand correctly, when you have a service agreement with Mitel to support your commercial Mitel sme server, you agree to install NO additional packages, that's so Mitel can ensure the integrity of the servers they support and not get caught up in bug & security & incompatibility issues created by well meaning but ignorant end users.

What's the right thing to do, I'm not sure, perhaps your script should be modified so that ALL the packages you suggest to be installed should be optional, and some BIG warnings given to users not to install if not absolutely needed.

[cool34000]

Psoren,


Well... If you had a look at what were inside smeplus.sh and also what was new on SME 7, you should have almost seen that ClamAV, SA and MySQL were included or updated and that it will lead you to some problems!
That's why I never even tried to update SME 6.x to SME 7.x!
In fact, to avoid problems, I never tryed to update any of my OS to the next major version (Windows or Linux) because it's always random!
I still can't understand why the dev-team spend time to make the update possible from 6.x to 7.x (and even less why they choosed to support smeplus.sh which were none of their concern)

What I've learned during this year of work on this script and with this great experience I have now with YUM (at least 1500 installs) is that things change all the time... Sometime it's better, but sometimes it's really not!
Don't believe me? Just hit yum+update+problem in the forum...
And I don't talk about RPMs that are included and then 1 month later no more needed as a dependency (but not uninstalled)

I'm already on an uninstall script since 3 months (thanks to WilliamSid who pointed me to that)
But to be honnest I don't know if I'm going to finish it now... Maybe I should do just like the others: make stuffs for my-self and don't share them with others.

Critical contribs (mostly webapps) are optionnal, but I was also already working on offering more choice on what's installed... To avoid such complains and to fit anyone's need. That's a lot of work to do... And again, I don't know if I will.


--------------------------------------------

Ray,


During this year, I've contacted a lot of contributors to help improve their RPMs when my little knowledge was enough to help them. So in this way, I think I've helped this community.
I've recently leaned howto make RPMs and I've made a few RPMs. I've tried to keep the SME spirit. I'm quite new to SME and Linux as I said... So I'm waiting for any comments to improve my RPMs...
All my work is here and everyone can download SRPMS to view what's inside and how it's installing. Anyone can feel free to test them and send their comments/advices or even update it on their own.
I've done it so many times before for others (just to see how it works or simply to learn and even try to search for improvements), I would be very happy if someone do the same for me.
It's the same for this script...

As an admin, I'm aware of potential problems/bugs/security issues that occurs when a lot of programs are installed. That's not only for SME, it's general.
That's why I've tested, tested and tested again this script (in all languages...) to avoid most problems and prevent from install problems.
That's also why a lot of contribs are not even available in my script.
A few contribs are modified by my script to allow better security (ie close access from the outside to phpsysinfo by default with a custom template)
Only what's has been tested is inside. That doesn't mean that it's perfect or bugless. But believe me, all I was able to do is in this script!

If I understand well how updates work, all packages which are in default repositories have been tested and approoved. Can we say that there are no bugs, no security issues? Well my experience don't tell me that!

I was not disappointed by the few "may's" but for the big warnings... It makes me think my work is really useless, a big sh*t and that I should delete it.

I don't know what to think about "you've installed non-standard contribs so we don't provide support"... I guess everyone using SME has at least one contrib installed?
And I hope contributors will not stop helping us (users) when a bug is found or when a new SME update needs a few modifications.
I would love to know what people would think if Microsoft was saying the same thing...

[cool34000]

By the way, here's the 1st messages you will see if you give this script a shot:

Code: [Select]

=============================================================
|      THIS SCRIPT IS PROVIDED AS IS WITH NO WARRANTY!      |
|            USE THIS SCRIPT  AT YOUR OWN RISKS!            |
=============================================================
|                                                           |
| Source informations:                                      |
|   - http://wiki.contribs.org/                                  |
|   - http://www.smeserver.org/                             |
|   - http://wiki.contribs.org/                             |
|   - http://bugs.contribs.org/                         |
|   - http://forums.ixus.fr/                                |
|   - http://www.dungog.net/                                |
|   - http://vanhees.cc/                                    |
|   - http://schirrms.net/                                  |
|   - http://sme.firewall-services.com/                     |
|   - http://hanscees.com/                                  |
|   - http://sudemo.info/                                   |
|   - http://www.vhconsult.com/                             |
|   - http://smeserver.fr/                                  |
|   - http://www.sonoracomm.com/                            |
|   - http://sme.swerts-knudsen.dk/                         |
|  and all that I've forgoten!                              |
=============================================================

 --> Type 'ok' to accept the disclaimer

[raem]

cool34000

> I still can't understand why the dev-team spend time to make the
> update possible from 6.x to 7.x (and even less why they choosed to
> support smeplus.sh which were none of their concern)

Because there were so many sme 6 servers out there and a good number that had used the smeplus script, and they were trying their best to make sme 7 upgrades work (at least) for everyone. They are the good guys remember.

Re yum & sme7.
Unfortunately there were some hiccups with the implementation of yum in sme7, but it was a totally new development (ie integrating yum & auto updates from Centos repositories etc etc), so of course bugs were expected, although not desired. It is a big project and there are very few developers. sme has a handful of developers, Microsoft & others have thousands of developers & a lot more money.


> I'm already on an uninstall script since 3 months (thanks to WilliamSid who pointed me to that)
> But to be honnest I don't know if I'm going to finish it now...

Your intentions have obviously been well meaning, and you have obviously done a lot of work & thought carefully about it, as far as I can tell.


>..... I've tested, tested and tested again this script...
> That's also why a lot of contribs are not even available in my script.
> A few contribs are modified by my script to allow better security


My quick read of the script indicates that you are not being "gung ho" about installing every upgrade for every package available, which is a good thing, but  rather it's a means of semi automatically installing a series of popular contribs, which people could install themselves on a one  by one basis anyway.

A uninstall script would be the piece that finishes the job, so to speak, and allows end users to undo what they've done with the install script, should the need arise.
I think you should finish the uninstall script, it will actually add to the value of the install script.

I also think you should make the installation of all contribs optional, and don't force all users to install every contrib on the first list of "obligatory"contribs.

Those two changes will make the conceptual usage of your script much better.


> Maybe I should do just like the others: make stuffs for my-self and don't share them with others.

When people criticise (or appear to criticise), one can feel that way, but don't be disheartened, I think the warnings are valid and should be heeded and taken on board, and you should use these comments to make your scripts better and in fact develop them in a way that will answer the posters concerns.
I think though that you feel their concerns have already been answered by you, in your earlier work on the scripts and researching contribs etc.

Don't be upset by what has been said, you asked for comments so take them all, good or less good.


> I'm quite new to SME and Linux as I said... So I'm waiting for any comments to improve my RPMs...

Work with Charlie, Shad, Gordon & other developers as they are very knowledgeable & experienced programmers.
Submit your work in the bugtracker for peer review, one rpm/contrib/script per bug, and have categories opened for any bugs with your particular rpms/contribs.


> I don't know what to think about "you've installed non-standard contribs so we don't provide support"...


That has always been sme policy. the developers are way too busy with the base code, to also provide FREE support for potentially hundreds of unsupported contribs, some of which the developers would not agree with implementing anyway.
Yes probably all of us have contribs installed and we are therefore on our own theoretically, but the more mainstream ones do have fair support in the forums & bugtracker. The further you move from popular contribs, the more you are "on you own" when things go wrong. You could say that we are all lucky to receive the free support we do.

[girkers]

cool34000,
 
I like what you have done and I for one will appreicate your work and effort.

As to the extra comments, whilst I feel the message is important to outline, I feel that the way it was said was more of the issue.

[cool34000]

Guys,

Don't be upset by what has been said, you asked for comments so take them all, good or less good.

First I'd like to appologize... That was the 2nd day that I was not sleeping working on my script. I shouldn't have been so upset.
After this big night, I feel a bit sorry about what I said/feel...
I've invested so much time in this script that I couldn't see the advices, and all I saw were the remarks I didn't want and even didn't expect.
I guess I was only expecting nice remarks...
Sorry, really sorry about that.

I think the warnings are valid and should be heeded and taken on board, and you should use these comments to make your scripts better and in fact develop them in a way that will answer the posters concerns.

Yes, warnings were (and still are) valid and I should have avoid or temperate them by telling my todo's list...
So here it is:
- SugarCRM : RPM needs to be rechecked (perms too lazy)
- PHPWebSite : RPM needs to be created
- More choice on what's installed (hard work because of the dependencies)
- Add the choice for eGroupWare version (v1.2 vs v1.4). Right now I'm installing v1.2.107
- Add Gallery RPM as an option
- Add Net2FTP RPM as an option
- Add Typo RPM as an option
- Need more testing on Snort RPM (I randomly get an error)
- Finish the uninstall script (I have to contact 2-3 contributors because a few contribs doesn't uninstall clean)
- Add HTBWonderShaper

Work with Charlie, Shad, Gordon & other developers as they are very knowledgeable & experienced programmers.

I think that I don't have necessary skills to help them... I don't know a lot of stuffs and to be honnest most of times I don't even understand what's said in the bugtracker or in the mailing list. As I said, 1 year ago I was ultra noob, and now I'm just a noob!
So I'm just helping others when I can... I'm really uncomfortable with the bugtracker: like I said I'm not a develloper: I'm just a simple guy/admin who likes to spend a lot of time reading forums, howtos and other stuffs like that.

we are all lucky to receive the free support we do.

Yes we are... And we should all say thanks everydays! Again, sorry...

[gregswallow]

As Charlie and Shad have already said, publishing an "update script" is a bad idea, when #1 it is not an "update", and #2 you are not involved with development and don't fully understand the consequences.  I know you call it an addon script, but it is perceived as an update.

Please voluntarily take it down from where you have published it and instead help write (and publish in the wiki) proper howto guides that include installation, upgrade, and uninstallation procedures for contribs that aren't packaged as rpms.  That would be much appreciated by everyone.  I know you meant well, but all the warnings in the world don't help.  

You have probably learned more than you think figuring out how to install and uninstall all these contribs.  Going from making scripts, to making rpms is not as hard as you'd think.  Join the devinfo mailing list and ask for help or suggestions if you'd like to try turning a script into an rpm for a particular contrib.  I promise the developers don't bite

[cool34000]

Hi,

Please voluntarily take it down

Done!
I will look in the Wiki and I will try to update it.
I've also suscribed to the mailing-list.

I promise the developers don't bite

I hope so, I'm not edible

[mmccarn]

cool3400 -  I, personally,  am very impressed with your constructive attitude.

It is encouraging to see someone willing to adapt to the instructions of the developers as you have done after committing so much time and effort to a project.

I feel obliged to acknowledge both your achievement and your exemplary attitude.

- Michael McCarn

[kruhm]

cool34000,

"things change all the time... "
you're on the right track & you're learning the hard way (as most of us do). Things do change all the time. This is why packages like YUM exist and important that admins use it.

The big message behind these 2 pages of posts is that we work better together as a team rather than working individually against each other.

Think how much further along we would be if your year efforts had been directed correctly in the bug tracker. The smedev & smeextras repos would be further along than they are.

"I think that I don't have necessary skills to help them... "
You do, just stick with it. I encourage you to join a team and get involved in the bug tracker.

[gregswallow]

The big message behind these 2 pages of posts is that we work better together as a team rather than working individually against each other.

Exactly.  I've been emailing back and forth with Sylvain (cool34000) and I think he's going to email Charlie and Shad as well and see how his skill and effort can best be directed.  I wasn't aware of all the work he's done other than just the addon script, and I apologized for kind of jumping the gun a bit.

Yes, the message here is all about cooperation and communication in development.  And the tools are in place.  Even if the core developers don't want to include your contribs in the base distribution, they can still be checked in to cvs and built automatically in the smedev repository:  http://mirror.contribs.org/smeserver/releases/7/smedev/i386/repodata/ - And there has been some discussion on a "package review" process - but lets discuss that on the devinfo mailing list.

Thanks,

[paulfung]

cool34000,

I have to say, I did the same thing you did too, not a 4300 lines one .sh file, but I have at lease 800 lines, but I never have the guts to release it...... as it will cause big troubles to many new comer, who is searching their way to get on to the "fast lane".

As a subscriber of the developer's list, I do understand why they are all worrying about the script.

I guess you can share the sh file with a "uninstall" script in the developer's list and make those experience guy to test those new function and consider, it will make some of our life easier, but not all the people in the forum.

I think, putting this "over powered" script in the public forum, many flood the forum with helping calls comming from the inexperienced users.  It is not easy to help them all ...... especially many of users here, are on their first touch with Linux, all those broken deps. in the future update or upgrade will give them a very bad experience to say “…Linux is a bad platform......”
(Look around with the questions in the forum; you will agree that many of them are new to Linux, even I myself are not a expert when sitting side by side with “old guys” here like CharlieBrady, one of the best helper I consider in this forum, take his advice, he is only trying to help us all.)

Anyway, cheer up, I guess we all welcome the work you did, may be just putting it with over stated words, when the big I said worry come in mind.

Share the work with the developer may help, but I do agree this forum is not a good place.

[allforone]

I am not a sme guru. I have been trying to get openx-change to work for a day now. I am going to give up and just go back to horde. So my question is how will you script work on a new install of 7.2 with the additional updates. then running your script.

[cool34000]

The script is no more available... And there wasn't an install of openx-change in it at any time.

I'm now working on a new version, just be patient.