Koozali.org: home of the SME Server

Obsolete Releases => SME Server 8.x => Topic started by: mikey777 on December 20, 2012, 09:59:33 AM

Title: Help Needed to speed up write speeds
Post by: mikey777 on December 20, 2012, 09:59:33 AM
Hi I am a newbie at linux but was able to install SME 8.0 successfully. We are using it as a file server to run our dos based distribution system with about 80 users. The problem i am experiencing is when users all work at the same time the writes speeds drop tremendously. Is there a way to increase the speeds as we are heavily dependent on the system speed for invoicing and allocations. Please help




Title: Re: Help Needed to speed up write speeds
Post by: Stefano on December 20, 2012, 10:31:35 AM
Hi, welcome here

you should tell us more on your hw, your setup and define "our dos based distribution system"

if you really need fast storage, you should use HW raid 10 with sas hds..
Title: Re: Help Needed to speed up write speeds
Post by: elmarconi on December 20, 2012, 11:19:08 AM
I will make one wild guess...

Filelocking issues and many small writes on Samba. http://ogre.ikratko.com/archives/347

Title: Re: Help Needed to speed up write speeds
Post by: mmccarn on December 20, 2012, 12:23:36 PM
I once had an experience that convinced me that server hardware is tremendously important.

A client with about 50 users suffered a motherboard failure in their Dell PowerEdge server.  I moved the hard disks, hard disk controller, and network card into a workstation with a CPU twice as fast as the old server.  The temporary server worked GREAT - until the third user fired up their Foxpro database -- at which point the new user and the two previous users all slowed to a crawl.

I concluded at the time that the difference was the dual DMA channels in the PowerEdge motherboard (one bus for the NIC and another for the disk controller).  Admittedly, this server was running Novell Netware, not Linux.

With luck, you may find something in the samba performance tuning HOWTO:
http://www.samba.org/samba/docs/man/Samba-HOWTO-Collection/speed.html
Title: Re: Help Needed to speed up write speeds
Post by: dante5do on December 21, 2012, 12:39:02 AM
Check SAMBA LOG
run a top to see memory
online and see if connections are lost samba smbstatus

regards
Dante Aguirre Quezada
Title: Re: Help Needed to speed up write speeds
Post by: mikey777 on January 22, 2013, 10:53:28 AM
Hi all thanks for all the replies but unfortunately still having the same problem. I tried editing the smb.conf file but it says do not edit this file and that all manual changes will be overwritten when file is regenerated and it is unmodifiable. As far as my DOS distribution system we are a ladies fashion distributors and we have a dos system written in clarion whereby we allocate as well as invoice to our stores.My main lag in speed is when the allocaters allocate by size and color the system take forever to update the process. There are about 7 allocaters all working at the same time. Please assist me as I am going buts trying to figure this out. My reporting is fast but when i write to the server its slow.  :( :(
Title: Re: Help Needed to speed up write speeds
Post by: Jáder on January 22, 2013, 11:15:33 AM
So... let's start from beggining. Tell us what is your HW (hardware) config:
1) what is your CPU
2) how many HDDs, what type size and cache

3) try these commands:
a) hdparm -tT /dev/sda

Compare your HDD speed with mine:
Code: [Select]
[root@guepardo ~]# hdparm -tT /dev/sda

/dev/sda:
 Timing cached reads:   2868 MB in  2.00 seconds = 1433.04 MB/sec
 Timing buffered disk reads:  228 MB in  3.18 seconds =  71.80 MB/sec


b) this tell me about my first HDD: hdparm -i /dev/sda

Code: [Select]
[root@guepardo ~]#  hdparm -i /dev/sda

/dev/sda:

 Model=ST31000333AS                            , FwRev=CC1H    , SerialNo=            9TE1WCN1
 Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs RotSpdTol>.5% }
 RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=4
 BuffType=unknown, BuffSize=0kB, MaxMultSect=16, MultSect=?8?
 CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=268435455
 IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
 PIO modes:  pio0 pio1 pio2 pio3 pio4
 DMA modes:  mdma0 mdma1 mdma2
 UDMA modes: udma0 udma1 udma2
 AdvancedPM=no WriteCache=enabled
 Drive conforms to: unknown:  ATA/ATAPI-4 ATA/ATAPI-5 ATA/ATAPI-6 ATA/ATAPI-7

 * signifies the current active mode

[root@guepardo ~]#

If your HDD is not using DMA, you have a big speed problem.
If your HDD is not fast enought, you must add another (several?) and use RAID OR change type (from SATA to SAS or SSD).

How do you compare speed... what do you have BEFORE SME to run this DOS system ?
A Novell Network ? What version ?
Title: Re: Help Needed to speed up write speeds
Post by: elmarconi on January 22, 2013, 11:25:41 AM
Hi all thanks for all the replies but unfortunately still having the same problem. I tried editing the smb.conf file but it says do not edit this file and that all manual changes will be overwritten when file is regenerated and it is unmodifiable. As far as my DOS distribution system we are a ladies fashion distributors and we have a DOS system written in Clarion whereby we allocate as well as invoice to our stores.My main lag in speed is when the allocaters allocate by size and color the system take forever to update the process. There are about 7 allocaters all working at the same time. Please assist me as I am going buts trying to figure this out. My reporting is fast but when i write to the server its slow.  :( :(

Do you run the DOS-based (Clarion) application on a DOS-machine or on a Windows-machine?

--edit--

http://wiki.contribs.org/SME_Server:Documentation:Technical_Manual:Chapter3#Samba_per_i-bay_settings_.28smbd.29

db accounts setprop ibay_name OpLocks disabled
signal-event ibay-modify
Title: Re: Help Needed to speed up write speeds
Post by: mikey777 on January 22, 2013, 11:42:10 AM
Thanks again. Ok HW config is i7 sandy bridge processor with 2Tb HDD and 16Gig DDR3 ram. I amd running application on windows Pc's and yes before I has a novell server ver 3.1.2. Will post HDD test shortly .
Title: Re: Help Needed to speed up write speeds
Post by: mikey777 on January 22, 2013, 12:26:55 PM
OK HDD speed test results : 42 256mb : 2.00sec = 21149.20 MB/sec buffer 260mb in 3.01sec = 86.45 MB/sec . Hard drive seems fine . Any other ideas and it is using DMA
Title: Re: Help Needed to speed up write speeds
Post by: mikey777 on January 22, 2013, 12:33:00 PM
no ibays setup
Title: Re: Help Needed to speed up write speeds
Post by: elmarconi on January 22, 2013, 12:58:03 PM
no ibays setup

Submit a "db accounts show ibay_name" of the i-bay you run your software in.

Title: Re: Help Needed to speed up write speeds
Post by: Stefano on January 22, 2013, 01:04:58 PM
no ibays setup

so tell us how does your server works.. no ibays -> no shares
Title: Re: Help Needed to speed up write speeds
Post by: mikey777 on January 22, 2013, 01:42:21 PM
Sorry like i said i am a newbie. All i did was install SME server 8 and than from the workstations I mapped a drive to my server ip\primary\files and than copied my clarion database and exe's to it and thats how it has been working  :oops: :oops:
Title: Re: Help Needed to speed up write speeds
Post by: Stefano on January 22, 2013, 01:44:53 PM
so you are working in Primary..

Code: [Select]
db accounts show Primaty

I suggest you to create another ibay and move there your data and change drive mapping to reflect your new share.. then you'll change config only for your new ibay
Title: Re: Help Needed to speed up write speeds
Post by: mikey777 on January 22, 2013, 01:47:22 PM
Ok thanks let me try that will keep you updated . Fingers crosssed
Title: Re: Help Needed to speed up write speeds
Post by: elmarconi on January 22, 2013, 05:00:13 PM
Thanks again. Ok HW config is i7 sandy bridge processor with 2Tb HDD and 16Gig DDR3 ram. I am running application on windows PC's and yes before I has a novell server ver 3.1.2. Will post HDD test shortly.

Which Windows version?

Please be as extensive as possible in your reports.  That will help.

Title: Re: Help Needed to speed up write speeds
Post by: mikey777 on January 23, 2013, 01:35:58 PM
Sorry iam using windows 2000 sp4 and windows xp sp3 on workstations
Title: Re: Help Needed to speed up write speeds
Post by: Stefano on January 23, 2013, 02:12:13 PM
mmmhhh

question : which is the Role of SME?
Title: Re: Help Needed to speed up write speeds
Post by: mikey777 on January 24, 2013, 08:21:36 AM
Hi .It has basically replaced the role of my Novell server as a database server to run my dos application.
Title: Re: Help Needed to speed up write speeds
Post by: Stefano on January 24, 2013, 09:49:29 AM
Hi .It has basically replaced the role of my Novell server as a database server to run my dos application.

"database server" is another thing :)
you are using SME as a pure file server, and you are working from Primary ibay
please follow my suggestion and let us know
please be aware that running (dos) exes from a shared folder via smb is not a good idea and, finally, you should really start a migration path toward something different

Edit: typo
Title: Re: Help Needed to speed up write speeds
Post by: elmarconi on January 24, 2013, 11:15:07 AM
I can add my experience we had with an ERP system we use.
Windows front-end, Database back-end, file-system import/export/listing.

Write speed horrible. After tracing all traffic (using procmon.exe from sysinternals )  the application generated we concluded that the application was writing in small packages, one line at a time, with a file-open and a file-close after each line.
The strange thing was that on Windows shares (on another Workstation) this was not so much a problem, but running on Samba it slowed down to a crawl. Google: samba many small writes slow http://ogre.ikratko.com/archives/347

We demonstrated this the supplier, he changed the write routine so that it buffered and then flushed the write in one file-open/write/file-close. Problem solved.