I already installed FM-Server 5.5 a couple of times for our clients and it works fine. I didn´t used an ibay and the fm-server didn´t like to start automatically so you have to type fmserverd start or change init-files to let it autostart.
You have to change some user and group priviligies as well and the .conf file should be changed.
I use a special user "fmuser" and put the data into his home-account. If you need to grab a database-file, simply log-in your mac or win-PC with the authentification of that fmuser and change the files. Don´t forget to stop the server with fmserverd stop before changing anything.
If the client is using Filemaker, there is no way to use ODBC because filemaker doesn´t support it in the way it should be.
Anyway FMServer is far to expensive for what it is.