According to the pfSense Hardware Sizing Guidance:
which I read to say "get a faster CPU" for your 100Mbps connection... 
The above chart considers multiple connections, this issue involves only 2 plus whatever is current at time of testing.
At the CL enter...
top or htop
That will help you determine if your CPU is taxed or overloaded.
iptraf
Will allow you to monitor eth1 traffic and will show the number of simultaneous connections thru eth1.
Don't be surprised if the list gets long.
You can monitor all traffic thru both eth0 & eth1 at the same time also.
You can open two shells and run both top and iptraf at the same time.
Should give you a clear picture as to where the problem lies.
Linux has many tools for diagnosing problems, always a good idea to become familiar with them and the info they provide.
Scenario 1 Gateway ModeSource Request > SME WEB IP (via SME's
external interface 100Mbit NIC connection 1) >>> Proxy SME's Gateway IP (via SME's
internal interface 100Mbit??? NIC connection 2)
1 connection thru SME's external interface 100Mbit NIC.
Max Bandwidth for the 1 connection thru that interface = 12.5MByte or there a bouts.
Scenario 2 Gateway or Server Only ModeSource Request > SME WEB IP (via SME's
external interface 100Mbit connection 1) >>> to Proxy Remote IP (via SME's
EXTERNAL 100Mbit connection 2)
2 connections thru SME's external interface 100Mbit NIC.
Max 100T NIC is 100Mbit = 12.5MByte
Which equals
12.5Mbyte / 2 = 6.25MByte
So the best you could possibly get is 6.25Mbyte and that's if you don't consider any thing else in the real networking world.
So let's correct it for the newly/recently presented real data of 11.4
11.4 / 2 = 5.7MBytes
Subtract some for the connection to the test server and some for the usual network comm stuff.
Let's say for round numbers 0.7Mbyte.....we now get....
5.7 - 0.7 = 5.0 MByte
5.0 MByte
The reported speeds slows down to 4-4,
5Mbyte.
I'd say, your in the ball ParK +/- a few bitty Bytes.
Works fine, till i put on the Proxy, then speeds slows down to 4-4,5Mbit
Quote
That's about right for server-only mode.
It will also be right in Gateway mode IF you proxy off the server to a remote IP addy.
1 inbound connection for Wan and 1 outbound connection to the remote IP thru the external interface nic.
2 connections thru the external interface nic.
Thus the bandwidth is approx 11.4 / 2 = 5.7MBytes
or
Gateway mode proxy to local gateway IP.
1 inbound connection for Wan and local proxy (routed to GW interface) no outbound external connection to the remote IP.
1 connections thru the external interface nic.
Thus the bandwidth is approx 11.4MBytes or max bandwidth available at the current time.
Surely it doesn't hurt to upgrade the system to a faster CPU to sustain additional proxy connections, however
the bottleneck will still be the 100Mbit nic if that is not upgraded.
My experience is years back 400MHZ CPU and 1000Mbit nic will handle 12-16 connections fairly well, the performance then
becomes a function of the Wan bandwidth and speed of the Harddisk.
Since you already have a gigabit router, a gigabit nic will be a lot less then a new system for sure.
Intel gigabit nic works very well with SME, the Intel server motherboard we use have daul Intel nic's built in.
At any one given time the server here has 10-80 connections, at this very moment 13 connections thru eth1 open and 1 passing constant packets
due to a remote shell connection.
If you place a large file i.e. an iso on your remote server and setup multiple downloads simultaneously you can monitor what is happening.
Understanding is in the mind of the beholder. YMMV
HTH