Thanks for all the suggestions, I finally found it. There was a cracked trace on the Cat6 Panel board on pin 7/8. Thats why I was having so much trouble, I checked every piece of wiring but this is the first time in 10plus years I've had a cracked trace on a punch panel. While I was testing it, and pushing it in, it worked, but plug all the cat 6 cables in and it didn't. Go figure.
Still, gotta love SME Servers!