Topic: Reverse proxy

[EdelingF]

I'm finally building my new server by using a XPC Barebone Shuttle SH55J2 (Intel Core i3, 4 Gb int., 1Tb HDD) Works great!
First wanted to install SME8 with virtualisation in it, but I have now installed Proxmox with 4 virtual SME8b6 servers. Works also great!
Installed SME SiteMaker, no problems; installed Wordpress, no problems, installed phpMyAdmin, also no problems.

But the only problem is to get connect the servers to the internet. I mean, to make the servers 'visisble' from outside my network. I thought, no problem, just make some changes in the modem/router like I did with my old server by using port-forwarding. Wrong!

I found out I need something like a reverse proxy. So, my plan was to use one of the virtual servers as a (easy to manage) proxy server. But after a week looking in forums, wiki's and website I'm still no further on how I should do that.
Is there someone with experience using SME8 as a reverse proxy? I have read in the forum that some SME users use Proxmox for virtualisation, so they probably have had the same problems.

Also, is there a easy configuarable proxy server available for SME8, preferably to be anaged through a browser?

[larieu]

I use proxmox to one machine (mostly for testing purposes) with several sme virtual machines

But I have also one router which can handle fully routing needs and a separate connection for it
because is only for testing I set everything like this

           Internet
               |
router with internet access
               |
               |
Intermediate LAN With NAT provided by router (or if you can ask your ISP to provide several public IP's... - one class of 8 will be like charm)
               |
               |
Proxmox server also with virtual machines (various)
               |
               |
        -------------
       |     |     |     |
local LAN machines


for example you could set one environment like this
Intermediate LAN on router side
192.168.1.254/24
192.168.2.254/24
192.168.3.254/24
.....


proxmox has 192.168.66.254 on LAN side
all my virtual machines has "bridged Ethernet"
and I have one SME as full server gateway and other only in "server only mode" + other virtual things

the first one has  for example  WAN side 192.168.1.253/24 gw 192.168.1.254,
other ones
192.168.1.252/24 gw 1.254
192.168.1.251/24 gw 1.254
or in other class (how works for your logical separation)
192.168.2.253/24 gw 2.254,
192.168.3.253/24 gw 3.254,

you'll need to provide accurate routing table to your router
also you should make some "DNS" entries to suit you
for example
you'll have one server (the main one) and a second one which you'll prefer to name it as "backup.mydomain.org"
you will make an name entry even in your main sme even in your router and put your sme as DNS client  for this also

but I think all this are basics for networking and you could get from other places on internet (or ask separate)

[purvis]

first, does the proxmox os have access to the internet. from the terminal,  can you ping www.yahoo.com.   
if that does not work, i do not see how your guest sme servers will work either.

i had no problems when i did my first install two weeks ago with proxmox
if i am not mistaken there is a check box to be set for the your sme virtual machine to auto connect to the ethernet.
that check box for the ethernet maybe off.

[EdelingF]

Proxmox and the VM's can contact the internet, but websites inside the VM's are not visible.
The websites are also still on my old server, which are visible by portforwarding HTTP, etc. in the router to the old server. This can only be done once, so I was planning to portforward to the new server and reroute to the old server and the VM's in the new server by using a proxy in the new server. This way I can move the website (Wordpress installations) one by one from the old server to the VM's on the new server. I planned seperate virtual servers for each domain with a Wordpress website.

[CharlieBrady]

I found out I need something like a reverse proxy. So, my plan was to use one of the virtual servers as a (easy to manage) proxy server. But after a week looking in forums, wiki's and website I'm still no further on how I should do that.

You can use the proxypass virtual domain feature of SME server:

http://bugs.contribs.org/show_bug.cgi?id=999
http://forums.contribs.org/index.php?topic=47160.0
http://forums.contribs.org/index.php?topic=46975.0

[purvis]

I had problems myself with proxypass. Maybe i did do it correctly.
I saw something poosibly worth mentioning.
I read where a program called Round for ubuntu could direct urls to the proper ip address.
If so and if proxmox and ubuntu are both debian linux based os then if Round does do redirection, it might load under Proxmox.
This would be great for what you are doing and my future doings also.
I would be interested, if you try this Round program, in any feed back you can provide.   

[CharlieBrady]

I had problems myself with proxypass.

Did you report your problems to the bug tracker? If not, why not?

[EdelingF]

I came over name 'Round' a few times, but the links to the download-site doesn't seem to work. I also can't find any info about it.
I'l get into proxypass to see if it fits my needs.

[Mike]

I also would like to setup 4 or 5 SME servers on a proxmox but I have not found the right way (and time) to do so just yet.
Reverse proxy is nice but does not solve all problems (as far as I know).
Best would be to have more IP-addresses.
That does solve all your problems but costs money and some providers don't even provide it.
IPv6 would solve this problem also but is not yet supported on SME and not everyone can connect to IPv6 as many people only have a IPv4 connection from their provider or do not have IPv6 enabled on all clients as many simple computer users do not even know what IPv6 is.
The next 10 years you might want to run dual stack on all systems that provide webservices to the internet so you are reachable for all people.
If you use reverse proxy you should think of the following:
If you have 4 seperate SME VM's and one VM that does the reverse proxy thing, than you will get mail problems.
SPAM checking servers will use reverse DNS to check if mail really comes from the server it seemed to be send from.
The SPAM checking servers will come back to the reverse proxy VM box when they request the hostname of the host that has the IP-address that the mail came from.
They will get back the hostname of the reverse proxy VM box which is different from the mail servers name and your mailservers will be blacklisted before you know it.
If someone thinks me wrong or has a solution to this problem than I would really like to hear it!

[EdelingF]

If you have 4 seperate SME VM's and one VM that does the reverse proxy thing, than you will get mail problems.
SPAM checking servers will use reverse DNS to check if mail really comes from the server it seemed to be send from.
The SPAM checking servers will come back to the reverse proxy VM box when they request the hostname of the host that has the IP-address that the mail came from.
They will get back the hostname of the reverse proxy VM box which is different from the mail servers name and your mailservers will be blacklisted before you know it.
If someone thinks me wrong or has a solution to this problem than I would really like to hear it!

Hmm, that's something I didn't think about....
I'll check this with my provider first.

Extra IP-addresses are only available for (expensive) business accounts. That's, in my case, not affordable. It's just hobby.
It would also be to easy by the way   

[janet]

Mike

If someone thinks me wrong or has a solution to this problem than I would really like to hear it!

I think the workaround is to configure each sme server to send mail via your ISP's mail server, which is done in the server manager Email panel.

[Mike]

Mike

I think the workaround is to configure each sme server to send mail via your ISP's mail server, which is done in the server manager Email panel.

Yes, that is a workarround alright but it is not the preferred solution I was hoping to hear.
I do not want to be dependent on my ISP's mailserver or filters they might put into place because they never tell anybody about it.
The idea of running your own mailserver is to be independent of anything but your internet connection.

[larieu]

I think you could set only one as mail server and other ones just send mails through this one

yes you'll be forced to use one account to be relayed by this first one but I think this is not the main issue

and if you really want to use other ones to receive mails just do appropriate settings to take the mails from the first one

[EdelingF]

This is the answer from my provider:

In general it is a bit old-fashioned to look at the PTR RR of a email sender. A fuzzy solution is to add several of the RRS to records in DNS, but not all mail servers support it. The easiest solution is to let a single server handle mail for all domains or to use more IP addresses.

I hope this makes sense, because I had to translate it from Dutch......

[Mike]

This is the answer from my provider:
 

I hope this makes sense, because I had to translate it from Dutch......

Kijk nou eens, die Nederlanders kom je ook overal tegen...
I will just go on in English so other people can understand it too.
Do you have a link to this solution because I'm Dutch too.
This sounds like something I would like to research to see if this is the solution to what I want to accomplish.

[Mike]

Hmm, that's something I didn't think about....
I'll check this with my provider first.

Extra IP-addresses are only available for (expensive) business accounts. That's, in my case, not affordable. It's just hobby.
It would also be to easy by the way 

Yes, I know axactly what you mean but....
If you have XS4ALL you can already run IPv4 and IPv6 so you will have the ability to run dualstack.
With IPv6 you will have a range of IP adresses and in a couple of years everyone will probably be able to talk IPv6.
Than your problems are solved....

[EdelingF]

Mike, the question was answered by the abuse helpdesk of XS4ALL (ISP).
Maybe we could use the Proxmox Mail Gateway for this?

[Mike]

Mike, the question was answered by the abuse helpdesk of XS4ALL (ISP).
Maybe we could use the Proxmox Mail Gateway for this?

Aaah, XS4ALL is the best Dutch provider and they know what they are talking about.
It's also my ISP!
Looked at the proxmox mail gateway but if I have multiple sme VM's each with their own domain than you cannot use the free version.
Also I am not sure if the proxmox mailgateway can do what we want it to because in the fifth screenshot you can see that they use internet IP ranges for their mailservers and I'm just a home user too so it is too expensive for me.

[cactus]

Kijk nou eens, die Nederlanders kom je ook overal tegen...

Yes, even on you holiday in the jungle... bleeh.

[EdelingF]

Maybe Mailcleaner can be of use to us: http://www.mailcleaner.org/
I'll try it in a VM this weekend

[Mike]

Maybe Mailcleaner can be of use to us: http://www.mailcleaner.org/
I'll try it in a VM this weekend

Mailcleaner indeed looks interesting.
I did some checking up and found this link quickly:
http://forum.mailcleaner.org/viewtopic.php?f=12&t=812
Like I said, it looks interesting and I will be waiting to hear about your report of your test this weekend.

[EdelingF]

That wasn't it. Mailcleaner I mean.
Trying ProxyPass now

[EdelingF]

Mmm, have made a ProxyPass entry and it appears to work inside my network.

Code: [Select]

[root@server ~]# db domains show mydomain.eu.org                         
mydomain.eu.org=domain
    Nameservers=internet
    ProxyPassTarget=http://66.220.149.11/
    TemplatePath=ProxyPassVirtualHosts
But it ends up like this in my browser:

Code: [Select]

https://66.220.149.11/egroupware/login.phpinstead of

Code: [Select]

https://mydomain.eu.org/egroupware/login.php
Am I missing something?

[janet]

EdelingF

Is the server at http://66.220.149.11/ configured to resolve the domain name
https://mydomain.eu.org/ ?

[EdelingF]

Yes, it is. It is the primary domain on a brand new SME8b6.
The domain was first on my old SME7.5 server, which is now used as a proxy.
I tried DNS locally and internet, but the outcome is the same.

I was wondering, I now used the ProxyPass-method explained in http://wiki.contribs.org/SME_Server:Documentation:FAQ#Proxy_Pass, but isn't it easier to use the server-manager to forward the domain to a local IP-address?
In hostnames and addresses I can forward a domain to a local IP-address. I can even decide to keep the mailserver on the proxy-server (which will probably give me problems in egroupware, I think).
I never looked at that anymore since I always build my servers the same way.

[EdelingF]

Also tried to use the hostnames and addresses in Server-manager to forward to my new server, but that doesn't seem to work at all.
So I removed the prox-rules and entered them again. If I only enter the domainname in a browser I end up on the page which says "This web site is under construction". If I add egroupware (installed under /opt) to the URL it still forwards to the internat IP-address, so outside my network I get an error.

[EdelingF]

Just found an old HowTo on Schirrm's website: SME : Using Apache Server as a reverse HTTP proxy (http://www.schirrms.net/sme/SMEApacheReverseProxy.php).

It should be something  like this :

Code: [Select]

mkdir -p /etc/e-smith/templates-custom/etc/httpd/conf/httpd.conf/
cd /etc/e-smith/templates-custom/etc/httpd/conf/httpd.conf/
vi 99reverseproxysite
And then in my case I think it should be:

Code: [Select]

<VirtualHost 0.0.0.0:80>
    ServerName mydomain.eu.org
    ServerAlias mydomain

    ProxyPass / http://66.220.149.11/
    ProxyPassReverse / http://66.220.149.11/

</VirtualHost>

<VirtualHost 0.0.0.0:443>
    ServerName mydomain.eu.org
    ServerAlias mydomain

    ProxyPass / http://66.220.149.11/
    ProxyPassReverse / http://66.220.149.11/

</VirtualHost>
(the IP-adres is of course not my IP-adres, otherwise my name would be Mark Zuckerberg)

This code has been written for SME6, but maybe someone can tell me if I can still use it for SME8?

Thinking ahead,  I think I also have to do this email, FTP, etc?

[Buckwheat]

Works on 6 it will work on 7 & 8

[Buckwheat]

You can copy & paste this script to make the changes.
Option to uninstall is included.

Save script as reverseproxysite.sh

Code: [Select]

#!/bin/bash

# Modify/Change IP addresses below & set perms to 744 and run script

host_ip="0.0.0.1"
proxy_ip="0.0.0.1"


app_name=reverseproxysite
template_num=99
template_path=etc/e-smith/templates/etc/httpd/conf/httpd.conf

#=======================================================
intro_install ()
#=======================================================
{
cat << EOF
=============== Install Information ===================
 This scripted created by buckwheat.
  reverseproxysite.sh ver 1.00.1 -- 10/11/2011
 
  To uninstall rerun script with...
 
  reverseproxysite.sh uninstall
 
=============== Installation Started ==================
 
EOF
}

#=======================================================
create_httpd_conf ()
#=======================================================
{
if [ ! -f /$template_path/$template_num$app_name ]; then

cat > /$template_path/$template_num$app_name << EOF

#------------------------------------------------------------
#   $template_num$app_name
#------------------------------------------------------------

<VirtualHost $host_ip:80>
    ServerName mydomain.eu.org
    ServerAlias mydomain

    ProxyPass / http://$proxy_ip/
    ProxyPassReverse / http://$proxy_ip/

</VirtualHost>

<VirtualHost $host_ip:443>
    ServerName mydomain.eu.org
    ServerAlias mydomain

    ProxyPass / http://$proxy_ip/
    ProxyPassReverse / http://$proxy_ip/

</VirtualHost>

EOF

echo -e "\n ====== $template_num$app_name template created"
else
echo -e "\n ====== $template_num$app_name found...skipping"
fi
}

#------------------------------------------------------------
expand_template_httpd.conf()
#------------------------------------------------------------
{
expand-template /etc/httpd/conf/httpd.conf
echo -e "\n ====== Expanding Templates complete"
}

#------------------------------------------------------------
httpd_service_restart()
#------------------------------------------------------------
{
sv h /service/httpd-e-smith
echo -e "\n ====== Restarting the httpd service done"
}

#------------------------------------------------------------
uninstall ()
#------------------------------------------------------------
{
cat << EOF
================== Uninstall Started ====================

EOF

rm -rf /$template_path/$template_num$app_name

expand_template_httpd.conf
httpd_service_restart

cat << EOF

=============== Uninstall Complete ======================
EOF
}

#------------------------------------------------------------
end ()
#------------------------------------------------------------
{
cat << EOF

=============== Install Complete ======================

EOF
exit 0
}

#################################
# Main Program #
#################################

clear

intro_install

if [[ ${1} == "uninstall" ]]; then
echo -e "\n ****** WARNING >> You are about to Uninstall $app_name!!!\n "
read -p " Are you sure you want to continue?(y|Y): "

if  [[ ${REPLY} == "Y" || ${REPLY} == "y" ]] ; then
uninstall
fi
exit 0
fi

create_httpd_conf
expand_template_httpd.conf
httpd_service_restart
end

exit 0

[EdelingF]

Buckwheat, thanks. It's beginning to get clearer for me now, but I still have a few questions looking at the code.
If I 'read' it correctly, the code says: "If you call on mydomain.eu.org at the main server (host_ip?) at port 80, then proxy to new/virtual server (proxy_IP?)"?
If correct, I assume I should do something like this for other ports like for instance port 21, 25 and 110?
Which files/maps should the perms be set to 744?

[Buckwheat]

Which files/maps should the perms be set to 744?

That was just a reminder for reverseproxysite.sh so it will run, default
linux perms for creation of a file is 644, and thus will not execute.

To run the script you need 744 (X)

If correct, I assume I should do something like this for other ports like for instance port 21, 25 and 110?

I haven't played with those ports, so have fun, and do let us know your expert findings.
If you play with it long enough you'll have expert findings.

I do know the http://wiki.contribs.org/SME_Server:Documentation:FAQ#Proxy_Pass works perfectly (w/o port forwards).
The server-manager doesn't seem to work for all occasions.

hth

[Buckwheat]

@EdelingF
From your OP

But the only problem is to get connect the servers to the internet. I mean, to make the servers 'visible' from outside my network. I thought, no problem, just make some changes in the modem/router like I did with my old server by using port-forwarding. Wrong!

I thought, no problem,

Correct!!

You should have been able to PF to the new server without problem/issue.
My guess is you might not have removed the old PF first and you had two port 80 PF's.
Which won't work.
IOW One port -- only one port forward for that port i.e. 80.

I found out I need something like a reverse proxy.

Maybe!!
Unless of course you have some special case or issue, which is not clear at this point.
Keep in mind, with a proxy you still may need to do some redirect dirty work on the proxy, depends on what your trying to accomplish.
hth