Koozali.org: home of the SME Server

odd browser problem just surfaced, could it be SME related?

Online TerryF

  • grumpy old man
  • *
  • 1,821
  • +6/-0
Re: odd browser problem just surfaced, could it be SME related?
« Reply #15 on: August 20, 2022, 02:04:19 AM »
Since I've never done a wget and local install before, tell me if I have correctly guessed the process for doing it.

First I cd to the folder where I want to store the RPM file.
Then I
Code: [Select]
wget https://mirror.de-labrusse.fr/smeserver/6/noarch/smeserver-dar2-0.0.3-1.el6.sme.noarch.rpmFinally I would
Code: [Select]
yum install smeserver-dar2-0.0.3-1.el6.sme.noarch.rpm

I usually try and keep it as foolproof as possible :-) yum localinstall /tmp/rpm_name.rpm although localinstall was supposed to only be needed on el5, el6 just install

see JPs warnings, if you want to play best to use a test VM and NOT a prod system
--
qui scribit bis legit

Online TerryF

  • grumpy old man
  • *
  • 1,821
  • +6/-0
Re: odd browser problem just surfaced, could it be SME related?
« Reply #16 on: August 20, 2022, 06:52:01 AM »
 DAR2 contrib has been added to smecontribs for SME10  see wiki for details https://wiki.koozali.org/DAR2

Please, any who have the need and are prepared to do a little extra curricula activity see Bug 12153 for intial port to sme10 
--
qui scribit bis legit

Offline wdepot

  • ***
  • 89
  • +0/-0
    • http://westerndepot.com
Re: odd browser problem just surfaced, could it be SME related?
« Reply #17 on: August 23, 2022, 01:04:21 AM »
panel can not load on sme10 for a sme9 or before contrib because of suid perl not available anymore


regarding your templates
- 91e-smithAccessPrimarysubdirs

will break your httpd with next update

-90e-smithAccess40ibays
will break also, and i highly suggest not overriding such essential fragment to add error page and rather add a fragment in Virtualhost folder with a condition based on domain


35SSL10SSLHonorCipherOrder exist already with a way to set what you want using db. set accordingly and remove the custom fragment to avoid issue if syntax change in a new release. 


is suspect 92Expires is **part of** the problem as after 2 days i am able to see your website once and then not anymore if i hit refresh or click another link to it.
also remember you have cloudflare doing caching for you in between when not behind your lan. 

speaking of cloudflare have you checked your settings there and compared to those of your other ibay which is working? I pointed multiple time in that direction and you never replied. 


regarding your php.ini override remember sme is using php-fpm all you do in php.ini is overrided by php-fpm so custom fragment are probably useles and might interfer with cli calls

regarding qmail custom fragment most of then have configurable setting via db.


the more custom you do the more risk of breaking on update you have

To answer the question about Cloudflare first, I've checked those settings every time someone reported a problem to us and they have never changed. The settings on all three domains match one another so I doubt that the problem lies there.

I've dumped the custom template for mod_expires. I had added it after reading somewhere that recommended adding this to reduce file requests to the server from browsers. Since you suspected it might be causing a problem and since our server is never under much of a load anyway I figured that we could do without it.

I also took a look at the original SME CipherOrder template and found the db property to set so I've dumped that custom template as well. Since I had to copy the custom php.ini fragment into the opt/remi/php## custom template folders for those settings to work in the ibays it makes sense to do like you said and drop it from the /etc custom template. When I get a chance I'll see what I can figure out for the qmail db settings.

The questions I have are about the other two templates that you said will cause problems. For setting the custom error document I can see where that could break due to the modification of an existing SME template. Can I do something like this instead:
Code: [Select]
<Directory /home/e-smith/files/ibays/Primary/html>
ErrorDocument statements
</Directory>
<Directory /home/e-smith/files/ibays/sierraplaza/html>
ErrorDocument statements
</Directory>
and then name that custom template something like 9999ErrorDocs to make sure it gets added to the bottom of the httpd.conf file below anything written by the SME master templates?

You also say that 91e-smithAccessPrimarysubdirs will break httpd on the next update. It this due to the name of the template that will cause it to end up in the middle of some future settings that it shouldn't be in and a rename to put it below SME added items will fix it or is there another problem with it? Also are you referring to the next update of SME10 or is it the upcoming SME11 that it will break?

Offline Jean-Philippe Pialasse

  • *
  • 2,747
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: odd browser problem just surfaced, could it be SME related?
« Reply #18 on: August 23, 2022, 02:43:34 AM »
the issue with your custom template will arise with 10.1 update coming soon. 



I have found what create your issue.  this is the double redirection in your ibay.

I can point to the .store oneget back refresh. quit the page and come back it works everytime. 
as soon as i go to the .com, first click on a link brings you to .store and from there whatever you do you get stuck with browsing your own ip.

i would either have cloudflare handle the redirection, either move the .com to another ibay and redirect from there.

you could also check how you do the redirection. apache redirect or apache mod rewrite. 

Offline wdepot

  • ***
  • 89
  • +0/-0
    • http://westerndepot.com
Re: odd browser problem just surfaced, could it be SME related?
« Reply #19 on: August 24, 2022, 01:27:24 AM »
the issue with your custom template will arise with 10.1 update coming soon. 



I have found what create your issue.  this is the double redirection in your ibay.

I can point to the .store one get back refresh. quit the page and come back it works everytime. 
as soon as i go to the .com, first click on a link brings you to .store and from there whatever you do you get stuck with browsing your own ip.

i would either have cloudflare handle the redirection, either move the .com to another ibay and redirect from there.

you could also check how you do the redirection. apache redirect or apache mod rewrite.
I think I've got the httpd.conf custom templates figured out now. I've got it trimmed down to three of them.

75AddTypesAV which you didn't comment on so I assume it will be fine.

ZZe-smithAccessPrimarysubdirs which contains the following to secure our web site administration page:
Code: [Select]
<Directory /home/e-smith/files/ibays/Primary/html/administration>
    Options None
    Options +Includes
    SSLRequireSSL
    AllowOverride None
    Order deny,allow
    Deny from all
    Allow from 127.0.0.1 10.0.3.0/255.255.255.0
    AuthType Basic
    AuthBasicProvider external
    AuthName "WD Site Admin"
    AuthExternal pwauth
    Require user admin
    Satisfy any
</Directory>
All I did with this file was rename it to something that should place it below any templates that SME might add in future updates. Let me know if there is anything in this file that might cause problems in the future and please be specific about exactly what it is that would cause a problem.

The final custom template for httpd.conf is ZYerrorDocsibays which contains:
Code: [Select]
<Directory /home/e-smith/files/ibays/Primary/html>
    ErrorDocument 400 /http_error.php?error_id=400
    ErrorDocument 401 /http_error.php?error_id=401
    ErrorDocument 403 /http_error.php?error_id=403
    ErrorDocument 404 /http_error.php?error_id=404
    ErrorDocument 405 /http_error.php?error_id=405
    ErrorDocument 408 /http_error.php?error_id=408
    ErrorDocument 415 /http_error.php?error_id=415
    ErrorDocument 416 /http_error.php?error_id=416
    ErrorDocument 417 /http_error.php?error_id=417
    ErrorDocument 500 /http_error.php?error_id=500
    ErrorDocument 501 /http_error.php?error_id=501
    ErrorDocument 502 /http_error.php?error_id=502
    ErrorDocument 503 /http_error.php?error_id=503
    ErrorDocument 504 /http_error.php?error_id=504
    ErrorDocument 505 /http_error.php?error_id=505
</Directory>
<Directory /home/e-smith/files/ibays/sierraplaza/html>
    ErrorDocument 400 /http_error.php?error_id=400
    ErrorDocument 401 /http_error.php?error_id=401
    ErrorDocument 403 /http_error.php?error_id=403
    ErrorDocument 404 /http_error.php?error_id=404
    ErrorDocument 405 /http_error.php?error_id=405
    ErrorDocument 408 /http_error.php?error_id=408
    ErrorDocument 415 /http_error.php?error_id=415
    ErrorDocument 416 /http_error.php?error_id=416
    ErrorDocument 417 /http_error.php?error_id=417
    ErrorDocument 500 /http_error.php?error_id=500
    ErrorDocument 501 /http_error.php?error_id=501
    ErrorDocument 502 /http_error.php?error_id=502
    ErrorDocument 503 /http_error.php?error_id=503
    ErrorDocument 504 /http_error.php?error_id=504
    ErrorDocument 505 /http_error.php?error_id=505
</Directory>
I tested this particular code on our test server and found it correctly triggers the error page so I got rid of the 90e-smithAccess40ibays custom template that would have caused problems with future updates.

I also took a look to find the proper db settings to force the use of only TLS1.1 or higher so I was able to get rid of that custom template. I'm guessing that since openSSL was updated to version 1.1.1 that the SSL cipher list for SME10 was updated to prefer TLS1.3 ciphers first.

So that brings us back to the problem I posted about in the first place. I'm confused about what you mean by double redirection in the ibay. Other than the fact that I have pointed both westerndepot.com and westerndepot.store to the primary ibay I have set no redirects of any kind in Apache. In fact I made sure to turn off forced SSL under Apache because that immediately created problems for anyone that tried to connect without SSL. The fact that all links within the site are set as https switches to SSL soon enough to suit our needs.

Granted all of the links and the cookie domain are set as westerndepot.store by the site but I would have thought that clicking one of the links when the site is accessed as westerndepot.com would have had essentially the same effect as clicking on a link to a different site entirely. Do you think I need to have PHP check to see whether the site has been accessed as westerndepot.com or westerndepot.store and then set the links accordingly? I'd just need to check the PHP $_SYSTEM variable to see which domain was used to access the site and have it set the defines for the site domain based on that.

Offline Jean-Philippe Pialasse

  • *
  • 2,747
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: odd browser problem just surfaced, could it be SME related?
« Reply #20 on: August 24, 2022, 02:29:37 AM »
as long as you use a redirector such as cloudflare, you need to keep that simple on your side. 

cloudflare is already spoofing your dns by telling he acts as your domain and then probably reverse proxy to your server.

adding more internal rewriting in the same ibay make it a reciepe for disaster.  keep it simple like your other website. 

put the .com ok a desicated ibay with either a php script or a httpd configuration to redirect all .store before accessing the website.  I think it will be kiss and will solve the issue. 

Offline wdepot

  • ***
  • 89
  • +0/-0
    • http://westerndepot.com
Re: odd browser problem just surfaced, could it be SME related?
« Reply #21 on: August 24, 2022, 08:19:19 PM »
as long as you use a redirector such as cloudflare, you need to keep that simple on your side. 

cloudflare is already spoofing your dns by telling he acts as your domain and then probably reverse proxy to your server.

adding more internal rewriting in the same ibay make it a reciepe for disaster.  keep it simple like your other website. 

put the .com ok a desicated ibay with either a php script or a httpd configuration to redirect all .store before accessing the website.  I think it will be kiss and will solve the issue.
I think I may have found the source of the problem. I was taking a look at the contents of the PHP $_SERVER variable this morning and found three different parts of the variable that reflect the way the site was accessed: SERVER_NAME, HTTP_HOST and SCRIPT_URI. With the standard SME settings all three reflected exactly the way the site was accessed so if I used www.westerndepot.com they all three would contain www.westerndepot.com. If I used westerndepot.store they would all three contain westerndepot.store and if I directly used the IP address then all three contained the IP address.

However when I looked at the online manual for the PHP $_SERVER variable I discovered this note:
Quote
'SERVER_NAME'
    The name of the server host under which the current script is executing. If the script is running on a virtual host, this will be the value defined for that virtual host.

        Note: Under Apache 2, you must set UseCanonicalName = On and ServerName. Otherwise, this value reflects the hostname supplied by the client, which can be spoofed. It is not safe to rely on this value in security-dependent contexts.
That prompted me to look at the contents of http.conf where I discovered that UseCanonicalName was turned off. I found the SME template for this setting and saw that the off is hard coded and not set by a db variable. I therefore copied that SME template to templates-custom and changed the off to on.

With UseCanonicalName now turned on HTTP_HOST still matches what was used to access the site but SERVER_NAME and SCRIPT_URI contain westerndepot.com if the site is accessed using westerndepot.com, www.westerndepot.com or the IP address and they contain westerndepot.store if accessed using either westerndepot.store or www.westerndepot.store.

All links on the web site are created from the following bit of code:
Code: [Select]
  define('HTTP_SERVER', 'https://westerndepot.store');
  define('HTTPS_SERVER', 'https://westerndepot.store');
  define('HTTP_COOKIE_DOMAIN', 'westerndepot.store');
  define('HTTPS_COOKIE_DOMAIN', 'westerndepot.store');
And I had been thinking of doing something like this:
Code: [Select]
if (stripos($_SERVER['SERVER_NAME'], 'westerndepot.com') !== false) {
  define('HTTP_SERVER', 'https://westerndepot.com');
  define('HTTPS_SERVER', 'https://westerndepot.com');
  define('HTTP_COOKIE_DOMAIN', 'westerndepot.com');
  define('HTTPS_COOKIE_DOMAIN', 'westerndepot.com');
} else {
  define('HTTP_SERVER', 'https://westerndepot.store');
  define('HTTPS_SERVER', 'https://westerndepot.store');
  define('HTTP_COOKIE_DOMAIN', 'westerndepot.store');
  define('HTTPS_COOKIE_DOMAIN', 'westerndepot.store');
}
to match the links to the way the site was accessed but with UseCanonicalName now turned on I'm not sure that I need to do that now. What is happening with the site on your end now that the usage of Canonical Names has been forced on?

Offline Jean-Philippe Pialasse

  • *
  • 2,747
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: odd browser problem just surfaced, could it be SME related?
« Reply #22 on: August 25, 2022, 03:05:10 AM »
you are doing things to create situations like the current one and also are harming your website reputation by dividing the traffic between two domains with exactly the same content. 

this is bad for two reasons :

- you get half visitor or so on each domains. 

- also having a website that looks like a duplicate reduce a lot the rating for google and other search engine. 

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: odd browser problem just surfaced, could it be SME related?
« Reply #23 on: August 25, 2022, 12:26:33 PM »
you are doing things to create situations like the current one and also are harming your website reputation by dividing the traffic between two domains with exactly the same content. 

Yup. Almost becoming a XY Info problem.

Quote
- also having a website that looks like a duplicate reduce a lot the rating for google and other search engine.

Yup. You will get clattered for doing this. A number of years ago Google went to town on skin sites and the like. They are pretty good at spotting sites that are mainly just duplicates with maybe a thin skin on top, both at the same address, and at different addresses.

Do one site. If you want a second, you need to make it substantially different.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline wdepot

  • ***
  • 89
  • +0/-0
    • http://westerndepot.com
Re: odd browser problem just surfaced, could it be SME related?
« Reply #24 on: August 25, 2022, 11:04:02 PM »
you are doing things to create situations like the current one and also are harming your website reputation by dividing the traffic between two domains with exactly the same content. 

this is bad for two reasons :

- you get half visitor or so on each domains. 

- also having a website that looks like a duplicate reduce a lot the rating for google and other search engine.
We never have advertised westerndepot.store or used that domain in our Google Product Feed. The only real reason we applied for it in the first place was because people were getting directed to odd places when trying to connect to westerndepot.com even though our DNS settings with Cloudflare had never changed. In those cases would tell them to try connecting to westerndepot.store instead.

I've permanently set all links on our site back to westerndepot.com and  have changed the DNS records with Cloudflare for westerndepot.store from A records pointing to our server to CNAME records pointing to westerndepot.com. With that done should I delete the westerndepot.store domain from our server or do I need to leave it there?

Offline Jean-Philippe Pialasse

  • *
  • 2,747
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: odd browser problem just surfaced, could it be SME related?
« Reply #25 on: August 25, 2022, 11:12:59 PM »
with all that done you still need to setup the store domain to a separate ibay and set it the way you want to redirect to .com.

as long as you have your domain pointing to your server the default will be to point to primary ibay unless set otherwise. 

again if it point to your website, google will naturally explore it and find out it is a duplicate.  you do not have to declare it to google , it will occur.

wether you declare it a cname or A it wil still end to your ip and then your website unless specified otherwise to your apache config ie, having domain pointed to another ibay stating wait we redirect you to .com

Offline wdepot

  • ***
  • 89
  • +0/-0
    • http://westerndepot.com
Re: odd browser problem just surfaced, could it be SME related?
« Reply #26 on: August 26, 2022, 01:08:59 AM »
with all that done you still need to setup the store domain to a separate ibay and set it the way you want to redirect to .com.

as long as you have your domain pointing to your server the default will be to point to primary ibay unless set otherwise. 

again if it point to your website, google will naturally explore it and find out it is a duplicate.  you do not have to declare it to google , it will occur.

whether you declare it a cname or A it wil still end to your ip and then your website unless specified otherwise to your apache config ie, having domain pointed to another ibay stating wait we redirect you to .com
Okay, I've created a new ibay specifically for westerndepot.store. Now how do I redirect any traffic that might go to it to the same file on westerndepot.com. In other words:
Code: [Select]
westerndepot.store/index.php?cpath=456 to westerndepot.com/index.php?cpath=456
westerndepot.store/specials.php to westerndepot.com/specials.php
and so forth.

I'm not seeing anything in server-manager under either Information Bays or I-Bays Web Hosting that would set this up so I'm assuming that I will need a custom template for Apache to add the needed redirect. I just have no idea how to word it.

Oh, and I just got a call from a guy reporting that he was getting a "server at 108.204.251.233 took too long to respond" when trying to access westerndepot.com so something is still screwed up somewhere since that IP is owned by AT&T not us. The DNS settings at Cloudflare haven't been changed so something else is causing the problem.

Offline Jean-Philippe Pialasse

  • *
  • 2,747
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: odd browser problem just surfaced, could it be SME related?
« Reply #27 on: August 26, 2022, 04:14:52 AM »
Code: [Select]
# host westerndepot.com
westerndepot.com has address 104.21.80.147
westerndepot.com has address 172.67.223.182
westerndepot.com has IPv6 address 2606:4700:3032::ac43:dfb6
westerndepot.com has IPv6 address 2606:4700:3037::6815:5093
westerndepot.com mail is handled by 10 mail.westerndepot.com.


while round robin dns is a thing, from my experience, you should not play with that because most often than needed the browser will pick the wrong ip

if your server is 104.21.80.147 , then remove 172.67.223.182 or vice et versa.


for what I tested, this works with 172.67.223.182, so I guess you need to tidy your DNS... again at cloudflare, as they are your NS.

for the redirection check the internet for either
httpd Redirect

or
httpd mod rewrite

either put it in a htaccess file or in a template custom.
also rather than an ibray you can also set your domain with its dedicated virtual host template see wiki for that.

Code: [Select]
# host westerndepot.store
westerndepot.store has address 188.114.96.0
westerndepot.store has address 188.114.97.0
westerndepot.store has IPv6 address 2a06:98c1:3121::
westerndepot.store has IPv6 address 2a06:98c1:3120::
westerndepot.store mail is handled by 10 mail.westerndepot.com.


Offline wdepot

  • ***
  • 89
  • +0/-0
    • http://westerndepot.com
Re: odd browser problem just surfaced, could it be SME related?
« Reply #28 on: August 26, 2022, 11:59:19 PM »
Code: [Select]
# host westerndepot.com
westerndepot.com has address 104.21.80.147
westerndepot.com has address 172.67.223.182
westerndepot.com has IPv6 address 2606:4700:3032::ac43:dfb6
westerndepot.com has IPv6 address 2606:4700:3037::6815:5093
westerndepot.com mail is handled by 10 mail.westerndepot.com.


while round robin dns is a thing, from my experience, you should not play with that because most often than needed the browser will pick the wrong ip

if your server is 104.21.80.147 , then remove 172.67.223.182 or vice et versa.


for what I tested, this works with 172.67.223.182, so I guess you need to tidy your DNS... again at cloudflare, as they are your NS.

for the redirection check the internet for either
httpd Redirect

or
httpd mod rewrite

either put it in a htaccess file or in a template custom.
also rather than an ibray you can also set your domain with its dedicated virtual host template see wiki for that.

Code: [Select]
# host westerndepot.store
westerndepot.store has address 188.114.96.0
westerndepot.store has address 188.114.97.0
westerndepot.store has IPv6 address 2a06:98c1:3121::
westerndepot.store has IPv6 address 2a06:98c1:3120::
westerndepot.store mail is handled by 10 mail.westerndepot.com.

The IP addresses you listed are all proxy IPs from Cloudflare.

I thought I found the proper code for the redirect but I am having a problem with it. I created a new custom template for httpd.conf named ZXredirectWesterndepotStore which contains the following:
Code: [Select]
<Directory /home/e-smith/files/ibays/store/html>
    RewriteEngine On
    RewriteRule ^(.*)$ https://westerndepot.com/$1 [R=301,L]
</Directory>
Once I did an expand-template on httpd.conf and a signal-event console-save I pointed westerndepot.store to the store ibay. The settings for this ibay are as follows:
Group admin
Write=admin Read=group
Public Access entire internet (no passwords)
Dynamic Content disabled
Force Secure disabled
and in web hosting I changed Directory Listing to disabled and left everything else at the default.

The problem now is that when I tested accessing anything using westerndepot.store all I get is a Forbidden error message. I could see a Not Found message with the redirect apparently not working properly since there is nothing in the ibay but the Forbidden message is puzzling since that ibay should be publicly readable with no password. What did I do wrong?
For now, until it can get resolved, I'm going to temporarily switch westerndepot.store back to the Primary ibay since I'm getting flooded with email messages from Fail2Ban due to this problem.

I had one thought occur to me last night. Is it possible that running westerndepot.com from the Primary ibay might have something to do with some people getting sent to odd IP addresses when trying to access the site? After all that is the ibay that is used if you put the IP address for the server into a web browser. We've been running westerndepot.com in the Primary ibay since the days of SME6 and didn't start having the problem reported to us until about halfway through the lifetime of SME9 so is it possible that some change to Apache could have started causing an occasional glitch with the IP and domain pointed to the same ibay? Would it be worth moving westerndepot.com from Primary to a new ibay?

Offline Jean-Philippe Pialasse

  • *
  • 2,747
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: odd browser problem just surfaced, could it be SME related?
« Reply #29 on: August 27, 2022, 03:54:31 AM »
thousand server are running using primary ibay without issue. 

the chances are higher that the more you add layer to you setting the more you hack things to make it seem to work as you intend.

the code you show to alter the server variable might work with one situation and just fails when one arrives thru 2 proxy (one from cloudflare and their own. ) or any other situation. 


regarding the current rewrite rule.  what are the setting of your ibay before the rule is added?
what is the httpd error log content when trying to access?