Use custom port in htacess

I have a domain setup with the Apache host listening to a custom post instead of default port 80.

The main page works fine, but setting the permalink structure to postname returns 404 for all posts.

Tried to edit the rewrite rule in htaccess to add the custom port, but still getting the same issue.