421 Service Not Available when attempting to connect

[awilliams219]

Hello,

I've been slamming my head against the wall for the better part of two days trying to get this to work. All attempts to connect to a server running proftpd 1.3.6 and mod_proxy 0.3 (and 0.4, but I'll get to that in a minute) have failed with the error "421 Service not available, remote server has closed connection".

The error happens immediately upon connecting to the proxy server, with no opportunity to log in. I have been unable to generate any log entries. The failure happens any time I LoadModule mod_proxy.c.

The proxy server works (albeit in a standard ftp server sort of way) without mod_proxy loaded, but adding the LoadModule directive (as well as the configuration for mod_proxy) causes all ftp connections to the server to fail. However, the service will start and appear to run correctly. The only entry that appears in any log file is where mod_proxy selects the backend ftp server. Trace logging does not appear to work, as no log entries are ever created in the trace log, no matter how high I have verbosity set.

I've recompiled several times, sometimes statically linked, sometimes as a dso. I've tried whittling my configuration down to the bare minimum. I've run tcpdump on the server acting as proxy - I can see the connection coming in to the server from the client, but no packets ever depart the proxy server heading for the backend server.

The server is Ubuntu Server edition 18.04 (currently running in a virtualbox environment attempting to test before releasing on prod). The UFW firewall in Ubuntu is completely disabled. netstat -lanp shows :21 as "listening" but lists no process name for the listener.

The virtualbox environment has 2 NICs - one hooked up to the outside world using NAT, and one host-only network on the 192.168.56.0/24 subnet. I've run TCP dump on both interfaces, and neither attempt to connect out to the backend server.

Our current configuration:

/etc/proftpd.conf:

# This is a basic ProFTPD configuration file (rename it to
# 'proftpd.conf' for actual use.  It establishes a single server
# and a single anonymous login.  It assumes that you have a user/group
# "nobody" and "ftp" for normal operation and anon.

ServerName			"ProFTPD Default Installation"
ServerType			standalone
DefaultServer			on

# Port 21 is the standard FTP port.
Port				21

# Don't use IPv6 support by default.
UseIPv6				off

# Umask 022 is a good standard umask to prevent new dirs and files
# from being group and world writable.
Umask				022

# To prevent DoS attacks, set the maximum number of child processes
# to 30.  If you need to allow more than 30 concurrent connections
# at once, simply increase this value.  Note that this ONLY works
# in standalone mode, in inetd mode you should use an inetd server
# that allows you to limit maximum number of processes per service
# (such as xinetd).
MaxInstances			30

# Set the user and group under which the server will run.
User				nobody
Group				nogroup

SyslogLevel debug

# To cause every FTP user to be "jailed" (chrooted) into their home
# directory, uncomment this line.
#DefaultRoot ~

# Normally, we want files to be overwriteable.
AllowOverwrite		on

# Bar use of SITE CHMOD by default
<Limit SITE_CHMOD>
  DenyAll
</Limit>

LoadModule mod_proxy.c

<IfModule mod_proxy.c>
  ProxyEngine on
  ProxyLog /var/log/proftpd/proxy_main.log
  ProxyTables /var/ftp/proxy
</IfModule>

<VirtualHost {{{ IP Address of Proxy Server }}}>
   FactsAdvertise off

   ProxyEngine on
   ProxyLog /var/log/proftpd/proxy_tmn.log
   ProxyRole reverse
   ProxyReverseConnectPolicy random
   ProxyReverseServers ftp://{{{ IP Address of Backend Server }}} 
</VirtualHost>

I've obfuscated the IP address since the server is publicly routable.

This is a stripped down config meant just to get the basic functionality. This server doesn't actually host any files and is just in place to proxy traffic to another server elsewhere.

Log entries:

As I mentioned, very little is displayed in the logs. Even with trace logging set to Trace proxy:20, nothing is ever written to the trace log.

This is the only line that ever actually makes it into a log file:

/var/log/proftpd/proxy_tmn.log:

2019-07-23 17:46:52,715 mod_proxy/0.3[5077]: selected backend server 'ftp://{{{ IP Address of Backend Server }}}'
2019-07-23 17:47:10,845 mod_proxy/0.3[5078]: selected backend server 'ftp://{{{ IP Address of Backend Server }}}'
2019-07-23 17:53:59,720 mod_proxy/0.3[5312]: selected backend server 'ftp://{{{ IP Address of Backend Server }}}'
2019-07-23 17:57:04,067 mod_proxy/0.3[5325]: selected backend server 'ftp://{{{ IP Address of Backend Server }}}'
2019-07-23 17:57:45,635 mod_proxy/0.3[5326]: selected backend server 'ftp://{{{ IP Address of Backend Server }}}'
2019-07-23 18:00:19,743 mod_proxy/0.3[5382]: selected backend server 'ftp://{{{ IP Address of Backend Server }}}'

Again, I've obfuscated the IP address since it's publicly routable.

About our setup:

The reason we're proxying is due to network topology changes with regard to our FTP infrastructure. We've recently closed a security hole in our FTP configuration. Part of this security change was to move the FTP service to its own dedicated server. Previously, this service was running on the web server on the root domain ( {{blahblah}}.com ). We have a lot of embedded systems out in the wild that call home for assets via FTP - hundreds of boxes, likely. Some have been running since as early as 2005. Calling all the boxes home to update the ftp configuration is not realistic at best, and impossible at worst. So the plan is to run an ftp reverse proxy on the web server pointing traffic at the shiny new ftp server.

Regarding 0.4:

The reason I'm using 0.3 is that 0.4 refused to compile for me. This should probably be it's own separate issue, and if you like, i'll raise that issue as well. The problem there is that the compiler throws an error related to redis, which we're not using. I don't remember the exact phrasing since I've given up on 0.4 and switched to 0.3, but the gist of it was that redis_prefix and redis_prefixsz were undefined in proxy_tls_redis_as_datastore in redis.c. If you want me to raise a separate issue on that, I'll get you a copypasta of the exact message. I'm not a c programmer, so I'm not useful for fixing it.

[Castaglia]

Yes, please do file a separate issue for the compilation error you're encountering; I'd like to address that as well.