WEB Proxy
Document revision 1.3 (12-Nov-2003)
This document applies to MikroTik RouterOS v2.7
The MikroTik RouterOS has the squid proxy server implementation.
Proxy server features:
- Regular http proxy
- Transparent proxy. Can be transparent and regular at the same time
- Access list by source, destination, URL and requested method
- Cache access list (specifies which objects to cache, and which not)
- Direct Access List (specifies which resources should be accessed directly,
and which - through an another proxy server)
- Logging facility
Packages required : web-proxy
License required : Basic
Home menu level : /ip web-proxy
Protocols utilized : HTTP/1.0 (RFC1945),
HTTP/1.1 (RFC2616),
FTP (RFC959)
Hardware usage: uses disk space, if available (see description below)
Software
Package Installation and Upgrading
IP Addresses
and Address Resolution Protocol (ARP)
Firewall
Filters and Network Address Translation (NAT)
Log Management
The web proxy can be used as transparent and normal web proxy at the same time.
In transparent mode it is possible to use it as standard web proxy, too.
However, in this case, proxy users may have trouble to reach web pages which are accessed transparently.
When setting up Web proxy, make sure it serves only your clients,
and is not misused as relay. Please read the security notice in the Access List Section!
Note that it may be useful to have Web proxy running even with no cache when
you want to use it as something like HTTP and FTP firewall (for example, denying access
to mp3 files) or to redirect requests to external proxy transparently
Submenu level : /ip web-proxy
[admin@MikroTik] ip web-proxy> print
enabled: no
address: 0.0.0.0:3128
hostname: ""
transparent-proxy: no
parent-proxy: 0.0.0.0:0
cache-administrator: "webmaster"
max-object-size: 4096 kB
max-cache-size: unlimited
status: stopped
reserved-for-cache: 10240 kB
[admin@MikroTik] ip web-proxy>
enabled (yes | no, default: no) - whether web-proxy is enabled or not
address (IP address:port, default: 0.0.0.0:3128) - IP address
(0.0.0.0 for any) and port (mandatory) on which proxy will be listening for requests
hostname (string, default: "") - hostname (DNS or IP address) of the web proxy
transparent-proxy (yes | no, default: no) - use transparent mode
parent-proxy (IP address:port, default: 0.0.0.0:0) - upper-level proxy
0.0.0.0:0 - disables parent proxy server
max-object-size (integer, default: 4096) - objects larger than this size
will not be saved on disk. The value is specified in kilobytes. If you wish to get a high bytes hit
ratio, you should probably increase this (one 32 MB object hit counts for 3200 10KB hits).
If you wish to increase speed more than your want to save bandwidth you should leave this low
max-cache-size (none | unlimited | integer) - maximal cache size
in MB
none - web proxy will not use cache
unlimited - web proxy will use as much drivespace, as possible
Statistics:
status (string) - displays status of the proxy server:
stopped - proxy is disabled and is not running
rebuilding-cache - proxy is enabled and running, existing cache is being verified
running - proxy is enabled and running
stopping - proxy is shutting down (max 10s)
clearing-cache - proxy is stopped, cache files are being removed
creating-cache - proxy is stopped, cache directory structure is being created
dns-missing - proxy is enabled, but not running because of unknown DNS server
(you should specify it under /ip dns)
invalid-address - proxy is enabled, but not running because of invalid address
(you should change address or port)
invalid-cache-administrator - proxy is enabled, but not running because of
invalid cache-administrator's e-mail address
invalid-hostname - proxy is enabled, but not running because of invalid
hostname (you should set a valid hostname value)
error-logged - proxy is not running because of unknown error. This error
is logged as System-Error. Please, send us this error and some description, how it happened
reserved-for-cache (integer) - maximal cache size, that is accessible to web-proxy
By default the proxy cache can use as much disk space as there is allocated for it.
When the system allocates the space for the proxy cache, 1/7th of the total partition (disk) size
is reserved for the system, but not less than 50MB. The rest is left for the proxy cache.
The system RAM size is considered as well when allocating the cache size.
The cache size is limited so, that there are at least 11.2MB of RAM
per 1GB of cache plus 32MB of RAM is reserved for the system. max-cache-size
is also taken in account, so the cache will not occupy more than it is specified
in this property. The effective limit is calculated as a minimum of all three
limits.
Expire time of cache entries can be different for each HTML page (specified in headers).
But, if there is no such header, the entry will be considered fresh for max 72 hours.
To enable the proxy on port 8080:
[admin@MikroTik] ip web-proxy> set enabled=yes address=0.0.0.0:8080
[admin@MikroTik] ip web-proxy> print
enabled: yes
address: 0.0.0.0:8080
hostname: ""
transparent-proxy: no
parent-proxy: 0.0.0.0:0
cache-administrator: "webmaster"
max-object-size: 4096 kB
max-cache-size: unlimited
status: rebuilding-cache
reserved-for-cache: 10240 kB
[admin@MikroTik] ip web-proxy>
Command name : /ip web-proxy monitor
Statistics:
status (string) - the same as for /ip web-proxy print
uptime (time) - uptime of the proxy server
clients (integer) - number of present and past proxy clients with different IP addresses (in current uptime)
requests (integer) - total number of requests to the proxy (in current uptime)
hits (integer) - number of requests satisfied with proxy's cache (in current uptime)
cache-size (integer) - current cache size in kilobytes
received-from-servers (integer) - how many kilobytes did proxy receive from remote
servers (in current uptime)
sent-to-clients (integer) - how many kilobytes did proxy send to the clients to
resolve their requests (in current uptime)
hits-sent-to-clients (integer) - how many kilobytes of sent traffic were taken from
the cache (in current uptime)
To monitor the web proxy:
[admin@MikroTik] > ip web-proxy monitor
status: running
uptime: 4d19h8m14s
clients: 9
requests: 10242
hits: 3839
cache-size: 328672 kB
received-from-servers: 58108 kB
sent-to-clients: 65454 kB
hits-sent-to-clients: 7552 kB
[admin@MikroTik] >
Submenu level : /ip web-proxy access
Access list is implemented in the same way as MikroTik firewall rules.
Rules are processed from the top to the bottom. First matching rule specifies
decision of what to do with this connection. Connections can be matched by
its source address, destination address, destination port, substring of
requested url or request method. If none of these parameters is specified, every connection
will match this rule.
If connection is matched by a rule, action property of this rule specifies
whether connection will be allowed or not. If connection does not match any
rule, it will be allowed.
src-address (IP address/mask, default: 0.0.0.0/0) - source addresst
dst-address (IP address/mask, default: 0.0.0.0/0) - destination address
dst-port (string, default: "") - destination port list
url (string) - the URL of the request (regular expression)
method (any | connect | delete | get | head | options | post | put | trace, default: any) -
method of the request (see RFC2616 for details)
action (allow | deny, default: allow)- action to take
There is one rule by default, that disallows connect method connections
other than to 443 (https) and to 563 (snews) ports. connect method is a
security hole that allow connections (transparent tunneling) to any computer
using any protocol. It is used mostly by spammers, as they found it very
convinient to use others' mail (SMTP) servers as anonymous mail relay to send
spam over the Internet.
It is strongly recommended to deny all IP addresses except those behind the
router as the proxy still may be used to access your internal-use-only (intranet)
web servers. Also, consult examples in Firewall Manual on how to protect your router.
Details about regular expressions used in url field can be found here:
http://www.cs.utah.edu/dept/old/texinfo/regex/regex_toc.html
The default rule:
[admin@MikroTik] ip web-proxy access> print
Flags: X - disabled
0 ;;; allow CONNECT only to SSL ports 443 [https] and 563 [snews]
src-address=0.0.0.0/0 dst-address=0.0.0.0/0 dst-port=!443,563 url=""
method=connect action=deny
[admin@MikroTik] ip web-proxy access> print brief
To disallow download of MP3 files and FTP connections other than from the 10.0.0.1 server:
[admin@MikroTik] ip web-proxy access> add url=".mp3" action=deny
[admin@MikroTik] ip web-proxy access> add src-address=10.0.0.1/32 action=allow
[admin@MikroTik] ip web-proxy access> add url="ftp://" action=deny
[admin@MikroTik] ip web-proxy access> print
Flags: X - disabled
0 ;;; allow CONNECT only to SSL ports 443 [https] and 563 [snews]
src-address=0.0.0.0/0 dst-address=0.0.0.0/0 dst-port=!443,563 url=""
method=connect action=deny
1 src-address=0.0.0.0/0 dst-address=0.0.0.0/0 dst-port="" url=".mp3"
method=any action=deny
2 src-address=10.0.0.1/32 dst-address=0.0.0.0/0 dst-port="" url=""
method=any action=allow
3 src-address=0.0.0.0/0 dst-address=0.0.0.0/0 dst-port="" url="ftp://"
method=any action=deny
[admin@MikroTik] ip web-proxy access>
Submenu level : /ip web-proxy direct
If parent-proxy is specified, it is possible to tell proxy server whether to
try to pass the request to the parent proxy or to resolve it connecting to the
requested server directly. Direct Access List is managed just like Proxy Access List
described in the previous chapter except the action argument.
src-address (IP address/mask, default: 0.0.0.0/0) - source addresst
dst-address (IP address/mask, default: 0.0.0.0/0) - destination address
dst-port (string, default: "") - destination port list
url (string) - the URL of the request (regular expression)
method (any | connect | delete | get | head | options | post | put | trace, default: any) -
method of the request (see RFC2616 for details)
action (allow | deny, default: allow)- action to take:
allow - always resolve matching requests directly, not through parent proxy
deny - resolve matching requests through parent proxy if there is one.
If there in no parent proxy, action will be the same as with allow
Default action (if no rules specified or request did not match any) is deny.
Submenu level : /ip web-proxy cache
Cache access list specifies, which requests (domains, servers, pages)
have to be cached locally by web proxy, and which not.
Access list is implemented exactly the same way as web proxy access list.
Default action is to cache object (if no matching rule is found).
src-address (IP address/mask, default: 0.0.0.0/0) - source addresst
dst-address (IP address/mask, default: 0.0.0.0/0) - destination address
dst-port (string, default: "") - destination port list
url (string) - the URL of the request (regular expression)
method (any | connect | delete | get | head | options | post | put | trace, default: any) -
method of the request (see RFC2616 for details)
action (allow | deny, default: allow)- action to take:
By default, one cache access rule is already added:
[admin@MikroTik] ip web-proxy cache> print
Flags: X - disabled
0 src-address=0.0.0.0/0 dst-address=0.0.0.0/0 dst-port=""
url="cgi-bin \?" method=any action=deny
[admin@MikroTik] ip web-proxy cache>
This rule defines, that all runtime generated pages (which are located within
cgi-bin directories or contain '?' in url) has not to be cached.
Objects, which are larger than max-object-size, are not cached.
Command name : /ip web-proxy clear-cache
Web proxy will automatically detect any problems with cache and will try
to solve them without loosing any cache data. But in case of a heavy damage to the file system,
the web proxy can't rebuild cache data. Cache can be deleted and new cache directories
created using the command.
[admin@MikroTik] ip web-proxy> clear-cache
Clear all web proxy cache, yes? [y/N]: y
cache will be cleared shortly
[admin@MikroTik] ip web-proxy>
For web proxy setup, do the following:
- Specify at least one dns server for the router:
/ip dns set primary-dns=159.148.60.2
- Set IP address and port on which proxy will listen for requests:
/ip web-proxy set address=0.0.0.0:8080
- If this proxy has to use another proxy, specify it:
/ip web-proxy set parent-proxy=192.168.1.1:8080
otherwise disable it:
/ip web-proxy set parent-proxy=0.0.0.0:0
- Specify cache administrator's e-mail address:
/ip web-proxy set cache-administrator=support@mt.lv
- Specify hostname (DNS or IP address) of the web proxy:
/ip web-proxy set hostname=proxy.mt.lv
- Allow access to web proxy oly from our network (for example, 192.168.0.0/16):
/ip web-proxy access add src-address=!192.168.0.0/16 action=deny
- Enable the proxy service:
/ip web-proxy set enabled=yes
Now it is possible to use this proxy, by setting it as a proxy for you web browser.
To enable the transparent mode, firewall rule in destination nat has to be added, specifying which connections
(to which ports) should be transparently redirected to the proxy.
For example, we have the following web-proxy settings:
[admin@MikroTik] ip web-proxy> print
enabled: yes
address: 0.0.0.0:8080
hostname: "proxy.mt.lv"
transparent-proxy: yes
parent-proxy: 10.5.5.1:8080
cache-administrator: "support@mt.lv"
max-object-size: 10000 kB
max-cache-size: unlimited
status: running
reserved-for-cache: 2633728 kB
[admin@MikroTik] ip web-proxy>
If we want all connections coming from interface ether1 and going
to port 80 to handle with web proxy transparently, and if our web proxy is
listening on port 8080, then we add following destination nat rule:
[admin@MikroTik] ip firewall dst-nat> add in-interface=ether1 protocol=tcp \
dst-address=!10.0.0.1/32:80 action=redirect to-dst-port=8080
[admin@MikroTik] ip firewall dst-nat> print
Flags: X - disabled, I - invalid
0 src-address=0.0.0.0/0:0-65535 in-interface=ether1
dst-address=!10.0.0.1/32:80 protocol=tcp icmp-options=any:any flow=""
src-mac-address=00:00:00:00:00:00 limit-count=0 limit-burst=0
limit-time=0s action=redirect to-dst-address=0.0.0.0 to-dst-port=8080
[admin@MikroTik] ip firewall dst-nat>
Here, the router's address and port 80 (10.0.0.1/32:80) have been excluded from
redirection to preserve the winbox functionality which uses TCP port 80 on the router.
More than one redirect rule can be added to redirect more than one port.
Note: only HTTP traffic is supported by web proxy transparent mode.
HTTPS and FTP are not going to work this way!
- When I turned on transparent proxy and redirected TCP port 80 to it, my WinBox stopped working.
TCP port 80 is used by WinBox when connecting to the router.
You should exclude the router's address:80 from redirection by using rule
/ip firewall dst-nat add dst-address=address/32:80 protocol=tcp action=accept
BEFORE the redirect rule. Alternatively, you can use just one rule
/ip firewall dst-nat add dst-address=!address/32:80 protocol=tcp action=redirect to-dst-port=8080
- I use firewall to block access to the router from the Internet. My proxy does not work.
Make sure you allow established TCP connections with tcp option non-syn-only
to the router before blocking everything else. The rule is like this:
/ip firewall rule input add protocol=tcp tcp-options=non-syn-only connection-state=established
© Copyright 1999-2002, MikroTik