Wednesday, December 10, 2008

Open source caching server

AOA
Contents
[hide]

* 1 How to use external open source caching server
* 2 Mikrotik with 3 interfaces
* 3 Squid
* 4 Squid Internet access is working
* 5 Time to route clients traffic to address 192.168.100.1 with clients on IP's with 192.168.0.0/23 range
* 6 Complete final view

[edit] How to use external open source caching server

Sometimes admin want to control users HTTP requests through custmized ACL's or other scripts.

Mikrotik can be used as PPPoE server where all PPPoE or clients on 192.168.0.0/23 network are connected with Mikrotik directly through LAN.


[edit] Mikrotik with 3 interfaces

1. LAN (for clients)
2. WAN (for Internet)
3. eth1 (for Squid)



Mikrotik simple view





[edit] Squid

Squid have 2 type of networks

1. LAN IP Address 192.168.100.1/24 without any gateway or route address.

Just for LAN connectivity with Mikrotik eth1 with 192.168.100.2/24 IP Address

2. PPPoE conection for Internet throw Mikrotik 'but' different IP Address 192.168.12.2 (normal clients IP address 192.168.0.0/23)



Internet setup for Squid




[edit] Squid Internet access is working


[edit] Time to route clients traffic to address 192.168.100.1 with clients on IP's with 192.168.0.0/23 range

Disable NAT ( IP / Firewall / NAT / masquerade srcnat 192.168.0.0/23

When normal client request for Internet to Mikrotik then Mikrotik all LAN network 192.168.0.0/23 passthrow to Squid 192.168.100.1 direct without any delay.






Example: Any request from 192.168.0.0/23 network (with routing mark client), Mikrotik route this request to 192.168.100.1, and if another network request then Mikrotik route this request to WAN route 192.168.11.1.



[edit] Complete final view

NAT Tutorial

MIKROTIK NAT

This is a short howto explaining how to set up a full-NAT on a Mikrotik RouterOS.

This setup allows you to hide (masquerade) your private IP address from a public network. This means, for example, that in your private network you can have whatever private IP you want which is then in turn translated to the public network IP given to you by your network provider. This tutorial can thus be used by clients who want to connect to a network without requiring a change to the internal IP addressing of their LAN.
[edit] Example

In the following example we have a wireless interface which connects to a public wireless network and an Ethernet interface for the local private network.

Suppose we have 192.168.1.2/24 (Internal Private Network)
Your RouterBoard's Ethernet interface is 192.168.1.1/24 (fixed Private Network IP)
Your RouterBoard's Wireless card is 10.140.1.30/26 (Public Network)
The Access Point you connect to is 10.140.1.1/26 (Public Network gateway)

Image:Nat with 1 pc.jpg


* First set the two IP addresses of the ethernet and wireless interfaces

Image:Mikrotik nat howto01.jpg

Image:Mikrotik nat howto02.jpg


* Second set the static route so that we can access the external Public network

most likely you will want to use 0.0.0.0/0 as the destination in your primary public route----

Image:Mikrotik nat howto03.jpg

Image:Mikrotik nat howto04.jpg


* We start building the NAT as follows

Image:Mikrotik nat howto05.jpg

Image:Mikrotik nat howto06.jpg


* First the Destination Network Address Translation setting (DST-NAT)

Only do this if you want to expose this specifice internal pc to all ports, this is the same as a DMZ-----

Image:Mikrotik nat howto07.jpg

Image:Mikrotik nat howto08.jpg


* Then the Source Network Address Translation setting (SRC-NAT)

---On the Action Screen you could instead choose masquerade--- Image:Mikrotik nat howto09.jpg

Image:Mikrotik nat howto10.jpg


* Private to Public Network Address Translation (NAT) is Complete!

Image:Mikrotik nat howto11.jpg

How to block one.lv

Some times peoples dont work and go to social networks like one.lv. In this example I will show you how you can stop this.

Add new address list with name onelv. This list must be updated manually from ripe.net.Search vaule ONELIMITED-NET

/ip firewall address-list
add list=onelv address=62.85.54.0/26
add list=onelv address=62.85.117.0/26
add list=onelv address=62.85.33.64/28
add list=onelv address=62.85.70.224/27
add list=onelv address=62.85.70.192/27
add list=onelv address=62.85.126.0/24

Create address list with ip witch doesnt have access to one.lv with name blockonelv

/ip firewall address-list
add list=blockonelv address=192.168.0.2
add list=blockonelv address=192.168.0.3
add list=blockonelv address=192.168.0.4
add list=blockonelv address=192.168.0.5
add list=blockonelv address=192.168.0.6

And now you can create firewall rule witch blocks one.lv for these ips

/ip firewall filter
add chain=forward action=drop src-address-list=blockonelv dst-address-list=onelv protocol=tcp dst-port=80

NTH in RouterOS 3.x

In v3.0 it is a little different implementation of NTH. It has only two parameters 'every' and 'packet'.
[edit] How it works in v3.0

Every rule has its own counter. When rule receives packet counter for current rule is increased by one. If counter matches value of 'every' packet will be matched and counter will be set to zero.

If passthrough is not set then packets will be marked as follows:

* first rule nth=2,1 rule will match every first packet of 2, hence, 50% of all the traffic that is matched by the rules
* second rule if passthrough=no will match ONLY 25% of traffic because in 3.0 you need only one rule to catch traffic not like 2.9

[edit] Example

Now it is possible to match 50% of all traffic only with one rule:

/ip firewall mangle
add action=mark-packet chain=prerouting new-packet-mark=AAA nth=2,1;

If more than one rule is needed, then there are two ways to match packets:

* first rule sees all packets and matches 1/3 of all, second rule sees 2/3 of packets and matches 1/2, third rule sees and matches all packets that passed through first two rules ( 1/3 of all packets ).

/ip firewall mangle
add action=mark-packet chain=prerouting new-packet-mark=AAA nth=3,1 passthrough=no;
add action=mark-packet chain=prerouting new-packet-mark=BBB nth=2,1 passthrough=no;
add action=mark-packet chain=prerouting new-packet-mark=CCC ;

* all rules can see all packets and each rule matches every 3-rd packet.

/ip firewall mangle
add action=mark-packet chain=prerouting new-packet-mark=AAA nth=3,1 passthrough=yes;
add action=mark-packet chain=prerouting new-packet-mark=BBB nth=3,2 passthrough=yes;
add action=mark-packet chain=prerouting new-packet-mark=CCC nth=3,3 passthrough=yes;

Calea

Communications Assistance for Law Enforcement Act requires the routers in USA to have ability to intercept and log network traffic. RouterOS now provides this facility by means of firewall rules. RouterOS can also function as a data retention server if the additional calea package is installed.


Contents
[hide]

* 1 CALEA features included in RouterOS
* 2 CALEA-server package
* 3 Calea user
* 4 Intercepting Packet Flow
* 5 Data Retention Server
* 6 Calea Server/Client Configuration Example
o 6.1 Client Configuration for the Intercept
o 6.2 Calea Server Configuration

CALEA features included in RouterOS

Multiple subject/multiple destination packet interception and streaming in following formats:

* Call Content Connection (CCC) Interface according to PKT-SP-ES-DCI-I01-060914 (PacketCable 2.0 PacketCable Electronic Surveillance Delivery Function to Collection Function Interface Specification)
* Call Content Connection (CCC) Interface according to ANSI/SCTE 24-13 2006 (IPCalblecom Electronic Surveillance Standard) that is approved method for Communication Content delivery to LEA according to ATIS-1000013.2007 (Lawfully Authorized Electronic Surveillance For Internet Access and Services)
* TZSP format - for reception with 'Ethereal', tcpdump, trafr (sniffer stream reader for linux) - http://www.mikrotik.com/download.html

CALEA-server package

* accepts multiple CCC streams (identified by destination port/source address/case id)
* stores communication content according to "IP Network Access Intercept Requirements and Method"(FBI-WISPA draft) specified "full content" intercept requirements (without out-of-band events)
* stores communication content of multiple subjects/cases
* stores communication content in libpcap format
* new libpcap file based on different conditions (interval/size/packet count)
* generates hash for each pcap file (md5/sha1/sha256)

Calea user

Calea provided options are available only for specific RouterOS user, as Calea server configuration as "tap" configuration. Specific user should have 'sniff' policy enabled at RouterOS user configuration,

/ user group set 0 policy=sniff

sniff policy is enabled by default for "full" and "write" user groups.
Intercepting Packet Flow

The IP Firewall and Interface Bridge now have one additional section,

* firewall section to intercept packets that are going trough firewall

/ ip firewall calea

* bridge section to intercept packets that are going trough bridge

/ interface bridge calea

Firewall and Bridge Calea menus contain same actions and matchers as "ip firewall filter" and "interface bridge filter", new avalailble actions:

* sniff - generates a tzsp stream that can be directed to any Wireshark (Ethereal) server
* sniff-pc - generates a Packet Cable stream that can be directed to a MikroTik RouterOS system with the calea package installed

By selecting either action, the following options will be available:

* sniff-id (Packet Cable protocol only) - packet stream case ID, that can be used to differentiate between separate traffic sets (e.g., between different users; or between client traffic and server traffic)
* sniff-target - IP address of the data retention server
* sniff-target-port - UDP port that the data retention server is listening on

Data Retention Server

The calea package provides an additional tool menu - /tool calea, that allows to save certain incoming data streams to a file. The server will create separate files for each packet stream (one data file and one hash file, if configured). The files will not grow indefinitely, but rather util a certain limit, after which a new set of files will be created for that stream. The limit is specified in size and extent of time, whichever is reached first.

Add a rule with the following properties:

* case-id - case ID set by the intercepting router
* case-name - case name is set on server to specify the folder, where intercepted data is stored
* intercept-ip - IP address of the intercepting router (IP address to receive the stream from)
* intercept-port - UDP port to listen on (port to receive the stream on)
* action - storage format (only pcap for now)
* pcap-file-stop-interval - maximal interval between creating new fileset, if size limit is not reached earlier
* pcap-file-stop-size - maximal filesize, in KiB
* pcap-file-stop-count - maximal packet count
* pcap-file-hash-method - hashing algorithm (md5, sha1 or sha256) for the data file (saved once the data file is completed and closed); no file is created if set to none

Calea Server/Client Configuration Example

Let's assume the particular network configuration, we need to intercept data from 192.168.0.10 Wireless Client and send it to the Calea Server located on Local Network: Image:Calea123.png
Client Configuration for the Intercept

Wireless Client is connected to Access Point, data interception has to be performed on Access Point for the particular network design.

* We have requirement to capture all data from the user Wireless Client with IP address of 192.168.0.10 We have to add two rules to make the interception,

/ ip firewall calea add action=sniff-pc chain=forward sniff-id=100 sniff-target=10.9.1.250 sniff-target-port=5555 \
src-address=192.168.0.10
/ ip firewall calea add action=sniff-pc chain=forward sniff-id=100 sniff-target=10.9.1.250 sniff-target-port=5555 \
dst-address=192.168.0.10

All traffic going trough the router for specified src/dst addresses is intercepted and sent to Calea Server (sniff-target) with sniff-id=100

* Calea package is not required for intercepting host.

Calea Server Configuration

* Calea package is required for server.
* One rule is required to accept the data from the Access Point to receive all intercepted traffic from the Access Point,

/ tool calea add action=pcap intercept-port=5555 case-id=100 intercept-ip=192.168.0.254

Intercept-port and case-id should be equal on server and client side, intercept-ip is IP address of the intercepting router (Access Point).

* Calea server received information is available under 'file' menu.

L7

layer7-protocol is a method of looking for patterns in connections.

First, add Regexp strings to the protocols menu, to define strings you will be looking for.

/ip firewall layer7-protocol add=

Then, use the defined protocols in firewall:

/ip firewall filter add layer7-protocol=

RouterOS will look for these strings in all connections passing the firewall rule where you use this. As this is resource intensive, make sure to filter out all good traffic before it hits this rule.

You can download a script with a list of common programs here (only for RouterOS v3 RC6). Pattern libraries can be found on the layer7 project page and on the protocol wiki.

How to Block Websites & Stop Downloading Using Proxy

This example will explain you “How to Block Web Sites” & “How to Stop Downloading”. I have use Web-Proxy test Package.

First, Configure Proxy.

/ip proxy
enabled: yes
src-address: 0.0.0.0
port: 8080
parent-proxy: 0.0.0.0:0
cache-drive: system
cache-administrator: "webmaster"
max-disk-cache-size: none
max-ram-cache-size: none
cache-only-on-disk: no
maximal-client-connections: 1000
maximal-server-connections: 1000
max-object-size: 512KiB
max-fresh-time: 3d

Now, Make it Transparent

/ip firewall nat
chain=dstnat protocol=tcp dst-port=80 action=redirect to-ports=8080

Make sure that your proxy is NOT a Open Proxy

/ip firewall filter
chain=input in-interface= src-address=0.0.0.0/0 protocol=tcp dst-port=8080 action=drop

Now for Blocking Websites

/ip proxy access
dst-host=www.vansol27.com action=deny

It will block website http://www.vansol27.com, We can always block the same for different networks by giving src-address. It will block for particular source address.

We can also stop downloading files like.mp3, .exe, .dat, .avi,…etc.

/ip proxy access
path=*.exe action=deny
path=*.mp3 action=deny
path=*.zip action=deny
path=*.rar action=deny.

Try with this also

/ip proxy access
dst-host=:mail action=deny

This will block all the websites contain word “mail” in url.

Example: It will block www.hotmail.com, mail.yahoo.com, www.rediffmail.com

ENJOY BLOCKING…….