Hi guys

Anyone know of a maintained porn site block list as well as a virus port list?

Thanks.

Keeping up with all the porn sites could be a full-time job.

I was just hoping there was a database around but I'm going to try and use the one at urlblacklist.com and see if I can manipulate it to my own need. Thanks.

Hilton,

You can use Linux Squid proxy with Dansguardian in your network as a Parent proxy.
It will automatically keep updatating the Blacklist database from internet.

To search Blacklisted website and configure in MikroTik...will take toooooooooo much TIME.

Hilton,

You can use Linux Squid proxy with Dansguardian in your network as a Parent proxy.

This sounds good but can I still then apply QoS (like bandwidth queues) for the users on my LANs?

You can always have a Bandwidth Management with your existing Mikrotik.

But how would you manage bandwidth queues on the mikrotik if the IP address that is accessing the internet is always that of the proxy server?

I've never got my head around this one.

It seems to be either proxy your users or use queues but not both.

If someone can correct me I'd be very happy.

Hence my need to block stuff in the internal web proxy.

Configure MikroTik as a Transparent Proxy server, I mean the Main Router for Bandwidth Distribution where you can configure Qos.

Configure Linux Squid proxy with DansGurdian as a Parent Proxy.

You can always define Linux Squid proxy as a Parent proxy in Mikrotik proxy.

The connectivity will be

INTERNET-->LINUX-->MIKROTIK-->LAN

Dansgurdian = Black listed Website
MikroTik = Bandwidth Management

Squid box can work on transparent bridge which passes the original input ip to output interface (so-called spoofing) using the tproxy method. We have few thousands of customers passing to the internet this way (LAN->SQUID->MT->INTERNET).

Squid box can work on transparent bridge which passes the original input ip to output interface (so-called spoofing) using the tproxy method. We have few thousands of customers passing to the internet this way (LAN->SQUID->MT->INTERNET).

but does this allow you to enforce simple queues as well to manage bandwidth? I'm talking about users on a LAN here.

Squid box can work on transparent bridge which passes the original input ip to output interface (so-called spoofing) using the tproxy method. We have few thousands of customers passing to the internet this way (LAN->SQUID->MT->INTERNET).

but does this allow you to enforce simple queues as well to manage bandwidth? I'm talking about users on a LAN here.

It's the transparent bridge, so you get on out interface exactly the same as you get on input (if don't include caching). The ip addresses from input interface on transparent bridge goes to the output interface unchanged (same ip), so the simple queue is possible.
You can choose to put it in front of the router or in the back and have slight different traffic results due to the caching (if you have squid in front of the router the client might have slight increase in bandwidth due to caching, if you put it in back you have normal traffic limiting but slight decrease in needed bandwidth from internet due to the cache).

Example, assume that squid saves 10% of total bandwidth and client is limited to 256kbit simple queue and the squid is in front:
CLIENT 1 -> SQUID -> MT -> INTERNET
281Kbit -> 256 Kbit -> 256kbit -> 256kbit

Example 2, assume that squid saves 10% of total bandwidth and client is limited to 256kbit simple queue and the squid is in back:
CLIENT 1 -> MT -> SQUID -> INTERNET
256Kbit -> 256 Kbit -> 230kbit -> 230kbit

One note: This configuration on squid is possible (called tproxy) but it's special and may be hard to setup.

It's the transparent bridge, so you get on out interface exactly the same as you get on input (if don't include caching). The ip addresses from input interface on transparent bridge goes to the output interface unchanged (same ip), so the simple queue is possible.

Example, assume that squid saves 10% of total bandwidth and client is limited to 256kbit simple queue and the squid is in front:
CLIENT 1 -> SQUID -> MT -> INTERNET
281Kbit -> 256 Kbit -> 256kbit -> 256kbit

抱歉Dragonmen trying to get my head around this. This example of yours is exactly what I want to achieve but with squid authentication (so I can get reports on usage per user). I also want to be able to set bandwidth limits per user (like those using Internet banking to have higher throughput). To get this all to work, I need to use the internal proxy redirect because it uses the transparent bridge (confused about this, packet flow diagram is a bit tricky)?

Am I on the right track?

Sorry, Hilton, you can't use proxy authentication in transparent mode.
For the bandwidth usage you can use some of the squid log analyzers to see how much traffic uses which ip address, if that satisfies you. This logs (access logs) can be quite huge, especially if you have many clients.
For the bandiwdth limiting you can use squid's delay pools which i don't recommend - use mikrotik for bandwidth limiting.
You can use mikrotik to auth the users and limit their bandwidth and squid just for caching.

For the bridge on the squid box here's the example how it works:
192.168.1.5:5000<----linux bridge----->92.66.12.8:25
192.168.1.115:665<-----linux bridge---->72.66.21.99:1221
without squid (web request on port 80):
192.168.1.221:8112<-----linux bridge---->16.84.12.68:80
with squid on bridge:
192.168.1.221:8112<------ (port 80 is redirected to squid port) squid (192.168.1.2)----->192.168.1.221:8112 (tproxy spoofed ip)<----->16.84.12.68:80

use opendns

Sorry, Hilton, you can't use proxy authentication in transparent mode.
For the bandwidth usage you can use some of the squid log analyzers to see how much traffic uses which ip address, if that satisfies you.
You can use mikrotik to auth the users and limit their bandwidth and squid just for caching.

Thanks Dragonmen, I will try this and be rest assured, will then bug you when I come unstuck :-)

You mean bug me when you stuck?
As i already said you might have alot of problems with setting up tproxy, but it really works for me and i don't have side effects like some other ppl.

Anyway, i advise you follow this tutorial:
http://fuzzylab00net.blog.dada.net/post ... entoo.html
It helped me.
Be very carfull with kernel and tproxy version. Not all tproxy patch can work on all kernels and not all squid versions can work on all tproxy versions. Use the same as in that tutorial.

Thanks Dragonmen.