-
-
hakhunamathata
刚刚加入了
- Posts: 2
- Joined:
How to Save Browsing history of clients through Mikrotik
Hello everybody, I hope you all doing well. I have a question to ask. I am using Mikro Tik Model - RB1100AHx2 , OS Level is 6 . How do I save or monitor my client's browsing history by a remote computer? I have already downloaded MT_Syslog software on my pc. Waiting for a suitable answer. Gimme the easiest way please because I am just beginner, Thanks in advance.
Re: How to Save Browsing history of clients through Mikrotik
You can log SYN packets:
and send log to remote machine:
where 1 it's a info level.
Define remote action
where remote it's a syslog server src-address - local interface
Then define rule on your syslog server:
You can also rotate this logs:
Code:Select all
/ip firewall filter add chain=forward action=log tcp-flags=syn protocol=tcp src-address=192.168.1.0/24 log=yes log-prefix="SYN-FORWARD"
Code:Select all
/system logging set 1 action=remoteDefine remote action
Code:Select all
/system logging action add name="remote" target=remote remote=192.168.1.1 remote-port=514 src-address=192.168.1.2 bsd-syslog=no syslog-time-format=bsd-syslog syslog-facility=daemon syslog-severity=autoThen define rule on your syslog server:
Code:Select all
#猫/etc/syslog-ng/conf.d/02-mikr雷竞技网站otik.conf源头e s_net { udp (); }; filter f_mikrotik_syn { host( "192.168.1.2" ); message("SYN-FORWARD");}; destination df_mikrotik_syn { file("/home/mikrotik/syn/${YEAR}.${MONTH}.${DAY}.log"); }; log { source ( s_net ); filter( f_mikrotik_syn ); destination ( df_mikrotik_syn ); };
Code:Select all
#!/bin/bash set -e renice 19 $$ > /dev/null BASE=/home/mikrotik/ DATA=`date -d "yesterday 13:00 " '+%Y.%m.%d'` DIRS=(syn) for i in ${DIRS[*]}; do if [ \! -f ${BASE}${i}/${DATA}.log ]; then echo "No logfile ${BASE}${i}/${DATA}.log" exit 1; fi gzip -c -9 ${BASE}${i}/${DATA}.log > ${BASE}${i}/${DATA}.log.gz chown mtlog:mtlog ${BASE}${i}/${DATA}.log.gz rm ${BASE}${i}/${DATA}.log done /etc/init.d/syslog-ng reload | grep -v 'Reload system logging: syslog-ng.'Re: How to Save Browsing history of clients through Mikrotik
Great, thank you for the info!
I would like to point that this firewall rule need to be moved to the top in order for this info to be captured.
I have couple of simple questions:
1. What is "syn" exactly? Does it capture ANY activity? Basically any TCP packets sent through router will be captured, right? HTTPS/VPN will be captured as well? I want to be able to tell if someone connects and uses encrypted VPN
2. I get logs like this. From here I can tell how packets were flowing.
3. I got all this info in SQL database. Now I can parse out all pieces. I know local IPs and MACs so it's easy to detect local clients. For external ones, I need to do reverse DNS lookups, is that something easily done?
My logic to get history report is to collect data for a day, for example.
1. Parse out every record, detect from/to
2. Eliminate "duplicates" - for example by timestamp if there is multiple packets within a minute, I can just leave 1 row
3. Get unique IPs and do reverse DNS lookups to see remote host names.
Then, for each individual internal host I will be able to display history.
Does this sound correct?
I would like to point that this firewall rule need to be moved to the top in order for this info to be captured.
I have couple of simple questions:
1. What is "syn" exactly? Does it capture ANY activity? Basically any TCP packets sent through router will be captured, right? HTTPS/VPN will be captured as well? I want to be able to tell if someone connects and uses encrypted VPN
2. I get logs like this. From here I can tell how packets were flowing.
SYN-FORWARD forward: in:ether1-gateway out:bridge-local, src-mac a4:7a:a4:2e:af:50, proto TCP (SYN,ACK), 157.240.2.20:443->192.168.33.106:50785, NAT 157.240.2.20:443->(23.114.87.129:50785->192.168.33.106:50785), len 60
3. I got all this info in SQL database. Now I can parse out all pieces. I know local IPs and MACs so it's easy to detect local clients. For external ones, I need to do reverse DNS lookups, is that something easily done?
My logic to get history report is to collect data for a day, for example.
1. Parse out every record, detect from/to
2. Eliminate "duplicates" - for example by timestamp if there is multiple packets within a minute, I can just leave 1 row
3. Get unique IPs and do reverse DNS lookups to see remote host names.
Then, for each individual internal host I will be able to display history.
Does this sound correct?
Re: How to Save Browsing history of clients through Mikrotik
Not in all cases. Imagine an DOS from one of your clients.Great, thank you for the info!
I would like to point that this firewall rule need to be moved to the top in order for this info to be captured.
In short - SYN it's a init package, please look at three way hanhdshakehttp://www.inetdaemon.com/tutorials/int ... hake.shtml. Any activity before two hosts starting from SYN package.I have couple of simple questions:
1. What is "syn" exactly? Does it capture ANY activity? Basically any TCP packets sent through router will be captured, right? HTTPS/VPN will be captured as well? I want to be able to tell if someone connects and uses encrypted VPN
And it's enough - all information about this connection are logged.
2. I get logs like this. From here I can tell how packets were flowing.SYN-FORWARD forward: in:ether1-gateway out:bridge-local, src-mac a4:7a:a4:2e:af:50, proto TCP (SYN,ACK), 157.240.2.20:443->192.168.33.106:50785, NAT 157.240.2.20:443->(23.114.87.129:50785->192.168.33.106:50785), len 60
First - there is no duplicated entries. If you have two lines with the same content one by one - there was two connections. Second I think it's a bad idea to make it on the fly. There is a really lot of information, in my case 18095665 lines, 3 GB log file (~ 20 hours, ~ 500 users). Third - revdns It's useless in this case. For example in my case google.com point to:3. I got all this info in SQL database. Now I can parse out all pieces. I know local IPs and MACs so it's easy to detect local clients. For external ones, I need to do reverse DNS lookups, is that something easily done?
My logic to get history report is to collect data for a day, for example.
1. Parse out every record, detect from/to
2. Eliminate "duplicates" - for example by timestamp if there is multiple packets within a minute, I can just leave 1 row
3. Get unique IPs and do reverse DNS lookups to see remote host names.
Then, for each individual internal host I will be able to display history.
Does this sound correct?
Code:Select all
$ host google.com google.com has address 216.58.209.78 $ host 216.58.209.78 78.209.58.216.in-addr.arpa domain name pointer waw02s06-in-f14.1e100.net. 78.209.58.216.in-addr.arpa domain name pointer waw02s06-in-f78.1e100.net. 78.209.58.216.in-addr.arpa domain name pointer waw02s06-in-f14.1e100.net. 78.209.58.216.in-addr.arpa domain name pointer waw02s06-in-f78.1e100.net.Re: How to Save Browsing history of clients through Mikrotik
Where should I put it? At the end (where script put's it) - there is nothing in logNot in all cases. Imagine an DOS from one of your clients.
Good to know. But even if it's 2 connections, why should I care? Browser can open multiple connections to download pictures, etc.First - there is no duplicated entries. If you have two lines with the same content one by one - there was two connections.
I think I should clean it up a little, tell me where I might be wrong?
Oh no, I'm not making it on a fly. I'm only parsing out pieces from a string and store them into DB. This part is done and yes, there is a lot!Second I think it's a bad idea to make it on the fly. There is a really lot of information, in my case 18095665 lines, 3 GB log file (~ 20 hours, ~ 500 users). Third - revdns It's useless in this case. For If you have an ip address just use whois. Store logs in file, and find needed informations when it's necessary.
Then, at the end of day I envision running some job to do post-processing. SQL script to get rid of duplicates. Then get DISTINCT external IPs and then do revdns on those. And I can also cache those in DB so I don't have to do it all over every day.
What about whois? Is there any programmatic way to pull this data?
ALSO! I guess it's important to make it even "better". If HTTP traffic - it would be nice to collect actual URLs which is not visible from TCP packets logging. I get it with secured protocols we can't but with regular HTTP traffic it would be nice what is being done.
Re: How to Save Browsing history of clients through Mikrotik
In my rules right behind antiddos/limits/connlimit rules.Where should I put it? At the end (where script put's it) - there is nothing in logNot in all cases. Imagine an DOS from one of your clients.
如果你想日志all conenctions - yes you are wrong. If you only want to know about connection with some host - you can filter logs.Good to know. But even if it's 2 connections, why should I care? Browser can open multiple connections to download pictures, etc.First - there is no duplicated entries. If you have two lines with the same content one by one - there was two connections.
I think I should clean it up a little, tell me where I might be wrong?
http://bfy.tw/9YYiOh no, I'm not making it on a fly. I'm only parsing out pieces from a string and store them into DB. This part is done and yes, there is a lot!Second I think it's a bad idea to make it on the fly. There is a really lot of information, in my case 18095665 lines, 3 GB log file (~ 20 hours, ~ 500 users). Third - revdns It's useless in this case. For If you have an ip address just use whois. Store logs in file, and find needed informations when it's necessary.
Then, at the end of day I envision running some job to do post-processing. SQL script to get rid of duplicates. Then get DISTINCT external IPs and then do revdns on those. And I can also cache those in DB so I don't have to do it all over every day.
What about whois? Is there any programmatic way to pull this data?
For example in linux systems you can do this:
Code:Select all
$ whois 216.58.209.78 # # ARIN WHOIS data and services are subject to the Terms of Use # available at: https://www.arin.net/whois_tou.html # # If you see inaccuracies in the results, please report at # https://www.arin.net/public/whoisinaccuracy/index.xhtml # # # The following results may also be obtained via: # https://whois.arin.net/rest/nets;q=216.58.209.78?showDetails=true&showARIN=false&showNonArinTopLevelNet=false&ext=netref2 # NetRange: 216.58.192.0 - 216.58.223.255 CIDR: 216.58.192.0/19 NetName: GOOGLE NetHandle: NET-216-58-192-0-1 Parent: NET216 (NET-216-0-0-0-0) NetType: Direct Allocation OriginAS: AS15169 Organization: Google Inc. (GOGL) RegDate: 2012-01-27 Updated: 2012-01-27 Ref: https://whois.arin.net/rest/net/NET-216-58-192-0-1 OrgName: Google Inc. OrgId: GOGL Address: 1600 Amphitheatre Parkway City: Mountain View StateProv: CA PostalCode: 94043 Country: US RegDate: 2000-03-30 Updated: 2015-11-06 Ref: https://whois.arin.net/rest/org/GOGL OrgTechHandle: ZG39-ARIN OrgTechName: Google Inc OrgTechPhone: +1-650-253-0000 OrgTechEmail: arin-contact@google.com OrgTechRef: https://whois.arin.net/rest/poc/ZG39-ARIN OrgAbuseHandle: ABUSE5250-ARIN OrgAbuseName: Abuse OrgAbusePhone: +1-650-253-0000 OrgAbuseEmail: network-abuse@google.com OrgAbuseRef: https://whois.arin.net/rest/poc/ABUSE5250-ARIN # # ARIN WHOIS data and services are subject to the Terms of Use # available at: https://www.arin.net/whois_tou.html # # If you see inaccuracies in the results, please report at # https://www.arin.net/public/whoisinaccuracy/index.xhtml #