The problem was first discovered when a Netflow collector started having resource problems. Due to the massive number of SYN packets in the DoS attack, Netflow was creating a new flow in memory for each new SYN packet, and having to remember about it until the flow timed out. Not good! After having figured out what was going on, I was able to capture a sample of traffic and start working out where it was coming from.
Here is a small sample, with the destinations partially obscured:
25771 11:48:32.584857 136.102.253.14 -> aaa.bbb.38.140 TCP 51515 > 22 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25772 11:48:32.584866 110.93.51.90 -> aaa.bbb.38.140 TCP 27433 > 22 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25773 11:48:32.584868 177.88.230.116 -> aaa.bbb.37.10 TCP 3177 > 80 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25774 11:48:32.584876 130.136.233.47 -> aaa.bbb.37.10 TCP 21115 > 80 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25775 11:48:32.584889 174.230.64.48 -> aaa.bbb.37.10 TCP 27701 > 80 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25776 11:48:32.584896 73.217.102.69 -> aaa.bbb.38.140 TCP 43969 > 22 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25777 11:48:32.584902 221.233.146.91 -> aaa.bbb.37.10 TCP 42246 > 80 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25778 11:48:32.584910 123.83.83.9 -> aaa.bbb.38.140 TCP 40649 > 22 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25779 11:48:32.584915 140.101.194.71 -> aaa.bbb.37.10 TCP 1129 > 80 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25780 11:48:32.584922 193.82.83.24 -> aaa.bbb.38.140 TCP 39930 > 22 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25781 11:48:32.584929 177.36.15.25 -> aaa.bbb.37.10 TCP 43185 > 80 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25782 11:48:32.584936 148.154.218.100 -> aaa.bbb.38.140 TCP 4283 > 22 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25783 11:48:32.584942 140.41.59.48 -> aaa.bbb.37.10 TCP 47192 > 80 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25784 11:48:32.584950 89.134.150.35 -> aaa.bbb.38.140 TCP 25345 > 22 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25785 11:48:32.584955 207.236.20.32 -> aaa.bbb.37.10 TCP 22479 > 80 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25786 11:48:32.584963 240.16.136.1 -> aaa.bbb.37.10 TCP 44095 > 80 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25787 11:48:32.584970 70.40.185.17 -> aaa.bbb.38.140 TCP 46254 > 22 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25788 11:48:32.584975 82.195.26.42 -> aaa.bbb.37.10 TCP 61179 > 80 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25789 11:48:32.584983 130.184.250.86 -> aaa.bbb.38.140 TCP 15603 > 22 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25790 11:48:32.584989 62.108.248.77 -> aaa.bbb.37.10 TCP 61996 > 80 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25791 11:48:32.584996 244.193.231.42 -> aaa.bbb.38.140 TCP 27477 > 22 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25792 11:48:32.585003 254.119.67.64 -> aaa.bbb.37.10 TCP 46181 > 80 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25793 11:48:32.585009 171.252.91.108 -> aaa.bbb.38.140 TCP 22252 > 22 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25794 11:48:32.585016 61.79.106.102 -> aaa.bbb.37.10 TCP 41133 > 80 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25795 11:48:32.585023 142.113.79.114 -> aaa.bbb.38.140 TCP 32631 > 22 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25796 11:48:32.585029 99.7.239.111 -> aaa.bbb.37.10 TCP 29999 > 80 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25797 11:48:32.585037 5.107.140.57 -> aaa.bbb.38.140 TCP 2415 > 22 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25798 11:48:32.585043 16.197.131.47 -> aaa.bbb.37.10 TCP 21038 > 80 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25799 11:48:32.585050 109.10.184.48 -> aaa.bbb.38.140 TCP 41172 > 22 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25800 11:48:32.585057 183.145.182.22 -> aaa.bbb.37.10 TCP 7899 > 80 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25801 11:48:32.585063 116.183.190.106 -> aaa.bbb.38.140 TCP 18717 > 22 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25802 11:48:32.585070 52.217.83.29 -> aaa.bbb.37.10 TCP 32629 > 80 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25803 11:48:32.585077 31.215.117.74 -> aaa.bbb.38.140 TCP 19412 > 22 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25804 11:48:32.585084 48.113.61.11 -> aaa.bbb.37.10 TCP 25432 > 80 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25805 11:48:32.585090 161.46.43.79 -> aaa.bbb.38.140 TCP 35808 > 22 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25806 11:48:32.585097 232.129.112.88 -> aaa.bbb.37.10 TCP 1616 > 80 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25807 11:48:32.585103 152.25.15.45 -> aaa.bbb.38.140 TCP 10276 > 22 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25808 11:48:32.585110 133.225.67.50 -> aaa.bbb.37.10 TCP 43895 > 80 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25809 11:48:32.585116 19.124.117.101 -> aaa.bbb.38.140 TCP 52242 > 22 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25810 11:48:32.585123 134.83.167.21 -> aaa.bbb.37.10 TCP 16292 > 80 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25811 11:48:32.585129 228.41.163.17 -> aaa.bbb.38.140 TCP 8024 > 22 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25812 11:48:32.585135 53.115.220.107 -> aaa.bbb.37.10 TCP 46765 > 80 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25813 11:48:32.585142 142.7.225.83 -> aaa.bbb.38.140 TCP 32227 > 22 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25814 11:48:32.585151 68.104.42.36 -> aaa.bbb.37.10 TCP 63759 > 80 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25815 11:48:32.585156 142.113.105.51 -> aaa.bbb.38.140 TCP 23125 > 22 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
25816 11:48:32.585169 163.167.156.8 -> aaa.bbb.38.140 TCP 37433 > 22 [SYN] Seq=0 Win=65535 Len=0 MSS=1460
So, a very large number of packets! The main things to notice are that they are coming from a massive variety of source addresses (none of which are valid for a packet going out of this network) and that they are destined for only two hosts. These two hosts are the targets, and the packets are being sent with spoofed source addresses, which makes tracking down the actual source quite challenging!
I was able to track down the source of the flows by adding a simple ACL (blocking packets to the two desinations) to different routers in the network, and seeing which ones scored the most hits. For example, this ACL was applied:
ip access-list extended Temp
deny ip any host aaa.bbb.37.10
permit ip any any
and then when reviewed a little while later:
# show access-list
Extended IP access list Temp
10 deny ip any host aaa.bbb.37.10 (2029998454 matches)
20 permit ip any any (42805196 matches)
Using this method I was quickly able to identify roughly where in the network the packets were coming from, but, to cut a long story short, it took a while to identify the source router.
From there, it was a case of identifying the sending host on the LAN. This can be done with mac-address accounting:
Configure:
(config-if)# ip accounting mac-address input
Check:
#sh interface F0/0 mac-accounting
FastEthernet0/0
Input (509 free)
00aa.ccf2.7c51(153): 5 packets, 411 bytes, last: 1380ms ago
00aa.ddcd.4a27(163): 1 packets, 103 bytes, last: 9236ms ago
00bb.ee85.2569(192): 6 packets, 1573 bytes, last: 4580ms ago
Total: 12 packets, 2087 bytes
Unfortunately the above output was an afterthought, and I don't have the "live" version. However, it quickly gave me the MAC of the offending host.
Due to the vast traffic flood, I had to wait until the next day before I could get physically on the console of the hacked PC and do some more digging!
More to follow very soon...
ReplyDeleteHpe JZ356A PriceCWS XXA
ReplyDeleteHpe Q9H62A Price DCADX