how i can identify which process is making UDP traffic on linux ?

Hello,

my machine is continously making udp dns traffic request. what i need to know is the PID of the process generating this traffic.

The normal way in TCP connection is to use netstat/lsof and get the process associated at the pid.

Is UDP the connection is stateles, so, when i call netastat/lsof i can see it only if the UDP socket is opened and it's sending traffic.

I have tried with lsof -i UDP and with nestat -anpue but i cant be able to find wich process is doing that request because i need to call lsof/netstat exactly when the udp traffic is sended, if i call lsof/netstat before/after the udp datagram is sended is impossible to view the opened UDP socket.

call netstat/lsof exactly when 3/4 udp packet is sended is IMPOSSIBLE.

how i can identify the infamous process ? I have already inspected the traffic to try to identify the sended PID from the content of the packet, but is not possible to identify it from the contect of the traffic.

anyone can help me ?

I'm root on this machine FEDORA 12 Linux noise.digint.lan 2.6.32.16-141.fc12.x86_64 #1 SMP Wed Jul 7 04:49:59 UTC 2010 x86_64 x86_64 x86_64 GNU/Linux

From serverfault boos

• I would use a net-sniffer like tcpdump or wireshark to view the DNS requests. The contents of the query can give an idea of what program is issuing them.

  boos : no information in the traffic sniffed i just already view.

  RedGrittyBrick : No info? Empty packets? What I meant was, if something is attempting to resolve updates.java.sun.com or rss.cnn.com you might usefully infer something from it.

  boos : dns query searching for the internal proxy. right now i found wich process is, but the question stil alive for a general problem solving technics
  From RedGrittyBrick

• You can use netstat, but you need the right flags, and it only works if the process that is sending the data is still alive. It won't find the traces of something that came briefly to life, sent UDP traffic, then went away. It also requires local root privileges. That said:

  Here's me starting an ncat on my local host, sending UDP traffic to port 2345 on a (non-existent) machine 10.11.12.13:

  [madhatta@risby]$ ncat -u 10.11.12.13 2345 < /dev/urandom
  

  Here's some tcpdump output proving that the traffic is going:

  [root@risby ~]# tcpdump -n -n port 2345
  tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
  listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
  12:41:32.391750 IP 192.168.3.11.57550 > 10.11.12.13.2345: UDP, length 8192
  12:41:32.399723 IP 192.168.3.11.57550 > 10.11.12.13.2345: UDP, length 8192
  12:41:32.401817 IP 192.168.3.11.57550 > 10.11.12.13.2345: UDP, length 8192
  12:41:32.407051 IP 192.168.3.11.57550 > 10.11.12.13.2345: UDP, length 8192
  12:41:32.413492 IP 192.168.3.11.57550 > 10.11.12.13.2345: UDP, length 8192
  12:41:32.417417 IP 192.168.3.11.57550 > 10.11.12.13.2345: UDP, length 8192
  

  Here's the useful bit, using netstat with the -a flag (to see port details) and the -p flag to see process ID details. It's the -p flag that requires root privileges:

  [root@risby ~]# netstat -apn|grep -w 2345
  udp        0      0 192.168.3.11:57550          10.11.12.13:2345            ESTABLISHED 9152/ncat     
  

  As you can see, pid 9152 is fingered as having a connection open to port 2345 on the specified remote host. Netstat helpfully also runs that through ps and tells me the process name is ncat.

  Hopefully that's of some use.

  ThorstenS : really nice done! :thumbup:

  zerolagtime : There's a catch though. If the problem is caused by a shell script spawning a subprocess that does the DNS lookup and that process quickly exits, then the source port (57550 above) will change all the time. In this case, the technique will not work and you'll have to take more drastic measures. Additionally, your netstat should have done a `grep -w 57550` because multiple processes could be doing DNS lookups to the same server. Your method would not distinguish them.

  MadHatter : I agree with both of your objections, zerolagtime (but thanks for your kind words anyway, ThorstenS!).
  From MadHatter

• Linux auditing can help. It will at least locate users and processes making datagram network connections. UDP packets are datagrams.

  First, install the auditd framework on your platform and ensure that auditctl -l returns something, even if it says that no rules are defined.

  Then, add a rule to watch the system call socket() and tag it for easy finding later (-k). I need to assume that you are on a 64-bit architecture, but you can substitute b32 in place of the b64 if you aren't.

  auditctl -a exit,always -F arch=b64 -F a0=2 -F a1=2 -S socket -k SOCKET
  

  You have to pick through man pages and header files to build this, but what it captures is essentially this system call: socket(PF_INET, SOCK_DGRAM, X) where the 3rd parameter is unspecified, but frequently zero. PF_INET is 2 and SOCK_DGRAM is 2. TCP connections would use SOCK_STREAM which would set a1=1. The -k SOCKET is our keyword we want to use when searching audit trails later. It can be anything, but I like to keep it simple.

  Let a few moments go by and review the audit trails. Optionally, you could force a couple of packets by pinging a host out on the net, which will cause a DNS lookup to occur, which uses UDP, which should trip our audit alert.

  ausearch -i -ts today -k SOCKET
  

  And output similar to the section below will appear. I'm abbreviating it to highlight the important parts

  type=SYSCALL ... arch=x86_64 syscall=socket success=yes exit=1 a0=2 a1=2 ... pid=14510 ... auid=zlagtime uid=zlagtime ... euid=zlagtime ... comm=ping exe=/usr/bin/ping key=SOCKET
  

  In the above output, we can see that the ping command caused the socket to be opened. I could then run strace -p 14510 on the process, if it was still running. The ppid (parent process ID) is also listed in case it is a script that spawns the problem child a lot.

  Now, if you have a lot of UDP traffic, this isn't going to be good enough and you'll have to resort to OProfile or SystemTap, both of which are currently beyond my expertise.

  This should help narrow things down in the general case.

  When you are done, remove the audit rule by using the same line you used to create it, only substitute -a with -d.

  auditctl -d exit,always -F arch=b64 -F a0=2 -F a1=2 -S socket -k SOCKET
  

  boos : i will try it, but i think is the right answer.
  From zerolagtime