1

After hitting some performance issues with one of our servers, I observed a Tomcat8 process called "Silence" using 100% continuously. I tried to find information about it on Google, but searching for Tomcat and Silence doesn't show any meaningful results.

Here is what I've got in top:

PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND                                                                                                                                     
7362 tomcat8   20   0  239956   6688   1344 S 99.7  0.2   1679:33 Silence

Can someone help me understand what this "Silence" process is and how to fix this problem? Of course there is nothing called "Silence" in our project, so I guess it is something Tomcat related.

Patkos Csaba
  • 2,500
  • 1
  • 20
  • 15
  • 1
    I would suggest that if you don't know what this process is (Tomcat 8 does not come with a binary of that name as far as I can tell) and the server is only supposed to run software that is known to you, it would be prudent to at least _suspect_ that the server is compromised. See https://serverfault.com/questions/218005/how-do-i-deal-with-a-compromised-server – Kusalananda Feb 15 '18 at 06:38
  • Try running `ls -l /proc/7632/exe` as root to see what that process might be executing, or `ls -l /proc/7632/fd` to identify any files it has open. If any of those are marked ` (deleted)`, you can still use these links under `/proc/` to extract their content, as long as the process is still running and holding those files open. Like @Kusalananda, I suspect you've been hit by a coinminer or some other exploit that has got in through your Tomcat8. – telcoM Feb 15 '18 at 06:41
  • Also possibly related: https://unix.stackexchange.com/questions/395284/how-can-i-kill-minerd-malware-on-an-aws-ec2-instance-compromised-server – Kusalananda Feb 15 '18 at 06:43
  • Have you tried to find the file and see what it is *before* coming here? – Rui F Ribeiro Feb 15 '18 at 08:43

1 Answers1

1

It turns out there was a file in /tmp/ called "Silence", plus another one called "BoomBoom". Further investigation revealed that it was a crypto currency miner malware, probably injected through a tomcat/apache/apache-commons-collections deserialization vulnerability. Thank you for the tips.

Patkos Csaba
  • 2,500
  • 1
  • 20
  • 15
  • Have a look at your tomcat setup, they are a using a current vulnerability – Rui F Ribeiro Feb 16 '18 at 09:23
  • 2
    Take the server offline immediately, and follow the steps in the linked posts. Don't try to "recover" by simply removing the malware, you don't know what else is installed or modified. _That system is not yours any more_. – Kusalananda Feb 16 '18 at 10:24