7

Using QNX 6.4.1, there is a command called pidin times that shows information about processes. I think it means PID INformation. Among other things, you can see how much CPU a process has used since it was started.

I have a system that is showing almost 2 minutes of processor utilization for /usr/sbin/random after the system has been running for about 10 hours. That seems like a lot, as nothing in my code calls /usr/sbin/random.

There is a lot of network activity (UDP and TCP) right now though, so I'm wondering if the network driver is calling random to get dynamic collision backoff times because of packet collisions.

Could this theory be correct? (Okay, how plausible is it?) If not, is there something else I should check? There are currently latency problems with this system that did not exist yesterday and I'd like to find out what's going on. This particular clue may help isolate the problem.

Update

Further investigation using nicinfo revealed no packet collisions at all. So there goes my dynamic collision backoff time theory. Any other ideas?

Another Update

While this helped be find the answer to my problem (SSHD was using random, of course!!), be careful. If you use SSH, it needs a working random to allow you to log on. For some reason, the call in my script to random.old didn't work, and I just about bricked my embedded system. So be careful.

kmort
  • 724
  • 1
  • 8
  • 19
  • `/usr/bin/random` is not used by the OS, I think it's a user application that prints random lines of a file. It's used in shell scripts. – Barmar Jun 06 '14 at 20:51
  • In general, kernel drivers don't use executables, they use kernel libraries. – Barmar Jun 06 '14 at 20:52
  • @Barmar Noted. I'm not sure what's going on, as it's a pretty lightweight embedded system with nothing but my software running on it. – kmort Jun 06 '14 at 21:42
  • @Barmar That's the BSD game. QNX has an unrelated [`random`](http://www.qnx.com/developers/docs/660/topic/com.qnx.doc.neutrino.utilities/topic/r/random.html?cp=3_3_4_18_3) utility which generates random number; I don't know what on the system might be using it. – Gilles 'SO- stop being evil' Jun 06 '14 at 23:28
  • Can you turn on process accounting, this should tell you who is running the command. – Barmar Jun 06 '14 at 23:29
  • @Barmar I googled `Process Accounting QNX` and `psacct QNX` and didn't get any hits. :-( – kmort Jun 09 '14 at 12:17

1 Answers1

4

Crazy troubleshooting idea: make a honeypot / poor-man's process accounting.

  1. Make a backup of /usr/bin/random

    cp -p /usr/bin/random /usr/bin/random.bak

  2. touch /tmp/who_is_calling_random.log ; chmod 622 /tmp/who_is_calling_random.log

  3. Replace /usr/bin/random with this shell script (note you can use a different path than /tmp if you need to, but make sure it's world writable).

    #!/bin/sh
echo "`date` $USER $$ $@" >> /tmp/who_is_calling_random.log
/usr/bin/random.bak "$@"

  4. chmod 755 /usr/bin/random

  5. Reboot the system.

  6. See what gathers in the honeypot log. This should be a log of who/what is behind the use of the random program.

    tail -f /tmp/who_is_calling_random.log

  7. Restore random from the backup you made in step #1.

  8. Reboot system.

Joshua Huber
  • 728
  • 6
  • 16
  • Wow. So this is a good idea UNLESS you are using SSH as your ONLY WAY to communicate with the embedded system. Then, it's **BAD**. You can't log in via SSH without `usr/sbin/random` running. So, while I temporarily broke my system because of this answer, it also led me to find something using random that should have been obvious. SSHD is very likely the culprit. Long and short of it is that this is a good idea to narrow down the problem, **BE CAREFUL** and try it on a VM first. Thanks Joshua. :-) – kmort Jun 09 '14 at 13:13
  • Re-reading my comment, it could sound a bit snarky. No snark was intended. Sincere thanks Joshua. – kmort Jun 09 '14 at 13:22
  • No snark taken. Glad this crazy idea helped. – Joshua Huber Jun 09 '14 at 15:38
  • The script should be `chmod 755`, not `chmod 777`. The temp file should be `chmod 622 /tmp/who_is_calling_random.log` – Barmar Jun 09 '14 at 16:02
  • @Barmar, thanks, you're right. I've updated the answer. This must be the first time I've used `622`. – Joshua Huber Jun 09 '14 at 16:06
  • Maybe me, too -- had to think about it a bit. – Barmar Jun 09 '14 at 16:08