While it's certainly possible for users to customize their prompt (and make that update the terminal's title), most use the default shell behavior.
It sounds as if your local machine is setting the title string as a side effect of your prompt, and that the remote machines are not changing the title.
Given that, you can update the title without interference by the remote machines, e.g., using a wrapper script for ssh such as this:
#!/bin/bash
# trim parameters, leaving just the last (user@hostname or just hostname)
title=$(echo "$*" | sed -e 's/^.* //')
printf '\033]0;%s\007' "$title"
/usr/bin/ssh "$@"
and putting that in your executable path ahead of /usr/bin, you could call that "ssh" and have it set your title string as you visit each remote machine. After exiting ssh, your local prompt would reset the title string back to the local machine.
In a followup comment, OP indicated that the connections are made to an IP-address. If the remote machine has a hostname, then it would make the procedure clearer to use that, or (if not in DNS) collect the hostnames into the local machine's /etc/hosts. Ultimately, DNS is the way to go...
For reference: