I'm on Arch Linux (a 32 bits alternative version), and recently I've discovered that the blue lines I used to see in Vim with :set cursorline were supposed to be underlined, and not blue.
That got me searching all around for a solution.
I'm using no graphical environment, so no desktop environment or window manager, only good ol' tty with zsh, my current favorite shell.
I've discovered that :hi CursorLine cterm=bold makes the cursor line a lot prettier, as it's now no longer blue, just a lighter color for the most part, and that's already made my life better.
I also tried cterm=underline (still renders the line blue), undercurl, tried :hi clear CursorLine then doing all over again, but nothing brings me the underlines I want.
I tried Vim on fbterm, because I believe it's a quasi-graphical terminal emulator, but I got the same behavior, only with an uglier super-wide font.
This not only applies to Vim, but to anything, it seems. I tried ANSI escape sequences on echoes, and when trying to underline text, I also got that blue color without underlines.
So I believe something is missing, be it a font, a shell config, a Vim config or whatever.
After searching quite a bit, I got no closer to an answer as to why my tty lacks those formatting options, so I decided to ask here.
It's also worth noting that I tried this on a Raspberry Pi 3B+ running Raspbian, and I got pretty much the same behavior on the tty.
The only place I managed to get that to work was on the X server I start from time to time to use Firefox. I spawned xterm on it and voila I got underlines even while typing commands on zsh.
I'll now try playing around with different terminal fonts to see if I get any closer to prettifying my tty.
Edit 01:
I recorded it with Asciinema, and it shows just fine there, but what I was actually seeing is as I described.

Edit 02:
I was reading this Arch Wiki page on the section about terminal emulators, and decided to try yaft, as it sounded it could be just the thing I was looking for.
It turned out I already had it installed, and using it does indeed enable at least some of the features I wanted, so that's great.