Which should I use if I run tmux inside kitty?
When running tmux then you must use TERM=screen, TERM=tmux, or one of the derivatives e.g. TERM=screen-256color or TERM=tmux-256color.
man tmux(1) states ...
The TERM environment variable must be set to “screen” for all programs running inside tmux. New windows will automatically have “TERM=screen” added to their environment, but care must be taken not to reset this in shell start-up files.
.. and further down clarifies ...
Set the default terminal for new windows created in this session - the default value of the TERM environment variable. For tmux to work correctly, this must be set to ‘screen’ or a derivative of it.
... they also emphasize must.
The tmux wiki continues with additional clarification and answers ...
Inside tmux TERM must be "screen", "tmux" or similar (such as "tmux-256color").
Don't bother reporting problems where it isn't!
Are there programs that need to be aware that they're running in screen?
Yes. Many programs, such as tput, will not display correctly when they're running in screen (or tmux) unless they're using the correct TERM value.
What is TERM and what does it do?
The environment variable TERM tells applications the name of a terminal description to read from the terminfo(5) database. Each description consists of a number of named capabilities which tell applications what to send to control the terminal. For example, the "cup" capability contains the escape sequence used to move the cursor up.
It is important that TERM points to the correct description for the terminal an application is running in - if it doesn't, applications may misbehave.
The infocmp(1) command shows the contents of a terminal description and the tic(1) command builds and installs a description from a file (the -x flag is normally required with both).
Would it be problematic to call screen running in kitty xterm-kitty?
Not necessarily, as long as the TERM value in the running screen is set correctly (i.e. TERM=screen-256color) and there's valid terminfo strings for it that screen or tmux knows how to handle.
Other derivatives (on my machine) are:
/usr/share/terminfo/s/screen
/usr/share/terminfo/s/screen-16color
/usr/share/terminfo/s/screen-256color
/usr/share/terminfo/s/screen.Eterm
/usr/share/terminfo/s/screen.gnome
/usr/share/terminfo/s/screen.konsole
/usr/share/terminfo/s/screen.linux
/usr/share/terminfo/s/screen.mlterm
/usr/share/terminfo/s/screen.mrxvt
/usr/share/terminfo/s/screen.rxvt
/usr/share/terminfo/s/screen.teraterm
/usr/share/terminfo/s/screen.vte
/usr/share/terminfo/s/screen.xterm-r6
/usr/share/terminfo/s/screen.xterm-new
/usr/share/terminfo/s/screen.xterm-xfree86
Once TERM is set, you could run infocmp. That will report if there's a valid terminfo for the value. Other tests can validate basic terminfo functionality, such as:
for terminfos in $(find /usr/share/terminfo/s -type f -name 'screen*' -printf '%f\n' -o -name 'tmux*' -printf '%f\n'); do export TERM=$terminfos; printf "%s: %sthis is foreground color 10 %sitalics%s and now all attributes are turned off\n" "$terminfos" "$(tput setaf 10)" "$(tput sitm)" "$(tput sgr0)"; done