After reading man ssh_config and man sshd_config I found that the evnironment variables exported by ssh are controlled by the SendEnv directive in /etc/ssh/ssh_config:
SendEnv
Specifies what variables from the local environ(7) should be sent to the server.
The server must also support it, and the server must be configured to accept these
environment variables. Note that the TERM environment variable is always sent
whenever a pseudo-terminal is requested as it is required by the protocol. Refer to
AcceptEnv in sshd_config(5) for how to configure the server. Variables are
specified by name, which may contain wildcard characters. Multiple environment
variables may be separated by whitespace or spread across multiple SendEnv
directives. The default is not to send any environment variables.
and, on the server, the AcceptEnv directive in (the server's) /etc/ssh/sshd_config:
AcceptEnv
Specifies what environment variables sent by the client will be copied into the
session's environ(7). See SendEnv in ssh_config(5) for how to configure the client.
The TERM environment variable is always sent whenever the client requests a pseudo-
terminal as it is required by the protocol. Variables are specified by name, which
may contain the wildcard characters ‘*’ and ‘?’. Multiple environment variables may
be separated by whitespace or spread across multiple AcceptEnv directives. Be
warned that some environment variables could be used to bypass restricted user
environments. For this reason, care should be taken in the use of this directive.
The default is not to accept any environment variables.
BUT your XDG_ variables refer to Your desktop session. Simply copying them as-is to another system makes no sense.
This seems like an "XY" problem. Why do you care about XDG_ variables on the server side of he ssh conversation? -X and -Y set up a local X server, and set $DISPLAY for you.