Summary
vim --version says -clipboard:
[Shift]+Left-Mouse-Button selection to copy from Vim, Shift+Insert or Middle-Mouse-Button or Edit > Paste selection to paste it in Xfce4-terminal.
vim --version says +clipboard:
Option 1: [Shift]+Left-Mouse-Button selection to copy from Vim, Shift+Insert or Middle-Mouse-Button or Edit > Paste selection to paste it in Xfce4-terminal.
Option 2: Yank text to * register. For example, "*Y to yank the whole line. Paste it in Xfce4-terminal with Shift+Insert or Middle-Mouse-Button or Edit > Paste selection.
Option 3: Yank text to + register. For example, "+Y to yank the whole line. Paste it in Xfce4-terminal with Ctrl+Shift+V or Edit > Paste.
The [Shift] in brackets is optional if your Vim does not mouse enabled. The above description will also work for other terminals, except Option 3 and menu items (Edit > ...).
Explanation
There are two main "clipboards" around: Primary and Clipboard. The content of Primary is set simply whenever text is selected. It needs one more action to get to Clipboard, usually a keyboard shortcut (a widespread one is Ctrl+C) or a menu item. Pasting from Primary is done with Middle-Mouse-Button (many terminals also use Shift+Insert). Pasting from Clipboard may be done with Ctrl+V, Ctrl+Shift+V... It is all up to the application.
Xfce4-terminal pastes from Primary with Shift+Insert or Middle-Mouse-Button or Edit > Paste selection menu item. It pastes from Clipboard with Ctrl+Shift+V or Edit > Paste. It copies to Primary with mouse selection and to Clipboard with Ctrl+Shift+C or menu item.
Minimalist terminal emulators such as Xterm have only Primary enabled by default (but it can be made to support Clipboard).
If +clipboard is enabled, Vim associates register * to Primary and register + to Clipboard.
If Vim is running in a terminal emulator and Vim's mouse support is on (:set mouse=a), then the mouse selects text to Visual mode. If you want the terminal's capability to select to Primary, then you need to press Shift while selecting with the mouse. From :help mouse:
Note: When enabling the mouse in a terminal, copy/paste will use the
"* register if there is access to an X-server. The xterm handling of
the mouse buttons can still be used by keeping the shift key pressed.
That is a common pattern of text-terminal tools that support mouse. For example, start nano without mouse support and with mouse support, nano --mouse. In the first case, you can simply select text with mouse, but in the latter case, you also need shift to be pressed just as with Vim. With Midnight Commander the same happens with mc --nomouse and mc. All those utilities mention it in their manuals.
You also may find this Freedesktop specification informative.