The libssh2 1.8 RPM used to be part of RHEL 8.0. However libssh2-devel and libssh2-docs were never included to RHEL and were provided in EPEL. That complicated the usability of this package. So as of RHEL 8.1 libssh2 was moved to the EPEL as well. However it wasn't removed from modules.yaml metadata file of the repository rhel-8-for-x86_64-appstream-rpms. That's the problem.
If you want to install the EPEL version, it is necessary to temporarily disable the AppStream repository:
# dnf --enablerepo=epel --disablerepo=rhel-8-for-x86_64-appstream-rpms install libssh2-1.9.0
Then the installation then pass.
The modules metadata available on the RHEL 8.3 installation DVD ISO do not know the libssh2 and when this media is used as repository, you won't face this problem.
Red Hat is aware of the issue:
https://bugzilla.redhat.com/show_bug.cgi?id=1805260
The CentOS 8.1+ is probably affected as well, however I haven't checked.