If your destination ssh daemon on the remote server runs on a non-standard port, and all the port related options to rsync only change settings if you’re running the rsync-daemon. Here is a solution to use rsync simply by specifying a different port
just Pass the port parameter to ssh with the -e option:
# rsync -avz -e "ssh -p $portNumber" user@remoteip:/path/to/files/ /local/path/