Solved my problem, and all the confusion was probably caused by my giving somewhat inaccurate information (sorry!).
The logs I showed above are from my Azure virtual machine, but then I also mentioned that I had turned the firewall off, which I had done on my local machine. So it eventually worked on my local machine when I fixed the public IP issue, and I now got it to work on the Azure virtual machine by setting up "endpoints" for the required ports in the Azure management portal, as well as creating an inbound rule in the Windows Firewall for traffic on the same ports, that comes from Port 20 on the remote machine.
Sorry for the confusion! Consider this resolved, and thank you for your help!