IKE uses UDP port 500 for both source and destination.
Checkout this video:
Introduction
UDP port 500 is used for IKE traffic. UDP port 4500 is used for NAT-T traffic.
IPSec policies configured on the VPN server and client computers use IKE to negotiate security parameters and agree on how traffic will be encrypted. Traffic that uses these security parameters is known as IKE or IPSec traffic.
When you configure a VPN server or client, you must specify an IKE phase 1 policy. This policy contains settings that are used during the initial negotiation of an IKE security association (SA). The phase 1 policy contains the following settings:
-Authentication method (pre-shared key or certificate)
-IKE exchange mode (main, aggressive, or quick mode)
-Diffie-Hellman group (1, 2, 5, 14, 15, 16, 17, 18, 19, 20)
-Encryption algorithm (DES, 3DES, AES)
-Hash algorithm (MD5 or SHA)
-Authentication algorithm (MD5 or SHA)
-Perfect forward secrecy (enabled or disabled)
-Lifetime (in seconds)
What is UDP port 500?
UDP port 500 is the port number for ISAKMP, the Internet Security Association and Key Management Protocol. ISAKMP is used to set up security associations and shared keys for use in IPsec. When two VPN devices need to communicate using IPsec, they use UDP port 500 to establish a secure connection.
What is UDP port 4500?
UDP port 4500 is used for IKE traffic from VPN Client to Server. IKE uses UDP port 4500 for both Phase 1 and Phase 2 traffic.
What is NAT-T?
NAT Traversal (NAT-T) is a mechanism used to allow hosts behind a NAT device to communicate with each other using IPsec. NAT-T uses UDP port 4500 to allow hosts behind a NAT device to communicate with each other using IPsec.
Conclusion
IKE traffic uses UDP port 500.