As we mentioned in section 2.1, creating a virtual network on top of a physical network is known as overlay networking. The underlying infrastructure becomes the underlay - also known as the physical (Layer 3) network. Several overlay methodologies exist. Two of the most widely-used are Virtual Extensible Local Area Network (VXLAN) and Generic Network Virtualization Encapsulation (GENEVE). It is important to note that VXLAN is vendor-neutral and has been recognized as RFC (Request for Comments) 7348, which is a formal document from the Internet Engineering Task Force (IETF).
VXLAN works on hardware (e.g., on routers or switches), on software (e.g., on a hypervisor) or on both (hybrid). It uses 24-bit binary identifiers (from 00000000000000000000 to 111111111111111111111111 and everything in between) meaning that a maximum 16,777,215 VXLANs are possible. Compare that to the maximum 4096 VLANs permitted by their 12-bit identifiers! A VXLAN ID is called a VXLAN Network Identifier (VNI). Each VNI is a separate virtual network that runs in the overlay network which is also known as bridge domains.
VXLAN Tunnel Endpoints (VTEPs) connect the physical network to the overlay network. Every VTEP has an IP address in the physical network and one or more VNIs in the overlay network. Encapsulated traffic (traffic that’s had certain information added to it at key stages of its journey - see section 4) is transferred between hosts via a stateless tunnel that is created between a source VTEP and a destination VTEP. By the time data from a host reaches a VXLAN switch, it’s in the form of frames, specifically “inner MAC frames” which include MAC (i.e., hardware) address information and data. The switches add a “VXLAN header” containing the 24-bit VNI.
The source VTEP then adds the IP address of the destination VTEP in an IP header, as well as its own IP address. It adds a User Data Protocol (UDP) header (UDP being the transport protocol that VXLAN uses). The MAC address of the next physical device that the frame will be delivered to on its journey is added in an Ethernet header. The physical network (the underlay) forwards the encapsulated frame on to the destination VTEP, which removes the headers in a process called decapsulation (mentioned again in section 4). The frame is then delivered to the destination host.
GENEVE is a relatively new entrant to the tunnel protocol field. It was jointly developed and drafted as an IETF proposal by Intel, Microsoft, Red Hat, and VMware and released in 2014. At the time this micro-course was written, GENEVE is currently going through the IETF process to become an RFC itself and so GENEVE is equally vendor-neutral. It works almost identically to VXLAN but is more flexible because it offers control plane independence between tunnel endpoints. And there’s a slight difference in terminology: GENEVE does not have VTEPs (VXLAN tunnel endpoints), just tunnel endpoints (TEPs).
It is important to note that NSX-V utilizes VXLAN overlay encapsulation, and NSX-T uses GENEVE.