Your Resource for All Things Apps, Ops, and Infrastructure

Taking Care of Basics: Network Virtualization, Automation, and Security with NSX

A couple of weeks ago, Ryan Alt, Delivery Manager, introduced AHEAD’s Strategy and Roadmap service to help customers evaluate use cases for software-defined networking (SDN). In these engagements, AHEAD’s subject matter experts help clients define both use case and deployment methodologies for various SDN technologies.

In this post, Tim Carr, Senior Solutions Architect and one of AHEAD’s VMware VCDX experts helps to set the landscape and layout out of what VMware NSX is all about. In following, stay tuned for a subsequent write-up of Cisco ACI by Scott Honey, Senior Technical Architect at AHEAD. 

Network virtualization and automation are no longer nice-to-haves for organizations. Today’s customers expect a portal or an app store so that requesting an enterprise service is simple, seamless, and automated. Because this innovation is driving IT beyond simply provisioning virtual machines to providing full application stacks, optimizing virtual networking and providing network services—like load balancing and firewalls—is required to bring a whole service to market. Couple this with the fact that all major cloud providers leverage software-defined networking to take advantage of these types of virtual services and you’ll quickly see why the tipping point for enterprises to meet this demand has arrived.

VMware: A Networking Company

Founded in the trenches of server virtualization, VMware—even prior to the acquisition and integration of the tools that now make up NSX—was already a heavyweight player in data center networking. The VMware Virtual Standard Switch and Virtual Distributed Switch are the largest deployed virtual switches on the planet in the enterprise. NSX simply takes what VMware already understood with the concept of distributed virtual switching and the components of the vShield suite to the next level by virtualizing Layer 2-7 networking services and providing higher levels of network security functionality.

VMware NSX 

NSX is a network overlay and network function virtualization (NFV) technology. Much like VMware vSphere uses x86 servers and software to virtualize single, physical machines, NSX leverages encapsulation technology to virtualize Layer 2 switching and Layer 3 routing functionality from the existing underlying network. This virtualization allows for the easy addition of other great networking services like load balancing and firewalling, which NSX can provide out of the box. Each hypervisor host understands the logical virtual networks, load balancing policies, and firewall rules that play in the environment and therefore can make educated decisions on how to forward traffic for each VM that they service. By nature, this architecture optimizes traffic and allows for impressive scalability of services.

Caution: We’ve Seen This Before!

It is important to conceptualize NSX as an overlay technology because just like in the early days of compute virtualization, one cannot expect a poor performing network to be made better with software. You wouldn’t attempt to virtualize a mission critical application on a physical machine that was less performant than the machine you were moving it from physically. You can’t expect to leverage old networking technology plus NSX and achieve better networking results. That said, assuming that your physical network is robust and performing well, NSX can give you the capability to add virtualized services and security to existing, traditional Core-Distribution-Access architectures or newer Spine-Leaf (CLOS) network topologies. 

NSX Architecture

NSX physical architecture is composed of three distinct layers: management plane, control plane, and data plane. Components of NSX reside in each of these areas and provide parts of the overall picture. 


Management Plane

The NSX manager is a simple appliance that is downloaded and deployed to your virtual infrastructure. The manager is connected to your existing vSphere vCenter infrastructure and is leveraged to deploy NSX controller appliances. It serves as the endpoint for a REST-API that can be leveraged for creating change in your virtual networking environment. Coupled with vCenter, all of your networking services are configured using either this REST-API or vSphere Web Client.

Control Plane

The NSX controller—deployed at install by the management appliance—is generally a three node cluster (for high-availability) of VMware appliances that manage the logical networking state by pushing out the networking information to each ESXi hosts’ NSX virtual switch. Under the covers, this cluster of controller hosts distributes VXLAN and logical router information to each of the ESXi hosts. This information sharing allows for broadcast suppression that is different from previous VXLAN implementations from VMware specifically related to vCloud Director.  

Data Plane 

The NSX vSwitch is the workhorse of NSX implementations. Like the Distributed Virtual Switch (DVS) in vSphere, it is a kernel-based networking switch that is present on each hypervisor (ESXi) host that is updated with new logical networks and rules from the Control Plane (NSX Controller). This switch is fully capable of line rate performance from each hypervisor host. It enables plugin modules for advanced VXLAN, firewall, and logical router services.

NSX Use Case: Making Your UCS Deployment More Effective 

UCS is a really great technology. VMware is really great technology. Combining them both allows for better scaling of networking resources while adding services to your virtual network that may not be otherwise available. While highly scalable, UCS does introduce a few small weaknesses to network architecture when it comes to bringing together networks and servers. For example, consider a VM that lives on the same hypervisor host as another VM, yet they both reside in separate networks. What seems like trivial communication between the two VMs actually requires a trip out of the host across the Fabric Interconnect and up to the router (probably a Nexus 7000) to make the call for the packet to go right back to the host that it came from. That’s a lot of chatter in your data center that NSX is capable of eliminating by adding the NSX virtual switch to the hypervisor. By simply adding NSX in this scenario, the packet hits the NSX DVS and is routed internally to the second VM.


A similar and more common use case occurs for VM-to-VM traffic on two separate blades in the same UCS domain. Traffic would normally need to hit the router (in this case, the Nexus 7000) once again in your architecture. By adding NSX, the traffic never leaves the Fabric Interconnect level of the UCS architecture. 


I call out these two use cases because I find it common that VMs are often placed on separate VLANS by use case. By nature, this requires routing to move traffic from one network to the next. That routing is taking place above the Fabric Interconnects in your UCS domain (likely at a set of 7000s). This byproduct is called hairpinning and it could be silently impacting how your applications are performing. More efficient networks are always a good thing and surely this provides networking and virtualization teams with what they need to be able to more efficiently manage east-west data center traffic, but with 10GB networking as the common standard in today’s data centers, network bandwidth is often not a bottleneck. The bigger business problems for customers who have already adopted new networking architectures is truly around being able to provision and secure networks programmatically.

NSX Use Case: Micro-Segmentation

Security. With big name breaches in the news on a weekly basis, it’s become one of the hottest data center topics today. Looking back into many of these breaches, hackers have often penetrated external firewalls only to have unrestricted access to internal networks with sensitive data once inside. A new era of securing our services is upon us. By leveraging the distributed firewall functionality of NSX, we’re capable of securing our applications in a Zero Trust model, providing another layer of protection against threats like these. Zero Trust calls for organizations to deny all traffic by default, even on internal networks, and to look at what services should be explicitly enabled on a case-by-case basis compartmentalizing every application and service. With legacy firewalls, this is a problem. Hairpinning of traffic to ensure that appropriate inspection has occurred can cripple already highly leveraged networks and also requires very large physical firewall appliances. This is where the distributed nature of the NSX distributed firewall can truly shine.

When deploying new applications on top of NSX, we have the option to define rules that manage what can and cannot talk to the VMs within the application group. The distributed firewall manages policy compliance with these rules on our behalf across the NSX infrastructure. For the security-minded, this is a technology that can be bolted onto existing virtual infrastructure and migrated to in a very seamless way. In many cases, existing hosts can be leveraged and VMs migrated and tested one by one to ensure that the new firewalling method hasn’t impacted the application.

With the NSX distributed firewall, the security policies are enforced at the virtual network interface level. The VM itself is completely unaware of this because the firewalling processes aren’t actually touching the VM’s operating system. This is perhaps the most secure model for hosting virtual machines as packets don’t even touch a network interface if they are not supposed to. Because the firewall module is deployed in each hypervisor host (ESXi) in your NSX network, it’s possible to achieve true linear scalability of these services while only requiring additional microseconds for processing time.  

Automation: Networking’s Next Frontier 

IT organizations are struggling to keep up with the demand for more efficiently delivered enterprise services. The holdup? Oftentimes, it’s related to the IT group being able to cover the basics. Very few organizations can deliver a virtual machine that’s on the appropriate network, backed up, secured, patched, and added to the organization’s CMDB in an automated fashion. If your organization can do this today, then you’re performing at a pretty high level. The next level of service automation takes these perfectly built VMs and groups them into true application stacks that require network services such as load balancing and firewalling. In many organizations, these load balancing and firewall changes are manual processes that require change review and approval. These processes, while designed to protect and ensure great IT service delivery, are a hurdle to business agility and are being automated by the best performing organizations into quick-moving, pre-approved changes.

NSX natively plugs into VMware’s vRealize Suite of products to make networking services provisioning something that can be done as a part of an automated workflow. De-provisioning is also just as important. Many organizations miss on this aspect of environment management leaving old firewall rules open to newly created services. In the best case, firewall rulesets pile up and become difficult for security teams to effectively manage. In the worst case scenario, a new service is provisioned where an old rule exists and opens a whole organization up to external attack. By automating and clearly defining rules for every service that is deployed, our networks become both more easily managed and more secure.

AHEAD’s stance is clear on service delivery; organizations need a central portal to manage IT services and not a bunch of GUIs for administrators to use. Automation in general is great, but without a consistent method to provide end users with the services that they need, they’ll look elsewhere. This is where a true enterprise service catalog shines. Part of achieving this service delivery vision is the requirement of making networking services, such as firewalling, network provisioning, and load balancing, something that we can consume in an automated and controlled manner.

For more information about network automation, contact AHEAD today to schedule a meeting with our experts in the AHEAD Lab and Briefing Center. We can discuss your platform in depth and how tools like NSX, when coupled with Cisco UCS and the VMware vRealize Suite, can take your service delivery capabilities to the next level.

AHEAD Lab and Briefing Center

Watch On-Demand Sessions from our Spring Summit, Automation @Scale!

Subscribe to the AHEAD i/o Newsletter