NSX-T Ports and Protocols

In most cases, the infrastructure components (vcenter, esxi, nsx, etc.) reside on the same network. Then they communicate with each other within the same subnet without having to cross routers and firewalls.

Normally this is the case … but if you operate within large enterprise it is likely that there are management clusters that manage other clusters dedicated only to workload.

These clusters can reside on networks separated by different Layer3 (router) or Layer 4-7 (firewall) devices. In this case it is necessary to communicate with network specialists to ensure full visibility between all the objects of the infrastructure.

You will hardly have a visibility of the type any any permit , you are more likely to have to provide a detailed list with source and destination addresses and ports TPC / UDP on which to allow access.

Have you ever had to provide this list of addresses and ports? To me yes and I can assure you that it is not as simple as it seems, the objects to communicate are many and on different services, forgetting some rules can result in a lot of time spent in troubleshooting 🙁

Things have changed over the years and now it is no longer necessary to search the installation manuals for lists with all the necessary ports.

Thanks to vmware who realized this very useful website VMware Ports and Protocols 🙂

Take some time to browse through all the vmware products, you will realize how many services and ports are necessary to make the various solutions communicate.

But go ahead! In this article, we’ll just find out which ports are needed for NSX-T! From the homepage of the previous site we select NSX-T Data Center.

It is possible to apply filters and select the rules by version and specific object.

Once the filters have been applied, it is also possible to export the list in pdf and excel 🙂

By filtering with version 3.1 and source Manager we obtain the list of rules needed by NSX Manager to communicate with all the objects it needs.

I summarize them below, the source is obviously the NSX Managers ip:

Destination
ProtocolPortService Description
External LDAP server
TCP389,636Active Directory/LDAP
NSX ManagerTCP9000, 5671, 1234, 443, 8080, 1235, 9040Distributed Datastore
Install-upgrade HTTP repository
NSX messaging
Distributed Datastore
KVM and ESXi host
TCP443
Management and provisioning connection
vCenter Server
TCP443NSX Manager to compute manager
Traceroute DestinationUDP33434-33523Traceroute (Troubleshooting)
Intermediate and Root CA ServersTCP80Certificate Revocation Lists (CRLs)
Syslog Servers
TCP/UDP6514, 514Syslog
SNMP Servers
TCP/UDP161, 162
SNMP
NTP Servers
UDP123NTP
Management SCP Servers
TCP22SSH (upload support bundle, backups, etc.)
DNS Servers
TCP/UDP53DNS
Public Cloud Gateway (PCG)
TCP443NSX RPC channel(s)
github.com
TCP443Download IDS Signature from Trustwave Signature Repository.

Same applies for Transport nodes and ESXi

Destination
ProtocolPortService Description
Intermediate and Root CA servers
TCP80Certificate Revocation Lists (CRLs)
NSX ManagerTCP1234, 8080, 1235, 5671, 443NSX Messaging channel
Install and upgrade HTTP repository
Management and provisioning connection
Syslog Servers
TCP/UDP6514, 514Syslog
NSX-T Data Center transport node
UDP3784, 3785
BFD Session between TEPs, in the datapath using TEP interface
GENEVE Termination End Point (TEP)UDP6081Transport network

NOTE : these are the ports needed by NSX, ESXi hosts will clearly need other ports for normal operation (NPT, DNS, SSH, etc.)

Some rules may seem redundant but remember that you have to distinguish between objects that start the session and their destinations, sometimes you need rules that allow traffic on both sides on the same ports.

For version 3.1 the list is of about 50 services, have a look … to know them allow you to save time 🙂

Posted in firewall, nsx | Tagged , | Comments Off on NSX-T Ports and Protocols

NSX-T Manager installation requirements

Crazy to get your hands on NSX-T? Want to browse through all of the Manager’s web console entries? Do you prefer learning by doing?

If you are like me I think so 🙂

However, experience has taught me that starting directly with the installation, skipping checks on requirements and proper sizing, sooner or later inevitably leads to having to reinstall everything 🙁

Don’t worry, the NSX-T Data Center guide provides Workflows for every type of installation, this one for vSphere:

  1. Review the NSX Manager installation requirements.
  2. Configure the necessary ports and protocols.
  3. Install the NSX Manager.
  4. Log in to the newly created NSX Manager.
  5. Configure a compute manager.
  6. Deploy additional NSX Manager nodes to form a cluster.
  7. Review the NSX Edge installation requirements.
  8. Install NSX Edges.
  9. Create an NSX Edge cluster.
  10. Create transport zones.
  11. Create host transport nodes.

This article goes into the first point, in succession I will publish the articles of all the other points 🙂 The version of NSX-T of reference is the 3.1.

The information reported is from NSX-T Data Center Installation Guide, paragraph System Requirements.

ESXi versions supported for the Host Transport Nodes role

For more details see directly the Product Interoperability Matrices

Minimum CPU and RAM requirements for the Transport Node profile

HypervisorCPU CoresMemory
vSphere ESXi416GB

Nota : HW requirements are the same also for KVM on Linux

Network Requirements

Each Host Transport Node must be equipped with NIC supported by the installed version of ESXi, there is no mention of minimum speed but it is good practice to use at least 2 10G NICs for performance and redundancy.

Note : 2 NICs are enough but 4 can facilitate the migration from VSS/VDS to the new NVDS. I will deepen this aspect in a next article.

Requirements for NSX Manager

The Manager is the heart of NSX, as well as allowing the management of all configurations also incorporates the role of controller, these are the requirements for the deployment.

Appliance SizeMemoryvCPUDisk SpaceNote
NSX Manager Extra Small8GB2300GBonly for the Cloud Service Manager (da NSX-T 3.0)
NSX Manager Small VM16GB4300GBlab and proof-of-concept deployments (da NSX-T 2.5.1)
NSX Manager Medium VM24GB6300GBtypical production environments - max 64 hypervisor
NSX Manager Large VM48GB12300GBlarge-scale deployments - more than 64 hypervisor

Network latency requirements

Maximum latency between Managers of a cluster : 10ms.

Maximum latency between Managers and Transport Nodes: 150ms.

Perhaps obvious but often underestimated data, especially on geographically distributed installations.

Addressing requirements and DNS configurations

An NSX installation comprises a cluster of 3 NSX Managers and an Edge cluster with at least 2 nodes. This for production environments, for PoCs or demos, the high reliability of the components can be overlooked.

IP addresses must be provided for each object and relative FQDNs on the relevant DNS zones.

Note : in addition to the A records, it is important to provide for reverse lookup zones, the lack of these records can introduce problems of reachability and communication between the various components.

If the GENEVE overlay is also present, a subnet (IP pool) must be allocated for the TEPs to be assigned to the Transport Nodes. Also on this issue we will have the opportunity to deepen 🙂

Total Sizing

And finally, let’s sum it up! How many resources should be allocated for a typical NSX-T production installation?

 vCPUMemoryStorage
NSX Manager Medium624GB300GB
Total x 3 Manager1872GB900GB
NSX Edge Medium48GB200GB
Total x 2 Edge816GB400GB
Total resources2688GB1.3 TB

The resources required by NSX-T are not negligible, on small infrastructures they can have a strong impact. On environments of a certain entity NSX Manager is installed on a dedicated Management cluster together with other objects such as vRealize Log Insight and vRealize Network Insight.

Knowing the NSX-T requirements and doing a correct sizing is certainly the first place to start for a correct installation.

I conclude the article leaving you with a small task, check if your infrastructure has the resources to install NSX! Don’t you have them? Then design a new Management cluster 🙂

Posted in nsx, vmug | Tagged , , | Comments Off on NSX-T Manager installation requirements

NSX … A journey of a thousand miles begins with a single step

Title a bit obvious perhaps, but I think very appropriate.

It has been a while now that I have been looking for the right way to start a series of articles on NSX, every time I find an inspiration I realize that there is always something before it should be explained and told. It’s a bit like taking something apart to understand how it works… you stop only when you have removed the last screw and you have on the table dozens of small pieces 🙂  The same thing is true for NSX, where do you start to understand and take ownership of this technology? What do you need to know first? What are the basic building blocks that make up NSX?

Let’s start with these simple questions to chart a roadmap that I hope will help those approaching NSX.

What is NSX?
Why not ask vmware directly? This is the link where to go to look around :

VMware NSX Data Center

It is certainly an overview page, but for this reason it has been designed to highlight the strengths of NSX:

– Agility Through Automation
– Consistent Multi-Cloud Operations
– Intrinsic security
– Save on Both CapEx and OpEx

Ok, the last one is very commercial … but you will understand that it is the consequence of consolidating network and security functions into a single distributed virtualization platform!

Probably someone was expecting definitions like SDN or similar, surprised to find topics like multi-cloud and automation?

And here we go a little more in detail:

vmware-nsx-solution-brief

In the datasheet we find the key features and a table of functionalities divided by license type

vmware-nsx-datasheet

NOTE: many Companies choose NSX mainly for micro-segmentation … attention that Distributed Firewalling is included only starting from Professional!

But it is not over yet … there are some aspects that need to be explored before jumping into the first installation!

We find clear guidance right in the ICM course( Install, Configure and Manage ) of NSX Data Center :

Prerequisites:

Solid understanding of concepts presented in the following courses:
VMware Data Center Virtualization Fundamentals
VMware Introduction to Network Virtualization with NSX
VMware Network Virtualization Fundamentals
Network and Security Architecture with NSX ( I added this one 🙂 )

These are all free courses that you can access with a basic subscription to vmware learning zone!

– Good understanding of TCP/IP services and network security and working experience with firewalls
– Working experience of enterprise switching and routing

These last two points are perhaps the most challenging for those who have no networking experience but if you have read this far you probably don’t lack interest!

NSX is a complete and complex product and a few quick reads are not enough to understand it in depth.

In my experience I can say that working on Linux environments and networking courses (CCNA like) can make learning NSX easier.

I leave you all these hints and see you at the next article!

Posted in nsx | Tagged , , , | Comments Off on NSX … A journey of a thousand miles begins with a single step

VMware Livefire

What is a livefire?

This term is often associated with military training. These are exercises where participants can live an experience very similar to a real fight.

In the technical field we can compare it to a training event where you operate in real environments, supported and led by an experienced instructor.

We associate VMware to Livefire and we immediately understand what we are talking about.

VMware Livefire is launched in early 2018 as an important tool to support the community of VMware partners. Yes, it is a tool designed for partners and not for end customers. Participation is by direct invitation only from the Livefire team so if you are a VMware partner and want to participate in a Livefire course, you just have to ask your VMware partner manager for information.

This year I had the opportunity to participate in two Livefire courses:

  • NSX-V Livefire – Architecture and Design
  • NSX-T Livefire – Next Generation Cloud Networking

Two very interesting but very different courses, the first (NSX-V) focused more on architecture and design, the second (NSX-T) much more technical and provided with laboratories in the Hands-on Lab style.

You can consult the course catalogue here, there’s something for everyone!

If you look at the course datasheets you’ll find that you need precise prerequisites, normally a VCP level certification.

The courses are undoubtedly challenging and require some experience (nothing to do with a normal install configure and manage course) but I can assure you that if you want to test yourself and meet international professionals and experts this is the experience for you!

With this article I would like to start sharing my experience in the NSX field and tell you about this fascinating technology that continues to evolve day by day.

Scanda

Posted in livefire, nsx, vmug | Tagged , , , | Comments Off on VMware Livefire