The battle of the emerging SmartNIC market for data centers. FPGA vs SoC? Xilinx, NVIDIA, Marvell, Broadcom or Napatech?

Here is my take on it.

Thread (1/23)

As the demand for hyperscale data centers increase, and in particular the speed requirements within them, the demand for advanced and workload customized network equipment rise. Up until today all servers have connected to a network using a network interface card (NIC).

However, ordinary NICs are starting to become obsolete in today’s world of faster servers, storage, and network connections. Why? Simply because CPUs are unable keep up with the rapidly increasing network processing demands.

Today, a NIC handles up to 100mn network packets per second. CPU cores simply aren’t up to the task of inspecting and acting upon tens of millions of packets per second each. This situation has resulted in a bottle neck and thus inefficiency. A minor "data center crisis".

Even worse, the increasingly demanding workloads cause overhead that burdens the CPUs and thus chewing into the available processing power. This, so called, “data center tax” imply that up to 30% of the CPU cycles are managing networking tasks. A clear waste of CPU power.

And that’s where SmartNIC’s come into play. Roughly a decade ago, we saw GPU products emerge as the first significant wave in hardware-acceleration technology. We are now about to see the second wave of hardware-acceleration emerge, with SmartNICs.

A simplified description of a SmartNIC is that it's a NIC which allows for software to be put into it by its user, after it has been purchased. It’s a customizeable NIC. Much like a smartphone, which allows you to install apps on it. Which a Nokia 3310 didn’t allow.

A SmartNICs job is to offload tasks that network system CPUs would otherwise have to deal with. For example TCP/IP acceleration, HTTP processing, encryption and establishing firewalls. See my, PowerPoint Pro™, visualization in the attached image.

SmartNICs pushes computation and thus acceleration to the edge of the network, freeing up CPUs to handle more solutions focused and critical processing tasks. They improve server performance and reduces its power consumption, resulting in lower cost of ownership.

The market for ethernet adapters with an onboard field customizable processor (SmartNICs) such as a field-programmable gate array (FPGA) or system on chip (SoC), is estimated to grow from around $650mn in 2020 to $2.3bn in 2024. A meaningful part of the TAM will be software.

Today, 25G+ (gigabit ethernet) cards account for around 43% of TAM. This is a number which is expected to increase to 76% in 2023, according to Mellanox. At speeds of 50Gb, 100Gb and 200Gb, SmartNICs will definitely be needed in hyperscale data centers.

But before we start talking about specific companies, it’s important to note that there exist different SmartNIC technology.

-ASICs (Application Specific Integrated Circuit).

-Systems on a Chip (SoCs), typically includes an onboard CPU.

-Field-Programmable Gate Arrays (FPGAs).

It is also important to highlight that large hyperscale operators such as AWS (Amazon), Azure (Microsoft) and Alibaba up until now have developed their own SmartNICs. But as the adoption and large scale need for SmartNICs increase going forward, I believe that this will change.

Why? Because today there are pretty much no off-the-shelf SmartNIC software available. Imagine today’s smartphones without App Store, Google Play and all apps that comes with them. You would have to program yourself. I thus believe that the SmartNIC software market will grow.

So, Azure’s Catapult, AWS’s Nitro and Alibaba Cloud’s X-Dragon are all designed in-house. But what technology do they use?

Microsoft Azure’s Catapult is based on FPGA.

AWS Nitro is based on SoC.

Alibaba Cloud’s X-Dragon is based on SoC.

There is clearly no technology consensus yet, and perhaps there never will be. So what companies are there to examine if you want decent exposure? Well, it all comes down to what technology you believe will be widely adopted, and if you want 100% exposure to SmartNICs or not.

Xilinx $XLNX was the first company to commericialize the FPGA back in mid-80s. Today, they are the FPGA market leader and controls approximately 50% of the TAM. Note that FPGAs have many areas of use, 5G base stations for example. SmartNIC FPGA is still a small niche for Xilinx.

In 2019 Xilinx was interested in acquiring Mellanox in order to climb the value chain. Intel was also interested but NVIDIA won the battle. Instead, Xilinx acquired Solarflare and released its FPGA based SmartNIC named Alveo this year. They now supply both FPGAs and end-products.

Meanwhile, Jensen Huang, NVIDIA’s $NVDA CEO, has been public in his dislike of FPGAs. NVIDIA will thus most likely focus on Mellanox’s BlueField (labled as “DPU” by NVIDIA) SmartNIC products (SoC based), that currently integrates ARM processors.

It will be interesting to see how NVIDIA utilizes their GPU expertise in the SmartNIC segment. The more cores a processing unit have (NVIDIA GPU’s outnumber CPU’s by far), the better for networking and data processing purposes. If done right, they have the potential to dominate.

Then you have Marvell $MRVL and Broadcom $AVGO (early market leaders) and the niched SmartNIC company Napatech $NAPA. Bear in mind that Xilinx currently supply several other SmartNIC companies with FPGAs (for example NVIDIA’s Innova SmartNICs and all Napatech’s FPGA based ones).

Conclusion:

Marvell and Broadcom are early market leaders.

Xilinx is a bet on that FPGAs and SoCs will co-exist in SmartNICs. They supply standard FPGAs, but also the end-product.

NVIDIA's DPUs could be a long-term “winner takes it all” bet.

Napatech is the most focused bet.

Addition:
From a memory perspective $MU, it is nice to see that SmartNICs will be a new area of DRAM use.

Anyway, I hope that y'all got some insights in the emerging SmartNIC segment, and that it might help you continue do your own research if you find it exciting.