Ask Brad Burres what his hobbies are in his spare time, and you might not expect his answer to be so simple: In addition to riding motorcycles, hiking, and leading Boy Scout field excursions, he also enjoys DIY. "When we use the nail gun, as long as we focus on it, it's not a problem," he said.

Interestingly, Brad Burres, an Intel researcher in the Ethernet Products Group, was one of the key people behind building the IPU. The IPU is a domain-specific processor designed to solve tough problems for server customers. As the number of CPU cores increases as computing demands increase, so does the overhead or the amount of infrastructure processing tasks. The last thing people want is to have powerful CPU cores bogged down in processing overhead so they can't run applications efficiently.

An analogy could be this: if you want to increase the size of your team to get work done faster, but that comes at a cost - consider the time it takes to assign tasks, make sure things get done properly, manage your team's shift schedule, etc. In a data center, infrastructure functions such as networking, storage, and security all require processing power, but not necessarily by the CPU, which is where the IPU comes in.

"As many as 30 to 40 percent of processor cycles are spent on infrastructure," says Brad Burres. Moving some of those infrastructure tasks to the IPU, then, frees up CPU cycles for more important workloads.

Brad Burres said: "The big infrastructure vendors all have a common goal of 'not running any infrastructure cycles on Xeon processors.' If you're a public cloud provider like Microsoft Azure, AWS or Google Cloud, You don't want to spend the slightest Xeon compute cycles on infrastructure tasks or networking."

Teamwork to solve problems

Intel teamed up with Google to develop its first ASIC IPU. Google, the web giant, has a specific list of needs, a partnership that requires deep cooperation and co-development across the board. Based on this, teams at Intel and Google have jointly developed programmable packet processing capabilities to support a range of use cases defined by Google. The end result of the partnership is Mount Evans, an ASIC chip that does the hardware offload and also utilizes the ARM N1 core for workloads that must be processed without the power of an Intel® Xeon® processor. However, this is not all the advantages of this chip. A lot of work has been done on both sides to ensure that Mount Evans can handle the infrastructure tasks as efficiently as possible. For example, Mount Evans supports up to 200 million packets per second (these packets are based on real workloads, not benchmark workloads).

Although the chip was co-designed by Intel and Google, it was by no means solely used to solve the problems facing Google. "Mount Evans isn't built just for Google," said Brad Burres. "While its optimization features are tailored to the problems Google found in its own operations, the chip is flexible enough to be used in many other places as well."

Several other customers are looking to leverage first-generation ASIC IPUs for similar needs in their data center operations.

Brad Burres said: "We heard from multiple customers that there were similar problems, and then spent time with friends who work in technology to discuss how to solve these problems and find commonalities, which led to the idea of ​​​​building an IPU." He added. Said that other companies facing similar infrastructure challenges have come up with their own solutions.

Evolution Theory: SmartNICs, IPUs, and the Future

Although IPU has only entered the public eye for just a year, Brad Burres has been thinking about it for many years. Development of Intel's first IPU began in 2016 to introduce another differentiated product that eventually morphed into a product that hit the market a year earlier. But in fact, the history of this chip can be traced back even earlier.

During his previous role as Chief Architect for SmartNICs, Brad Burres focused on areas such as network offload, storage offload and overall packet processing, specifically targeting top cloud service providers. So these are familiar territory for a particular processor that will ultimately be designed for those areas. The transition from SmartNICs to IPUs is a logical evolution, not just a terminology change. In fact, the difference between the two interprets the future of the IPU more broadly.

Brad Burres explained, "SmartNIC is a temporary term. A normal NIC doesn't have intelligence, it's just a fixed conduit without the help of an FPGA or compute. SmartNICs not only have intelligence, they have cores and flexibility, but It's limited to the network. With additional capabilities such as storage and telemetry, we have more opportunities to optimize, and the term 'smart NIC' no longer accurately describes what these chips do."

With fluctuating demand, it is important for companies whose overall business revolves around leasing server capacity to release general-purpose computing power. In the process, overprovisioning to meet increased demand for certain features is a waste, and introducing more powerful Xeon processors is not very economical. Based on this, customers scrambled to purchase Mount Evans IPU inventory to meet the above challenges.

Brad Burres said: "The market for IPUs is very strong. It's not just a matter of chip shipments, Intel has to make sure its software can also meet customer needs, such as scaling to the edge. It's not like Xeon processors, when we put After a Xeon processor is sold to a customer, the customer writes the software. We play a very important role in shipping the IPU and want to ensure incremental progress.”

The IPU update roadmap released in May lays out Intel IPUs for the next four years, including some details on the second-generation Mount Morgan. While it's early days, the teams working on Mount Morgan and subsequent third-generation IPUs are working with the Xeon product teams to coordinate and ensure they follow their respective paths in parallel and converge if necessary.

"The future is limitless." When asked about the future development of IPU, Brad Burres said: "The innovation rate of a new field is quite high, which is its charm. IPU is flexible enough to support more use cases. The use case for Mount Evans is in networking and infrastructure, and the next step is storage. Then you start to see some I/O tasks move down to the IPU. There is room for innovation - and we are in tip of the iceberg."

This idea may surprise many, but engineers prefer the "early and fun" phase. Because at this stage, "overhead tasks" (such as a calendar full of meetings) are compressed to a minimum. Developing an IPU isn't as simple as using a nail gun on a Sunday afternoon, but it's a little easier than many people think.