At a time when artificial intelligence (AI)-centric system-on-chips (SoCs) are growing in size and complexity, network-on-chip (NoC) tiling hand in hand with mesh topology can support faster development of compute chip designs.
That’s the premise around which Arteris has launched tiling as the next evolutionary step in its NoC IP offerings to facilitate scaling, condense design time, speed testing, and reduce design risk. The Campbell, California-based supplier of IPs is combining NoC tiling with mesh topologies for SoC designs catering to larger AI data volumes and complex algorithms.
Figure 1 Mesh topologies complement NoC tiling to further reduce the overall SoC connectivity execution time by up to 50% versus manually integrated, non-tiled designs. Source: Arteris
SiMa.ai, a developer of machine learning (ML) SoCs, has created an Arm-based, multi-modal, software-centric edge AI platform using this mesh-based NoC IP. The upstart’s AI chip models range from CNNs to multi-modal GenAI and everything in between with scalable performance per watt.
But before we delve into further details about this new NoC technology for SoC designs, below is a brief recap of what it’s all about and why it has been launched now.
What’s NoC tiling
NoC tiling allows SoC architects to create modular, scalable designs by replicating soft tiles across the chip. And each soft tile represents a self-contained functional unit, enabling faster integration, verification and optimization.
Without NoC tiling in a neural processing unit, each neural interface unit (NIU) and transport element inside NoC is unique, and it must be implemented separately and connected to the processing element individually. That increases complexity and configuration time for the designer, which impacts time to market and makes verification effort a lot trickier.
Figure 2 NoC tiling organizes NIUs into modular, repeatable blocks to improve scalability, efficiency, and reliability in SoC designs. Source: Arteris
The tiling technique is designed to repeat modular units automatically, eliminating the need to break the design and configure each element. In other words, it divides the design into modular, repeatable units called “tiles”, enabling significant scalability, power efficiency, reduced latency, and faster development without redesigning the entire NoC architecture.
Take the example of a coherent mesh NoC with tiled CPU clusters, each containing up to 32 CPUs (Figure 3). A 5×5 mesh configuration allows 16 CPU clusters access to maximum memory bandwidth. The remaining mesh sockets are used for caches and service networks.
Figure 3 By supporting NoC tiling, mesh interconnect topologies become a common building block in AI-centric SoC designs. Source: Arteris
Mesh topology complements NoC tiling by providing an effective underlying communication infrastructure for regular processing elements. Each AI accelerator is connected to the NoC mesh, allowing seamless data exchange and collaboration in the vision processing workflow.
Otherwise, without NoC tiling, every NIU and transport element is unique and implemented separately, requiring a manual configuration step despite the same processing element in each case. And, with NoC tiling, effort to implement the NIUs—the most logically intense elements in the NoC—is drastically reduced.
Below is a sneak peek at three specific design premises accelerating AI- and ML-based semiconductor designs.
- Scalable performance
The number of processing elements often scales non-linearly; though they scale linearly initially until memory bottlenecks are reached. Here, NoC tiling allows designers to define one processing element and its connection point and then scale that arbitrarily until the workload is met without any redesign effort.
As a result, NoC tiling supported by mesh topology enables AI-centric SoCs to easily scale by 10x+ without changing the basic design. “It enables repeating modular units within the same chip, and that allows architects to easily create scalable and modular designs, enabling faster innovation and more reliable, power-efficient AI chip development,” said Andy Nightingale, VP of product management and marketing at Arteris.
- Power reduction
Another advantage is that NoC tiling allows easy partitioning for power reduction, so power management connectivity is replicated from within each individual tile. “Tiling connects into power-saving technology and replicates all that automatically,” Nightingale added.
NoC tiles use dynamic frequency scaling to turn off dynamically, cutting power by 20% on average, which is vital for energy-efficient and sustainable AI applications. Here, NoC tile boundaries interface into existing NoC clock and voltage domains as needed. So, groups of NoC tiles can be turned off when not needed.
- Dynamic reuse
The pre-tested NoC tiles can be reused, cutting the SoC integration time by up to 50% and thus shortening the time to market for AI chips. This pre-configured and pre-verified interconnect feature addresses the growing demand for faster and more frequent innovation cycles in AI chips.
NoC tiling: Why now?
When asked why NoC tiling has arrived now, Nightingale told EDN that while the complexity of AI chips is going up, there are still the same number of chip designers. “Anything we can do to increase the automation and decrease the design risk, especially when you have massively parallel processing in AI chips like TPUs,” he said.
He added that when you delve into the SoC design details, they are embarrassingly parallel with repeat, repeat, and repeat, and that leads to very regular structures. “So, when AI comes along and puts requirements in hardware, chip designer has the choice of working on each individual processing element or taking advantage of technology and say connect everything for me.”
Figure 4 NoC tiling enables a chip designer to define a modular unit just once and then repeat it multiple times within the same SoC design. Source: Arteris
Nightingale concluded by saying that SoC designers have been asking for this feature for a long time. “While other NoC suppliers have tiling on the check box, Arteris is first to bring this stuff out.”
Related Content
- SoC Interconnect: Don’t DIY!
- What is the future for Network-on-Chip?
- Why verification matters in network-on-chip (NoC) design
- Network-on-chip (NoC) interconnect topologies explained
- SoC design: When is a network-on-chip (NoC) not enough
The post Why NoC tiling matters in AI-centric SoC designs appeared first on EDN.