—First in a three-part series

Introduced in 2014,the RISC-V instruction set architecture has been evolving at a pace that Arm and x86 ISAs have never experienced.Initially,RISC-V cores were used solely for microcontrollers and applications that did not require high performance,but rather benefited from low cost and low power.Since RISC-V is an open-source architecture,it quickly gained popularity among dozens and then hundreds of companies,each of which contributed to further development of the ISA.

Nowadays,there are tiny RISC-V cores suitable for microcontrollers and DSPs,more advanced cores suitable for SSD controllers,Linux-capable cores for embedded applications,specialized cores that can be used for AI workloads,and“fat”cores that can serve data center and high-performance computing(HPC)applications.

In fact,since the RISC-V technology is so versatile and easily customizable by chip designers,it is very well suitable for AI and HPC applications that are developing very rapidly these days.It can take years to add support for a data format to an x86 or Arm microarchitecture because both ISAs are controlled by essentially three companies:AMD and Intel when it comes to x86 and Arm Holdings when it comes to Arm.

Yet,companies like Red Semiconductor,SemiDynamics,SiFive,Tenstorrent,MIPS,and Ventana Micro tend to advance their cores in terms of supported data formats and instructions much faster than anyone in the x86 or Arm worlds.

While RISC-V has yet to see its strategical infliction point in AI and HPC realms,it looks like the ISA has a lot of chances to get widespread adoption in market segments that benefit a lot from maximum cost-efficiency,flexibility,customizability and reduced dependency.Analysts do not expect RISC-V to gain a significant AI and HPC market share over the next few years,though it is entirely possible that the ISA will get much more widespread in the longer term.

In this first article of a three-part series,we had a chance to talk with analysts and developers of various RISC-V processors,including those who design general-purpose CPU IP cores and application-specific accelerator IPs,about RISC-V’s prospects in AI and HPC market segments as they are seen today.

RISC-V activity in AI and HPC is increasing

After RISC-V emerged on the radar in 2016,it did not get a lot of headlines as it was in its infancy and SiFive—the company founded by the original developers of RISC-V from the University of California Berkeley—only offered very basic IPs aimed at microcontrollers,which do not get a lot of attention even in business media.

But companies with significant resources that tend to control the whole stack of their products,from silicon to firmware to software,realized RISC-V’s potential.As a result,by the end of 2010s we witnessed adoption of RISC-V by industry giants like Nvidia(microcontrollers within GPUs),Qualcomm(modems),Samsung(for 5G mmWave RF),Seagate(HDD controllers)and Western Digital(SSD controllers).Then RISC-V International,the organization that oversees development of the RISC-V ISA,ratified the RISC-V Vector Extension(RVV)version 1.0 in 2021,which opened doors for the ISA to the worlds of AI and HPC computing.

Tenstorrent Grayskull AI accelerators based on RISC-V ISA(Source:Tenstorrent)

Consequently,starting in the late 2010s and early 2020s,announcements about RISC-V-based IP designed for performance demanding applications began to emerge.

“There was initially some discouraging commentary about RISC-V’s inability to perform graphics or AI acceleration,”Thomas Hackenberg,principal analyst for the memory and computing division at Yole Development,told EE Times.“This was alleviated with the RISC-V Vector Extension Version 1.0 ratified by RISC-V International in 2021.Without this,there was almost no interest in AI in HPC applications or anywhere.The RVA22 profile released in March 2023 also adds features like hypervisor support and further vector instructions,so it is just now reaching products for commercialization of AI acceleration.”

But even before the RVV extensions were introduced,there were signals where RISC-V was going.Among the key signals,there was the appointment of Jim Keller,a legendary high-performance CPU designer,to the CTO position at Tenstorrent.Others were the inception of Ventana Micro and introduction of SiFive’s high-performance cores aimed at HPC applications.To date,the industry has seen multiple AI and HPC-related announcements from the aforementioned companies and many others.In fact,the interest towards RISC-V in AI,HPC and consumer electronics spaces is accelerating,as observed by the comments from the following companies below.

“There is absolutely a growing demand for RISC-V in AI and HPC applications since the open RISC-V standard offers unmatched flexibility,extensibility and scalability,”Ian Ferguson,senior director at SiFive,told EE Times.

“[AI and HPC]markets are highly competitive,and they need every ounce of performance and power efficiency they can get,[so the interest towards RISC-V among AI and HPC companies is growing],”Roger Espasa,CEO of SemiDynamics,told EE Times.

“We have seen a dramatic increase[of interest]in RISC-V-based AI and HPC solutions,”said a spokesperson for Ventana Micro in an interview with EE Times.“We are seeing a lot of different companies looking at RISC-V for AI,from large hyperscalers to startups.”

“More and more companies are becoming attracted to RISC-V,”said Christophe Bianchi,CTO at Ansys,in an interview with EE Times.“For example,tech giants like Google have recognized the potential of RISC-V and announced that Android 15 would officially support microprocessors with the RISC-V architecture.”

“A wide range of industries and companies are evaluating RISC-V-based solutions for their future computing needs,from academic institutions to commercial data center providers and government agencies,”said James Prior,marketing executive at MIPS,in an interview with EE Times.

Analysts,who tend to look at the market that exists today,argue that the interest in RISC-V is in an emergent phase.

“Two to three years ago,the RISC-V ecosystem was just beginning to experience mass commercial adoption,and it is arguably still in that phase,”Yole’s Hackenberg said.“There is definitely more adoption or RISC-V compared to three years ago.However,this interest is still in a very emergent phase.While there are a number of RISC-V International member companies and institutions of higher learning working on developing this technology,there is little mass-market commercial adoption yet,but that is likely to change based on two trends:first,the growing success of RISC-V as a viable commercial application,and second,the huge demand for more power-and cost-efficient ways of handling AI processing.”

“RISC-V is accelerating now primarily due to adoption in SoC designs,”said Karl Freund,founder and principal analyst at Cambrian AI.“Think TVs,appliances and so on.AI and HPC remain a glint in the eye for RISC-V for now,but the flexibility and licensing terms could be attractive going forward.For embedded applications,adopting RISC-V is low risk.”

Ready for customization

On the one hand,being an open-source ISA,RISC-V is by definition ready for customization by its adopters,which is an important thing for rapidly developing applications,such as AI and HPC.On the other hand,there are standardized RISC-V profiles that ensure compatibility with already available software.These are among the key factors why companies working on processors for AI workloads are looking towards RISC-V.

SiFive’s HiFive P550 development board.(Source:SiFive)

“RISC-V’s open nature allows us to create highly specialized and optimized processors for AI applications,”said David Harold,head of business development at Red Semiconductor,in an interview with EE Times.“We have used open-source cores to evaluate structure.However,we are implementing our own core.”

“AI and HPC platform providers can build vertically integrated platforms targeted for their application,enabled by standardized profiles to use—plus the freedom to customize them,”MIPS’Prior said.“This is joined by a robust ecosystem of both commercial and open-source tools(libraries,compilers,tools and operating systems).Hardware vendor‘moats’are another concern that can be addressed with RISC-V,as hyperscalers will be implementing hardware based on an open specification,with strong open-source software toolchain support.This is a game-changer for the HPC and Gen AI communities as they can maintain autonomy to design and build bespoke solutions and platforms that fit their needs.”

“The big[advantage of RISC-V]is the combination of having both a standard you can build on top of,and the freedom to quickly adapt/customize for specific applications and learnings in these areas,”said the spokesperson for Ventana Micro.

“Most RISC-V developments contain a high level of customization,”Ansys’Bianchi said.“Hardware accelerators for RNNs,vector processing leveraging variable-length SIMD instruction set,and so on.”

In fact,in the best-case scenario,companies can develop a RISC-V-based SoC containing general-purpose cores,GPU clusters and NPU all using a similar software stack.

“The flexibility or RISC-V allows SoC teams working on AI and HPC solutions to add custom instructions and tailored solutions for both markets,reaching their ever-increasing performance targets,”SemiDynamics’Espasa said.“With RISC-V,you can tailor the CPU,the GPU and the NPU to your exact needs.Point in case,SemiDynamics has created an‘all-in-one’solution that fuses CPU,GPU and NPU into a single,unified ISA.This implies a great advantage:a single,unified software stack.Even better,this software stack is completely open,so you get another key advantage against a closed solution like CUDA.”

But the lack of license fees and customizability does not mean that RISC-V is a better fit compared to Arm or perhaps even x86 for all applications,warns Yole’s Hackenberg.Companies must have a robust business model to justify the development of a custom solution and experienced personnel to design it.

“While the lack of a license fee influences the emergence of new players or broadens an incumbent’s portfolio in the competitive landscape,the reality of financial savings is tricky in such a complex field,”Hackenberg said.“Without a robust business model and experienced talent,a company could easily find itself spending more than if it had licensed a standard solution and enlisted the source supplier’s support to optimize that IP for its own product.Customizing a RISC-V core could include a free license,but that is not usually the end of the cost involved.”

Among the critical aspects of a processor design is which data formats and instructions it is set to support as appropriate execution units take die space and affect cost and power.When it comes to x86 and Arm processors,there are essentially two companies that introduce new instructions and data formats to x86,and one company that does the same for Arm.The same applies to CUDA and CDNA:two GPU architectures for AI and HPC.One company develops CUDA,and one works on CDNA.With RISC-V,things are different.

“One of the important differences is that enhancements to the instruction set are not determined by one company,”SiFive’s Ferguson said.“Instead,in the RISC-V community the ability to define and implement new instructions is worked on collaboratively(when software needs a common approach)or individually(for optimization of first-party software).”

Quick to customize

Adding support of a new data format to Arm takes over two years,which is too slow for companies trying to address emerging applications.While this greatly helps to maintain ISA integrity,it makes it more difficult to adapt these architectures to emerging workloads—particularly in the AI and HPC spaces,two major gigatrends of today’s semiconductor market.

“[Adding an instruction or data format to RISC-V]is relatively simple,probably measured in weeks,”Ferguson said.“The bigger challenge relates to supporting that from a software perspective.If third-party support is needed,that is the more challenging piece of the equation.”

Meta’s MTIA inference accelerator.(Source:Meta)

“The process is well understood and defined,so it becomes a fairly quick job to add[an instruction or a data format]to our basic Maths Toolbox or algorithmic functions,”Red Semiconductor’s Harold said,but admitted that development of the company’s custom cores had not concluded.

“Addition of an instruction in RISC-V is a relatively straightforward process of coming up with an instruction definition,providing the needed justification for the instruction and driving it through the RISC-V International[RVI]Technical Steering Committee in either creating a new extension for the instruction set or add the new instruction to an existing extension,”explained the spokesperson for Ventana Micro.“The goal is always to drive all new instructions/extensions through RVI to prevent fragmentation in the industry.That said,you can always quickly blaze ahead to incubate an acceleration idea on top of the common base if you really need to get something done.That is what is bringing a lot of AI applications to RISC-V.”

While the addition of an instruction into a RISC-V design sounds easy,it is still a complex process that does not happen overnight as hardware implementation takes time and adds complexities to design.

“[Time that it takes to add an instruction or data format]will vary quite widely depending on the type of instruction or data format being included and if it requires changes to registers,lookup tables,buffers,etc.,”MIPS’Prior said.“Typically,there will be choices to make on how far to carry an implementation into hardware based on ability to simulate,verify and test the new functions.”

Of course,a new instruction requires software support.SemiDynamics says that it can first get its customer a beta RTL design(depending on the complexity,we are talking about weeks or months for simple designs and up to 12 months for a medium complexity design)and FPGA bitstream with new instructions well ahead of the first tape out to get software development started.As a result,when the actual hardware is ready,software stack development is well underway,which greatly speeds up time to market.

“We get a beta-quality RTL to the customer really fast,and we offer them an FPGA bitstream so they can start software development and performance tuning,”SemiDynamics’Espasa said.“While they are doing that development,we proceed with the full verification of the RTL provided,which takes a bit longer.Overlapping both parts greatly reduces the time to market overall.”

Safeguards for ISA integrity

RISC-V cores can be easily customized,and bigger companies can even design their own cores with custom instructions and various data formats support.However,this might affect software compatibility and ISA integrity at some point.If some company designs hardware that will run solely its own software,that will hardly ever become a problem.For other cases,there are some safeguards.

“If the customer owns the entirety of the software stack,then there are no real issues,”SiFive’s Ferguson said.“We saw this in the storage segment where Western Digital,which owns the software they wish to run on an SSD,was able to add custom instructions that optimized the implementation of the software on the platform.For applications that are running third-party software,the addition of instructions has to be done in a coordinated way.We have seen this in the RISC-V community with the definition and standardization of vector extensions.”

To ensure that the RISC-V ISA is developing consistently,the RISC-V International organization had formed working groups that define evolution of the instruction set in a particular direction.Also,there are RISC-V profiles that are designed to ensure compatibility between software and hardware.

“Complying with RISC-V profiles can ensure baseline compatibility between different platforms aimed at rich OS stacks and binary distributions,”Prior said.

“There are working groups looking at high-performance computing too with a view to advancing the ISA for further improvements on these types of workloads,”Ferguson explained.“Furthermore,RISC-V Profiles,which are a base ISA with one or more extensions that are specified as a group[such as RVA23],are designed to increase portability across different implementations.”

Perhaps the best way to ensure compatibility with existing applications is to develop a core that is 100%compliant with the ISA and then add differentiators(new instructions,new data formats,etc.)on top of it.

“Our approach does not‘break’anything about existing ISA or code but clearly testing is key to any IP,”Harold said.“Our core will be compliant with code developed in line with the RISC-V documentation,but our extensions open up new options for performance improvement,power reduction and security improvement for our customers.”