Interview with Jim Keller, CEO of TensTorrent
“If you give me 18 months, I’ll make a semiconductor for autonomous driving.”
In early 2016, an engineer told Tesla CEO Elon Musk, who was focusing on advancing autonomous driving technology. After thinking about it for a while, Musk quickly replied, “Okay. Instead, I ask you to be prepared in case you fail,” he says, accepting his offer. To everyone’s skepticism, the engineer gave Musk the self-driving chip “HW (hardware) 3” exactly 18 months later. It was a decisive moment when Tesla laid the foundation for Full Self-Driving (FSD) technology. The person who made this happen at the time was Jim Keller, who is called the “legend of semiconductor design.” Not only Tesla’s self-driving chips, but also the ‘Athlon’ and ‘Ryzen’ series that made AMD what it is today were born through his hands. Apple has created the mobile application processors (APs) ‘A4’ and ‘A5’, which are the brains of smartphones.
I recently caught up with Jim Keller on Zoom. He is currently the CEO of TensTorrent, a Canadian artificial intelligence (AI) semiconductor company. Here, he is spearheading the design of AI semiconductors. The AI semiconductor ‘Grayskull’ unveiled by TensTorrent this year boasts a performance capable of up to 315 trillion operations per second (TOPS) at INT8 (8-bit integer). On the other hand, it consumes 200W of power, which is more power-efficient than other AI semiconductors in the industry. “High power efficiency and open source are TensTorrent’s competitive advantages,” he explains. In particular, he was optimistic about the future that AI will bring. A self-proclaimed AI optimist, he said, “AI is a disruptive technology, but humans have handled it well before, and AI will create more new opportunities for humanity.” Also known as the “Wizard of Semiconductors,” how does he see the future of the semiconductor industry in the AI era? The following is a question and answer.
“The basics are the best… We need to teach art and basic science rather than coding.”
―As the head of an AI semiconductor company, do you often talk to AIs like ChatGPT?
“In fact, I rarely talk to AI. There is no particular reason. I’ve spent years designing chips for video games, but I’ve never played video games. Instead, I read a book.”
―With the rapid development of AI, there is a fierce debate between “doomers” (doomers) who say that AI development should be slowed down and “boomers” (developmentists) who say that AI should be actively utilized. Which one are you?
“I don’t think it’s a doomer (laughs). It’s more like a boomer. AI is a very interesting and disruptive technology. But haven’t humans been good at dealing with destructive technology before? We already live in a world where differences in intelligence, wealth, and the size of companies exist among people. AI can widen these gaps, but it can also create new opportunities for humanity. We’ll see what happens in the future.”
― Recently, AI has predicted that it will replace all software.
“Yes. I’m convinced. Just as CD-ROMs disappeared and smartphones replaced computers, AI will quickly replace traditional software. In the past, humans wrote thousands of lines of code for speech recognition, but now AI recognizes speech through a ‘neural network’ (a computing technology that mimics a human neural network). In the future, traditional programs will be replaced by AI-generated programs, which will be reviewed by humans. Today, it takes millions of hours to perfect self-driving car technology, but AI could soon shorten that time, just like my daughters learned to drive in 30 minutes. They didn’t have to drive millions of miles. Change is already beginning.”
―There are a lot of young engineers who want to work in AI, what advice would you give?
“AI is a very new field. There is a lot to learn. It’s good to jump into. However, engineers working for large companies are often forced to focus on creating new versions of old products. This means that it is difficult to learn new things. This kind of thing will be fine for 1~2 years, but eventually, you have to find a place where you can take on a new challenge.”
―How should we teach the next generation in the context of rapid AI development?
“We need to teach art and basic science. Now I teach programming in high school and CAD in college, and it’s crazy. You have to read, you have to write, you have to think, you have to make art, you have to do theater, you have to learn a musical instrument, you have to learn mathematics, physics, chemistry, biology, and history. The basics are always the best. I never studied computer science or computer architecture in college. Instead, he studied physics, electrical engineering, and the theory of electromagnetism established in the 1850s. He also studied calculus, astronomy, biology, and philosophy, which were created in the 1600s. I also read books by ancient Greek philosophers and Sun Tzu’s Tactics. In fact, I’m pretty dyslexic. It’s hard to write. But I really liked math. The mathematical formulas ‘Laplace Transformation’ and ‘Fourier Transformation’ that were born in the 18th~19th century were the best things I learned. They helped me not only learn how to think,- but also to find what I liked.
“Open source always wins”
I asked Keller about TensTorrent, which is the head. He joined TensTorrent at the end of 2020, leaving Intel as Senior Vice President to join TensTorrent as its Chief Technology Officer. He was then appointed CEO in January last year. Recently, Grace Skull is preparing to release its sequel, ‘Quasar’, which will be manufactured at the foundry (semiconductor contract manufacturing) plant that Samsung is building in Texas, USA.
―What is the scheduled release date of the quasar?
“We expect it to be around the end of this year. We plan to sell these semiconductors to general manufacturing companies, and to a wide range of customers, including chip developers, data center developers, and software developers. AI semiconductors will be used in more and more products. That means our audience is very diverse.”
―Samsung Electronics Foundry was entrusted with the production of quasars. Why did you choose Samsung instead of TSMC?
“We’ve worked with both Samsung and TSMC. Earlier, TSMC made our own 6nm (nanometer, billionth of a meter) semiconductor, and Samsung will make the next generation of 4nm semiconductor quasars. It is the result of a comprehensive consideration of each company’s technology level, readiness, patent (IP) availability, price, and design support. Samsung Electronics (KWR 75,300 ▼ 400 -0.53%) is a great partner. I have been working with Samsung since 1994. Of course, TSMC is also a good partner. Both companies are companies that I trust.”
― Recently, we also established a branch in Korea.
“It’s to support our Korean customers. Right now, we have only a handful of people working in our Korean office, but we plan to hire tech talent soon. There are a lot of great engineers in Korea.”
―TensTorrent’s semiconductors are based on the Risk Five (RISC-V) architecture. Do you think Risk Five can outperform the architecture of major companies?
“Absolutely. Currently, AMD and Intel have a monopoly on their architecture, preventing other companies from using it. If you want to use ARM’s architecture, you’ll have to pay a lot of money. Risk Five, on the other hand, is open-source that anyone can use. Even now, about 20~30 companies use Risk Five technology. As a result, Intel· Risk Five will apply not only to AMD and Nvidia’s chips-, but also to chips that Google and Meta will use.”
―Is it the power of open source?
“Yes. Open source always wins. More people work together to innovate. Risk Five will become the standard for next-generation architectures.”
Risk Five is an open-source instruction set structure (ISA) developed by UC Berkeley in 2010. ISAs from companies like ARM require you to purchase a license, but Risk Five is available for free. This gives anyone the freedom to design and use CPUs.
―It seems that NVIDIA has a virtual monopoly on the AI semiconductor market right now, but what is TensTorrent’s path forward?
“First, let’s just look at the AI market. NVIDIA doesn’t have a monopoly on the entire AI market. The AI market is huge. Much of the AI runs on Intel and AMD’s computers via the cloud. APPLE USES AI ON THE IPHONE AND MAC. Of course, Nvidia is indeed competitive in the market for high-cost, high-performance chips. However, this is not TensTorrent’s target market. We have the distinction of having excellent cost-effectiveness of our products. It doesn’t compete directly with Nvidia. Create products for markets with different needs. It has the advantage of creating processors that run at lower power and making the technology easily extensible through open source. AI engines and CPU technologies are also provided through licensing so that anyone can create their products. The media is focused on Nvidia, which has a high market cap, but there are plenty of opportunities in other markets as well.”
Meta, OpenAI, and even many Big Tech are trying to produce their own AI chips. Isn’t it bad news for traditional semiconductor companies?
“It’s not necessarily a bad thing. For example, AMD and Intel have a wide range of business areas. We don’t know the exact percentage, but AMD and Intel probably have about 30% of their business related to large enterprise customers. Its customers include small companies and consumers who don’t make their chips. In addition, traditional semiconductor companies have invested heavily in many cutting-edge technologies as well as AI. You’ll be able to find a new market or niche. The current changes in the market can be overcome sufficiently. The AI semiconductor market is still evolving. I think there’s an opportunity for both new and old players.”
―Do you have any plans for an initial public offering (IPO)?
“Absolutely. We’re going to have an IPO in a few years.”
“I’m not interested in money, promotion… It’s just good to take on a new challenge.”
Jim Keller has written several masterpieces in semiconductor history over the past few decades. But he never stayed in one company. If you come up with a groundbreaking product, they will move on to another company. That’s why the industry calls him a “fixer.” What is the secret of how he was able to come up with innovative products every time?
―AMD, Apple, Intel, Tesla, and many others have left amazing products. I could have stayed in one place and enjoyed my success, but I’m curious as to why I moved from one company to another.
“I love the whole thing. My goal is to find new opportunities and challenges. Apple made several products, but after Steve Jobs’ death, he became interested in new opportunities. That’s why AMD successfully created a Zen processor and design team, and Tesla completed the self-driving chip “HW3″ in 18 months. Later, ‘HW4’ and the autonomous driving supercomputer ‘Dojo’ were born based on this.”
―Does this mean that if you finish another product on TensTorrent, you can leave for another place?
“It’s not. We’ve got a few products that have already been designed, and we’re creating some cool new ones. I have a lot of ideas. My job is to make TensTorrent very successful. There is a lot to do. And I’m having a lot of fun doing what I’m doing.”
―Is your goal to grow TensTorrent into a company like Nvidia, Intel, or AMD?
“Absolutely not. Customers have already partnered with NVIDIA, and Intel· I’m using AMD products. I want to create another company. Musk also did not say that he would make an electric car like Toyota. He went where electric cars led him, and he ended up creating a new kind of car.”
―Then, what are the conditions for becoming a winner in the AI semiconductor market?
“There are several. First, it’s important to have curiosity, problem-solving skills, enthusiasm, and a hard-working attitude. In particular, AI chip performance is crucial to stand out among competitors. When we buy a computer, we usually consider performance, programmability, availability, business conditions, licensing, power consumption, and so on. If you’re in this market, you need to have an edge in one or more of them.”
―As AI becomes more sophisticated, will there be changes in the von Neumann architecture, a traditional computer structure consisting of computing devices and storage devices?
“The three main categories of computer technology have not changed for a long time. Computation, memory, and input/output (IO). The calculation is the most complicated part. Modern computers have CPUs, GPUs (graphics processing units), TPUs (tensor processing units), and TPUs (tensor processing units). AI-only semiconductors) and three calculation methods. But the paradigm is changing. In the past, 90 percent of a computer’s calculations were done by the CPU and 10 percent by the GPU, but in the future, the CPU will do 10 percent, the GPU will do 10 percent, and the TPU will do 80 percent.”
―You’ve worked with tech giants from Steve Jobs to Elon Musk. What advice would you give to future tech leaders based on your experience at that time?
“You have to enjoy your work. When I was younger, my friend’s wife joked to us, ‘What do you drink with water?’ and we worked all day and talked about work all night. She loved her work and was passionate about it. If you don’t like what you’re doing or you don’t like your boss, quit that company and find a place where you can work with someone you’re passionate about.”
―Lastly, tell us how you have been able to be innovative every time.
“There is no secret. Focus on what needs to be done and what problems to solve. And while many people work for promotion and money, I don’t. My only concern is to solve the problem. And you have to be very open. If you don’t know something, ask questions. Ever since I was a child, if I didn’t understand, I would ask questions. Some people don’t ask questions to look smart, but then they don’t learn anything. It’s a mistake that young people usually make. Don’t try to be smart, try to learn.”