How to make your CPU as fast as a GPU - Advances in Sparsity w/ Nir Shavit

#ai #sparsity #gpu 

Sparsity is awesome, but only recently has it become possible to properly handle sparse models at good performance. Neural Magic does exactly this, using a plain CPU. No specialized hardware needed, just clever algorithms for pruning and forward-propagation of neural networks. Nir Shavit and I talk about how this is possible, what it means in terms of applications, and why sparsity should play a much larger role in the Deep Learning community.

Sponsor: AssemblyAI

Link: https://www.assemblyai.com/?utm_sourc...

Check out Neural Magic: https://neuralmagic.com/

and DeepSparse: https://github.com/neuralmagic/deepsp...

OUTLINE:

0:00 Introduction

1:08 Sponsor: AssemblyAI

2:50 Start of Interview

4:15 How the NIR company was founded? 

5:10 What is Sparsity about? 

9:30 Link between the human brain and sparsity

12:10 Where should the extra resource that the human brain doesn't have go?

14:40 Analogy for Sparse Architecture

16:48 Possible future for Sparse Architecture as standard architure for Neural Networks

20:08 Pruning & Sparsification

22:57 What keeps us from building sparse models?

25:34 Why are GPUs so unsuited for sparse models?

28:47 CPU and GPU in connection with memory

30:14 What Neural Magic does?

32:54 How do you deal with overlaps in tensor columns?

33:41 The best type of sparsity to execute tons of CPU

37:24 What kind of architecture would make the best use out of a combined system of CPUs and GPUs?

41:04 Graph Neural Networks in connection to sparsity

43:04 Intrinsic connection between the Sparsification of Neural Networks, Non Layer-Wise Computation, Blockchain Technology, Smart Contracts and Distributed Computing

45:23 Neural Magic's target audience

48:16 Is there a type of model where it works particularly well and the type where it doesn't?

Links:

Homepage: https://ykilcher.com

Merch: https://ykilcher.com/merch

YouTube:   

 / yannickilcher  

Twitter: https://twitter.com/ykilcher

Discord: https://ykilcher.com/discord

LinkedIn: https://www.linkedin.com/in/ykilcher

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):

SubscribeStar: https://www.subscribestar.com/yannick...

Patreon: https://www.patreon.com/yannickilcher

Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq

Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2

Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m

Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n