DeepSeek Disrupts AI Landscape with Innovative Sparsity Technique

The emergence of DeepSeek, an open-source large language model from a China-based hedge fund, poses a challenge to established AI companies like OpenAI by utilizing an innovative approach known as "sparsity." Researchers at Apple, led by Samir Abnar, have found that by selectively activating only certain parameters within the neural network, DeepSeek can function with lower computational demands while achieving competitive performance levels. Their study indicates that optimizing sparsity can enhance accuracy and reduce pretraining loss, suggesting a shift in the AI landscape that could empower smaller labs and researchers to develop effective models with fewer resources.

Opens on ZDNet