Apple Inc. (AAPL.US) AI dilemma solved? Negotiating with AI startup PrismML, hoping to slim down large models to fit into the iPhone.

Apple Inc. (AAPL.US) is currently in discussions with a small startup company, PrismML, in Silicon Valley. PrismML claims to have the ability to significantly compress powerful AI models, making them suitable for direct running on iPhones. Technological Breakthrough: 54GB model compressed to less than 4GB, 27 billion parameters running on iPhones PrismML is a spin-off company of the California Institute of Technology (Caltech) funded by Khosla Ventures, and on Tuesday they publicly released a compressed version of the open-source model Qwen sponsored by Alibaba Group Holding Limited ADR. The company stated that they have compressed the original model from about 54GB to less than 4GB, enabling all 27 billion parameters to run on iPhone 15 and newer models. PrismML CEO Babak Hassibi revealed that Apple Inc. and other companies are evaluating the startup's model, testing its operational speed, efficiency, and performance on devices. "They are indeed evaluating our technology at the moment," Hassibi said regarding Apple Inc. He described the discussions as being in a "very early" stage, with the final direction still unclear, but "things are progressing steadily." Apple Inc. AI Dilemma: Balancing edge intelligence and trade-offs The timing of PrismML's model release coincides with Apple Inc.'s official opening of iOS 27 public beta, allowing iPhone users to experience the long-delayed comprehensive upgrade of Siri for the first time. Apple Inc. is trying to make Siri more competitive in the competition with OpenAI and Anthropic, while still insisting on keeping more personal information and AI processing on the device side. However, the most powerful AI models often require a large amount of memory and computing power to run on smartphones, which poses a core contradiction in Apple Inc.'s AI strategy. Apple Inc.'s current approach is to send complex requests to cloud-based models for processing, but to run more AI capabilities directly on iPhones, reducing data transmission latency, lowering cloud computing costs, strengthening privacy protection commitments, and enabling certain functions to work normally even without network connection. Carolina Milanesi, President and Chief Analyst of Creative Strategies, pointed out that smaller models will allow Apple Inc. to move more challenging functions such as computational photography, video generation, and health or fitness tools that rely on sensitive personal data to run locally on iPhones. "The more that we can do on the device, the better the result," she indicated, using health and medication data as an example, emphasizing users' desire for such information to remain private. 16-bit simplification to "1 or 3 values," speed increases by 6 to 8 times PrismML stated that its compression technology achieves a significant simplification of how internal information in the model is stored - reducing each value from 16 bits to only 1 or 3 possible values, significantly reducing the memory required for model storage and operation. Hassibi likened this breakthrough to the evolution from 8-bit to 4-bit computing in the chip industry, but taking it a step further. Allegedly, the compressed model's memory usage is only one-tenth to one-fifteenth of the traditional version, with response speed increasing by 6 to 8 times and energy consumption decreasing by 3 to 6 times. However, Hassibi admitted to performance degradation. PrismML's models typically experience a loss of a few percentage points in overall performance, with the degradation of factual recall being higher than abilities such as reasoning, mathematical operations, and coding. PrismML released two compressed models, targeting Siasun Robot & Automation and autonomous driving PrismML released two compressed models for free this time, designed to run on daily devices, covering iPhones, MacBooks, and PCs with NVIDIA chips. This technology originated from Hassibi's research team at the California Institute of Technology (Caltech). The school holds the underlying patents and has exclusively authorized PrismML to use them. In March of this year, the company completed a $16.25 million seed round financing led by Khosla Ventures, with participation from multiple institutions. Hassibi revealed that Alphabet Inc. Class C's open-source model Gemma will be the next compression target, and thereafter, they will challenge larger scale models - including those typically requiring data center hardware to run from top laboratories. According to PrismML, the ultimate application scenarios for this technology will extend beyond smartphones and laptops to include Siasun Robot & Automation, autonomous driving systems, and other products requiring rapid decision-making without relying on cloud connections. "It is crucial that intelligence be localized and run quickly," Hassibi said. Apple Inc. has an advantage in edge processing, with scalability testing and endurance tests remaining to be resolved Apple Inc. has been running some AI functions locally on its devices, including translation, partial summarization, and features closely related to personal information. More complex requests are routed to Apple Inc.'s private cloud infrastructure or external third-party models. Horace Dediu, founder of Asymco, indicated that Apple Inc. is likely trying to keep the majority of common Siri interactions on the device side, only offloading the heaviest tasks to the cloud. He noted that the advantage lies not only in reducing memory usage but in hosting a more powerful model within the same physical constraints. "They're trying to figure out how big and how smart of a model can fit on a device," Dediu said. Keeping common requests local could bring Apple Inc. lower latency, stronger privacy protection, potential licensing fees, and reduced costs for cloud services. Due to its in-house design of iPhone chips and software, the integration of hardware and software gives Apple Inc. more refined control over running these models on devices, potentially giving them a unique advantage in deploying these models. Analysts are cautiously optimistic about PrismML's technology, emphasizing the need for tests beyond controlled demonstrations. Tarun Pathak, Research Director at Counterpoint Research, pointed out that the performance of models in handling long prompts, battery consumption in multitasking scenarios, and reliability in responding to millions of queries will be key. "The ultimate test will be millions of queries, thousands of device combinations, and large-scale robust testing," Pathak said. Phil Solis, Client Processor Research Director at IDC, believed that power consumption might be the biggest unknown. Even with reduced memory requirements for the model, if its capabilities are frequently used or continuously run in the background (such as for proxy tasks), it could still severely impact smartphone battery life. Chip Demand Dilemma: Efficiency improvements do not equal reduced demand As PrismML releases compressed models, the market is engaged in a fierce debate over whether AI efficiency improvements will ultimately reduce the need for storage chips and expensive data center infrastructure. Storage has become one of the biggest bottlenecks and cost factors in consumer electronics and AI server fields. Morgan Stanley predicts that the average cost per bit of DRAM for Apple Inc. in the fiscal year 2027 may increase by about 190% year-on-year, with NAND costs rising by about 180%. The institution estimates that Apple Inc. may raise the starting price of the iPhone 18 series by around $200 for similarly configured models to protect profit margins. PrismML stated that its technology could allow cloud-based models that previously required 8 GPUs to run with only 1, and furthermore, models that previously had to rely on servers could be migrated to smartphones and laptops. However, Gil Luria, an analyst at D.A. Davidson, pointed out that model compression will not eliminate the need for processors or storage, but will shift more chips from data centers to smartphones and other devices. "It's not that you don't need chips anymore, you still need GPUs, you still need memory," Luria said. He also added that running AI on personal devices may not be as efficient as shared data center infrastructure, as the chips in phones may be mostly idle most of the time. Moreover, efficiency breakthroughs often lead to increased usage rather than lower expenses - cheaper and faster AI will create new products and encourage users to run models more frequently. The public release of PrismML's version provides an opportunity for ordinary users and investors to verify its performance improvement outside the laboratory. For Apple Inc., running more powerful AI models on iPhones will help achieve a substantial upgrade of Siri without compromising on privacy protection and the benefits of hardware-software integration. Counterpoint's Pathak summarized, "The combination of cloud-side and edge AI can provide a more complete, efficient, and privacy-focused AI experience. Complex tasks will be offloaded to the cloud, while tasks involving privacy, latency sensitivity, and privacy will be executed on devices."