Apple Intelligence Server
About
Apple's Intelligence system harnesses the power of a family of large language models (LLMs) known for their robust architecture and adaptability. Central to this suite is the AFM-server, a massive cloud-based model crafted from the ground up. It significantly surpasses the compact, 3-billion parameter AFM-on-device model in size. Both models share a Transformer decoder-only structure and have been pre-trained on an extensive dataset of 6.3 trillion tokens. While the AFM-server is designed without pruning from a larger model, Apple enhances functionality using pluggable LoRA adapters, which allow the models to be dynamically adapted for specific tasks. This results in efficient and tailored performance without comprehensive retraining. In parallel, the OpenELM models, part of Apple’s innovation, utilize strategic layer-wise scaling for optimal parameter distribution, are available in various parameter sizes, and stand out as open-source, unlike the proprietary AFM models 136.