#agi #ai #llm #technology #tiiny | Norman Paulsen | 28 comments

120 billion parameters now fit in your pocket, without a cloud connection. The Tiiny AI Pocket Lab, officially verified by Guinness World Records, represents a significant shift in how large language models will be distributed and accessed. This pocket-sized device, measuring 14.2 x 8 x 2.53 cm and weighing 300 grams, operates as a complete AI inference system capable of running models with up to 120 billion parameters entirely offline. The technical architecture challenges conventional assumptions about computational requirements for advanced LLMs. The device combines an ARMv9.2 12-core CPU with a dedicated neural processing unit delivering approximately 190 TOPS of AI compute, supported by 80GB of LPDDR5X memory and 1TB of storage, accomplishing what previously required cloud infrastructure or high end GPUs. By eliminating dependency on cloud servers, the device reduces operational costs, removes latency constraints, and addresses sustainability concerns inherent to data center scale operations, making advanced AI models viable for individual users and resource constrained environments. The architectural innovation relies on two proprietary optimization techniques: TurboSparse, which activates only necessary neurons, and PowerInfer, an open-source engine that distributes workloads between CPU and NPU for efficiency. What would you do with a pocket size AI? #AGI #AI #LLM #Technology #Tiiny | 28 comments on LinkedIn