Fugaku-LLM
About
The Fugaku-LLM family comprises large language models developed using the Fugaku supercomputer, one of the most powerful computational systems in Japan. These models, including the Fugaku-LLM-13B, are distinguished by their 13 billion parameters, making them significantly larger than many other Japanese models, which typically have fewer than 7 billion parameters. Fugaku-LLM models are trained from scratch using proprietary Japanese data, ensuring high transparency and safety. They excel in Japanese language tasks, achieving top scores in benchmarks like the Japanese MT-Bench, particularly in humanities and social sciences. The models are designed for both research and commercial applications, leveraging advanced distributed parallel learning techniques to maximize the supercomputer's performance.