TOKYO - Fujitsu has developed a new artificial intelligence architecture called PHOTON that achieved up to 475 times the processing efficiency of a conventional Transformer model in research tests, raising the possibility of significantly reducing the cost of generative AI while eventually competing with services such as ChatGPT and Claude.
The Japanese technology group designed PHOTON by fundamentally reconsidering the Transformer architecture that underpins leading generative AI systems including OpenAI's ChatGPT, Anthropic's Claude and Google's Gemini. Rather than repeatedly scanning an entire sequence of text, PHOTON processes information hierarchically, grouping tokens into blocks and searching through compressed layers of information.
The project is being led by Fujitsu Research Director Yuma Ichikawa. PHOTON remains a relatively small experimental model with 1.2 billion parameters, far below advanced models that can reach hundreds of billions or even trillions of parameters, but Ichikawa said its performance should improve as the model grows.
"If the model becomes larger, we will have effectively won," Ichikawa said, expressing confidence that PHOTON could eventually surpass existing systems in both performance and cost.
PHOTON is designed to divide a question into multiple subqueries and process them simultaneously before combining the results into a single response. In one demonstration, the system broke a query into 66 separate tasks, investigated them in parallel and then used a language model to integrate the findings.
The architecture does not necessarily attempt to produce the best possible answer from a single processing sequence. Instead, it makes each sequence considerably lighter, allowing multiple approaches to run in parallel and collectively produce a stronger answer.
Fujitsu's research paper found that PHOTON could outperform a Transformer model with as few as nine parallel sequences, Ichikawa said. As the architecture is scaled up, the company expects it to handle much larger numbers of simultaneous tasks without the severe loss of speed or increase in memory consumption associated with current models.
The reported 475-fold improvement does not mean that PHOTON generates each word 475 times faster. The figure measures the number of tokens the model can produce using a fixed graphics processing unit, or GPU, compared with a conventional Transformer under the experimental configuration.
In other words, a GPU running PHOTON was able to generate up to 475 times as much language output per unit of computing capacity. Actual response speed depends on the model, hardware and number of parallel sequences being processed.
In the 66-query demonstration, a rough calculation suggested that PHOTON could process the parallel workload about eight times faster than a Transformer handling the same total amount of output. For a simple question requiring only five parallel sequences, the theoretical advantage under the same calculation could approach 90 times, although the actual number would depend on the system configuration.
Fujitsu plans to make the number of subqueries adaptive, allowing the model to devote more processing paths to difficult questions while using fewer for simple requests. The separate answers would then be combined by the language model itself, similar to the way some existing AI coding and multi-agent systems integrate the work of several agents.
The project reflects growing concern over the cost of running generative AI. Companies are increasingly deploying AI systems for software development and other complex tasks, but heavy use can generate extremely high computing bills.
Ichikawa said AI usage costs have also become an issue within Fujitsu and that PHOTON could offer one possible solution. Organizations with limited computing resources could use the architecture to lower costs, while those with extensive resources could employ more parallel processing to pursue performance beyond current Transformer models.
The Transformer was introduced by a Google research team in 2017 and became the foundation of modern large language models. It predicts the next token in a sequence by reviewing earlier tokens and determining which parts are most relevant.
To perform that process, Transformer models retain information known as a key-value cache, or KV cache. The cache allows the system to refer back to previous words and exchanges as it generates each new token.
The approach produces strong results, but memory requirements rise rapidly as conversations become longer or many responses are generated simultaneously. In heavily parallel workloads, the KV cache can consume more GPU memory than the model's own parameters, creating a major bottleneck.
A Transformer effectively looks back over the full history of a conversation every time it produces another word. Ichikawa compared the process with a person recalling every detail of an entire conversation before speaking each new word, an approach that is accurate but potentially wasteful.
Other architectures, including recurrent neural networks and systems such as Mamba, attempt to improve efficiency by compressing earlier information into a single evolving state. These models avoid scanning the full sequence each time, but information from the distant past can gradually weaken or disappear as more material is compressed into the same representation.
PHOTON seeks a middle path. It retains detailed information but arranges it in a hierarchy of chunks. Small groups of tokens are summarized into blocks, and those blocks can then be grouped into higher-level summaries.
The research model first combines four tokens into one block and then combines four of those blocks again, reducing the amount of information handled at the upper level to one-sixteenth of the original token sequence. Transformer mechanisms are still used inside the blocks to preserve and organize information.
Ichikawa described the conventional Transformer as a horizontal, token-by-token scanner because it moves across the entire sequence to identify relevant information. PHOTON instead performs what the research calls a vertical scan, searching through hierarchical summaries before accessing the detailed information needed to generate the next token.
The number and size of these layers can be adjusted. Fujitsu is also considering systems that could automatically decide how many tokens to place in each block and how many hierarchical levels to use, depending on the task.
Because the computational burden of examining a long sequence can grow roughly in proportion to the square of its length, reducing the effective sequence through hierarchical grouping can produce large efficiency gains. The maximum 475-fold result was achieved under the most favorable configuration reported in Fujitsu's paper and is not a fixed improvement for every task.
PHOTON still faces a fundamental performance gap at its present size. When it processes only one query and produces one answer, it generally performs worse than a comparable Transformer model.
Fujitsu's experiments, however, showed that this gap narrowed as model size increased. If single-query performance approaches that of Transformer systems while PHOTON retains its parallel-processing advantage, the company believes the architecture could become particularly powerful for long-context tasks and large-scale AI agents.
Fujitsu has worked on the project for about two years as part of a long-term effort to create a fundamentally different AI architecture. Ichikawa said developing models at this lower architectural layer will become increasingly important as the performance gap between openly available models and proprietary systems narrows.
The rapid progress of open models, including advanced systems developed in China, suggests that access to a large closed model alone may no longer provide a lasting advantage, he said. Competition could increasingly shift toward the underlying architecture and the ability to achieve better performance with fewer computing resources.
That challenge is especially important for Japan, which lacks the computing resources available to the largest American and Chinese technology companies. Ichikawa said changing the architecture may be essential if Japanese developers are to compete globally.
He also warned that Japan could fall behind if access to leading foreign proprietary models were restricted in the future. As AI becomes indispensable to industrial and software development, the country needs a domestically developed model capable of operating at the same level as leading overseas systems, he said.
Fujitsu ultimately intends PHOTON to compete directly with services such as Claude and ChatGPT. Ichikawa described the project as an ambitious, long-term initiative aimed not merely at adapting an overseas model for Japanese use, but at developing a candidate for the architecture that could succeed the Transformer.
Source: テレ東BIZ
