Announcement Center
Templar: Decentralized Large Language Model Pre-training Subnet in the Bittensor Ecosystem
CoinW Research Institute
Abstract
Templar (Subnet 3, abbreviated as SN3) is a decentralized large language model pretraining subnet built within the Bittensor ecosystem. Its core lies in transforming the extremely expensive and centralized AI large model pretraining process into an industrial-grade network system collaboratively powered by distributed computing resources worldwide, through innovative mechanisms such as the "SparseLoCo algorithm for communication compression" and the "Gauntlet protocol for contribution evaluation." Architecturally, it breaks the physical network limitations of traditional centralized data centers, not only lowering the threshold for computing power access but also achieving protection for sensitive data and resistance to censorship. From an ecosystem and data performance perspective, Templar has successfully delivered the "Covenant-72B" model with 72 billion parameters, demonstrating the absolute feasibility of decentralized training for industrial applications. Templar fills the heavy industrial foundational model training segment within the Bittensor ecosystem and has the long-term potential to evolve into a resilient open-source AI infrastructure capable of running parallel to Web2 tech giants.
1.Starting from typical representatives of the Web2 era: the current situation and limitations of large model training
1.1 Compute Monopoly in the Web2 Era
With the evolution of generative AI infrastructure, the distribution and control of computing power have become a core issue in global technology competition. In conventional understanding and the Web2 era, the compute moat for developing cutting-edge AI models has been established by a handful of tech giants. Large cloud providers such as AWS and Azure monopolize the underlying physical compute resources, while AI model service providers like OpenAI, Anthropic, and Google provide closed‑source or semi‑open‑source model training and inference services through massive centralized data centers composed of hundreds of thousands of high‑end GPUs.
1.2 Core Limitations of the Traditional Model
Although centralized giants have significantly advanced the development of frontier models, this highly capital‑ and hardware‑intensive approach has obvious limitations. The first is extreme centralization and high barriers: ordinary developers and small‑to‑medium‑sized enterprises can hardly participate, and innovation is monopolized by a few technology oligarchs. The second is systemic fragility: once a large centralized data center suffers a physical or network failure, it creates a single point of failure risk that can lead to widespread service outages. More critically, there are data privacy and compliance risks: user and enterprise data must be uploaded to the private clouds of these giants, raising significant concerns about data leaks. Finally, this architecture is highly vulnerable to specific hardware export bans or whitelist controls, lacking anti‑fragility against censorship and geopolitical factors.
2.Templar (SN3): Reconstruct large model pre-training using a 'decentralized network'
2.1What is Templar: A decentralized large model training laboratory
In order to break the monopoly of tech giants, Templar (SN3) was born. Unlike traditional platforms, it is not simply moving data around, but directly targets the most expensive and core part of the AI industry chain—the decentralized pre-training of ultra-large-scale language models. Simply put, the role Templar plays is similar to that of Anthropic or OpenAI in the Web2 realm; it can be understood as the large model training labs under these companies, but its operation is built on a distributed network.
2.2From Closed Systems to Open Collaboration: What Problem Templar Solved
The core change of Templar lies in transforming the original training process, which was confined within a single centralized data center, into a permissionless open collaborative network. It breaks the traditional whitelist mechanism, allowing any idle high-end GPU around the world to join or leave the network at any time, achieving truly permissionless participation and extremely high system resilience. At the same time, Templar adopts a federated learning model of 'data stays, models move,' where training data remains on the miner's local device, and the network only transmits encrypted and compressed gradient data, perfectly addressing the privacy challenges of training sensitive data across countries and institutions. The decentralized node architecture provides global AI research and development with an 'darknet computing power channel' to circumvent sanctions from any single sovereign country, achieving extreme censorship resistance.
2.3 Role in the TAO ecosystem: Core heavy industry pre-training base
In the ecosystem, what supports Templar is the top-tier Covenant AI team. They have built a clear division of labor: Basilica (SN39) serves as the infrastructure backbone network providing trustless computing power rental; Grail (SN81) is responsible for reinforcement learning fine-tuning to ensure the models align with human values; and Templar (SN3) is positioned as a 'heavy industry base,' specifically responsible for the core pre-training of from-scratch foundational models.
3.Core Architecture: How Large-Scale Pretraining Is Completed in the Network
3.1Distributed Collaboration: Breaking the Limits of Physical Networks
Traditional AI training relies heavily on high-speed proprietary fiber interconnects in centralized data centers. Templar, on the other hand, compresses the communication data between nodes by more than a hundred times through its unique SparseLoCo algorithm. This means that the network no longer depends on top-tier fiber; it only requires ordinary home internet broadband to connect computing nodes distributed across the globe for collaborative work.
3.2 Trustless Quality Control
In open networks without permission, to prevent miners from "slacking" or acting maliciously, Templar introduced the Gauntlet protocol. This is a rigorous two-layer filtering and dynamic credit rating (OpenSkill) system. Validators in the network strictly test the actual contribution of the data submitted by each miner to the model, and only nodes that genuinely improve the model quality can receive high scores and rewards, effectively eliminating the possibility of cheating from an architectural standpoint.
3.3 Industrial-grade Delivery Verification: Covenant-72B Model
This architecture has been thoroughly validated in practical engineering. Templar recently successfully delivered the 'Covenant-72B' model, which features a dense architecture with 72 billion parameters and spans a massive corpus of approximately 1.1 trillion tokens. More than 70 completely independent, geographically distributed peer nodes participated in this training project, allowing nodes to dynamically join or leave at any time. In zero-shot benchmark tests, Covenant-72B scored 67.1 on the MMLU test, fully comparable to Meta's LLaMA-2-70B model, which was trained in centralized data centers with enormous computing power, whereas Covenant-72B used only 1.1 trillion tokens, 50% of the latter. Its counterpart in the Web2 industry is similar to Anthropic's OPUS or OpenAI's Codex.
4.Incentive and Collaboration Mechanisms: How Online Collaboration Forms a 'Positive Cycle'
4.1Dynamic Tokenomics Driven (dTAO)
The efficient operation of Templar is inseparable from Bittensor's cutting-edge 'dynamic TAO (dTAO)' tokenomics. In this network:
Miners: Nodes distributed around the world contribute hardware computing power and perform local model training. After submitting high-quality training results, they earn token rewards.
Validator: Acts as the 'goalkeeper', responsible for inspecting miners' work, maintaining system fairness and security, and ensuring the model evolves in the right direction.
Investor: Purchasing and staking Templar's standalone Alpha tokens is essentially a leveraged bet on the future model quality and computing power scale of the subnet.
4.2 The ecological closed loop of computing power, value, and network
In this free market game, model quality is deeply tied to economic returns. The better the models produced by Templar, the higher the value of its tokens; and high-value tokens, in turn, attract more and stronger computing power miners from around the world to migrate here. This mechanism creates a positive flywheel that tightly binds computing power, value, and network effects, forming an unbreakable ecological loop.
5.Ecological Status
From the current ecological operation perspective, the Templar project already possesses a considerable market scale. Its current market value is 88 million USD. In terms of participant structure, more than 70 independent and distributed peer nodes have substantially participated in the joint pre-training project of the 72B-level ultra-large model. As the Covenant AI team builds peripheral subnets such as SN39 (computing power leasing) and SN81 (reinforcement learning fine-tuning) around SN3, a complete upstream and downstream network system centered on decentralized large model production has initially begun to take shape.
As of April 7, the price of Templar's alpha token is approximately 0.071 TAO, with around 7,519 token-holding addresses, 253 miners, 3 validators, and an emission ratio of 2.58%. Meanwhile, in its liquidity pool, the TAO proportion is 6.71%, and the Alpha proportion is 93.29%. Based on price and token-holding numbers, Templar has established a certain user base and level of attention, but it is still in the early diffusion stage overall.
Data Resource：https://bittensormarketcap.com/subnets/3
6.Competitive Landscape and Advantages and Disadvantages
6.1 Industry Positioning and Core Competencies
In the context of highly centralized computing resources, Templar provides the industry with a technical solution capable of resisting single-point risks. Its core advantage lies in a paradigm shift driven by mechanism design: it achieves truly permissionless participation and utilizes federated learning to safeguard data sovereignty and privacy. By combining game theory mechanisms with distributed training, Templar greatly reduces developers' dependence on a single computing power supplier and builds computing channels that can circumvent censorship.
6.2 Continuously Evolving Challenges and Planning
Decentralized training faces long-standing challenges of network communication bottlenecks and hardware heterogeneity as it scales up. In response, Templar has planned continuous optimization and refinement at the technical level. On one hand, asynchronous training optimization is pursued through algorithm upgrades and community geek rewards, aiming to completely eliminate stuttering caused by network limitations. On the other hand, Templar is developing a new generation of heterogeneous architecture, designed to allow widely deployed consumer-grade graphics cards (such as the RTX 4090) to combine into virtual clusters, truly awakening the massive long-tail computing power in the hands of the public.
7.Future Outlook: Can Decentralized Industrial-Grade Large Models Be Established?
From the current stage, Templar (SN3) has successfully demonstrated the feasibility of delivering high-performance industrial-grade models through distributed computing power by delivering the Covenant-72B. Whether it can make further breakthroughs in the future depends on the establishment of a commercial closed loop and the introduction of liquidity.
In terms of commercial implementation, Templar plans to introduce a "token-gated" mechanism to provide token holders with exclusive access to high-end models, aiming to achieve a complete closed loop from R&D to commercial value capture. On a more macro financial level, as the compliance of the crypto industry accelerates (such as the approval of ETFs by TAO), the inflow of traditional institutional capital will directly increase its economic budget, thereby attracting higher-quality nodes to join in attempts to train next-generation large models.
In the long-term projection, the maturity of AI agent technology will bring massive demand, and Templar is expected to provide permissionless computing and payment infrastructure for it. Templar is not just a supplement to the Web2 model; it is also promoting AI technology toward open-source transparency, exploring a resilient path that can run parallel to centralized labs and facilitate global collaboration.
Reference
Templar Official Website: https://www.tplr.ai/
Templar Twitter: https://x.com/tplr_ai
Templar Discord: https://discord.com/invite/N5xgygBJ9r
Taostats K-line: https://taostats.io/subnets/3/chart
bittensormarketcap data：https://bittensormarketcap.com/subnets/3
Templar: Decentralized Large Language Model Pre-training Subnet in the Bittensor Ecosystem
CoinW Research Institute
Abstract
Templar (Subnet 3, abbreviated as SN3) is a decentralized large language model pretraining subnet built within the Bittensor ecosystem. Its core lies in transforming the extremely expensive and centralized AI large model pretraining process into an industrial-grade network system collaboratively powered by distributed computing resources worldwide, through innovative mechanisms such as the "SparseLoCo algorithm for communication compression" and the "Gauntlet protocol for contribution evaluation." Architecturally, it breaks the physical network limitations of traditional centralized data centers, not only lowering the threshold for computing power access but also achieving protection for sensitive data and resistance to censorship. From an ecosystem and data performance perspective, Templar has successfully delivered the "Covenant-72B" model with 72 billion parameters, demonstrating the absolute feasibility of decentralized training for industrial applications. Templar fills the heavy industrial foundational model training segment within the Bittensor ecosystem and has the long-term potential to evolve into a resilient open-source AI infrastructure capable of running parallel to Web2 tech giants.
1.Starting from typical representatives of the Web2 era: the current situation and limitations of large model training
1.1 Compute Monopoly in the Web2 Era
With the evolution of generative AI infrastructure, the distribution and control of computing power have become a core issue in global technology competition. In conventional understanding and the Web2 era, the compute moat for developing cutting-edge AI models has been established by a handful of tech giants. Large cloud providers such as AWS and Azure monopolize the underlying physical compute resources, while AI model service providers like OpenAI, Anthropic, and Google provide closed‑source or semi‑open‑source model training and inference services through massive centralized data centers composed of hundreds of thousands of high‑end GPUs.
1.2 Core Limitations of the Traditional Model
Although centralized giants have significantly advanced the development of frontier models, this highly capital‑ and hardware‑intensive approach has obvious limitations. The first is extreme centralization and high barriers: ordinary developers and small‑to‑medium‑sized enterprises can hardly participate, and innovation is monopolized by a few technology oligarchs. The second is systemic fragility: once a large centralized data center suffers a physical or network failure, it creates a single point of failure risk that can lead to widespread service outages. More critically, there are data privacy and compliance risks: user and enterprise data must be uploaded to the private clouds of these giants, raising significant concerns about data leaks. Finally, this architecture is highly vulnerable to specific hardware export bans or whitelist controls, lacking anti‑fragility against censorship and geopolitical factors.
2.Templar (SN3): Reconstruct large model pre-training using a 'decentralized network'
2.1What is Templar: A decentralized large model training laboratory
In order to break the monopoly of tech giants, Templar (SN3) was born. Unlike traditional platforms, it is not simply moving data around, but directly targets the most expensive and core part of the AI industry chain—the decentralized pre-training of ultra-large-scale language models. Simply put, the role Templar plays is similar to that of Anthropic or OpenAI in the Web2 realm; it can be understood as the large model training labs under these companies, but its operation is built on a distributed network.
2.2From Closed Systems to Open Collaboration: What Problem Templar Solved
The core change of Templar lies in transforming the original training process, which was confined within a single centralized data center, into a permissionless open collaborative network. It breaks the traditional whitelist mechanism, allowing any idle high-end GPU around the world to join or leave the network at any time, achieving truly permissionless participation and extremely high system resilience. At the same time, Templar adopts a federated learning model of 'data stays, models move,' where training data remains on the miner's local device, and the network only transmits encrypted and compressed gradient data, perfectly addressing the privacy challenges of training sensitive data across countries and institutions. The decentralized node architecture provides global AI research and development with an 'darknet computing power channel' to circumvent sanctions from any single sovereign country, achieving extreme censorship resistance.
2.3 Role in the TAO ecosystem: Core heavy industry pre-training base
In the ecosystem, what supports Templar is the top-tier Covenant AI team. They have built a clear division of labor: Basilica (SN39) serves as the infrastructure backbone network providing trustless computing power rental; Grail (SN81) is responsible for reinforcement learning fine-tuning to ensure the models align with human values; and Templar (SN3) is positioned as a 'heavy industry base,' specifically responsible for the core pre-training of from-scratch foundational models.
3.Core Architecture: How Large-Scale Pretraining Is Completed in the Network
3.1Distributed Collaboration: Breaking the Limits of Physical Networks
Traditional AI training relies heavily on high-speed proprietary fiber interconnects in centralized data centers. Templar, on the other hand, compresses the communication data between nodes by more than a hundred times through its unique SparseLoCo algorithm. This means that the network no longer depends on top-tier fiber; it only requires ordinary home internet broadband to connect computing nodes distributed across the globe for collaborative work.
3.2 Trustless Quality Control
In open networks without permission, to prevent miners from "slacking" or acting maliciously, Templar introduced the Gauntlet protocol. This is a rigorous two-layer filtering and dynamic credit rating (OpenSkill) system. Validators in the network strictly test the actual contribution of the data submitted by each miner to the model, and only nodes that genuinely improve the model quality can receive high scores and rewards, effectively eliminating the possibility of cheating from an architectural standpoint.
3.3 Industrial-grade Delivery Verification: Covenant-72B Model
This architecture has been thoroughly validated in practical engineering. Templar recently successfully delivered the 'Covenant-72B' model, which features a dense architecture with 72 billion parameters and spans a massive corpus of approximately 1.1 trillion tokens. More than 70 completely independent, geographically distributed peer nodes participated in this training project, allowing nodes to dynamically join or leave at any time. In zero-shot benchmark tests, Covenant-72B scored 67.1 on the MMLU test, fully comparable to Meta's LLaMA-2-70B model, which was trained in centralized data centers with enormous computing power, whereas Covenant-72B used only 1.1 trillion tokens, 50% of the latter. Its counterpart in the Web2 industry is similar to Anthropic's OPUS or OpenAI's Codex.
4.Incentive and Collaboration Mechanisms: How Online Collaboration Forms a 'Positive Cycle'
4.1Dynamic Tokenomics Driven (dTAO)
The efficient operation of Templar is inseparable from Bittensor's cutting-edge 'dynamic TAO (dTAO)' tokenomics. In this network:
Miners: Nodes distributed around the world contribute hardware computing power and perform local model training. After submitting high-quality training results, they earn token rewards.
Validator: Acts as the 'goalkeeper', responsible for inspecting miners' work, maintaining system fairness and security, and ensuring the model evolves in the right direction.
Investor: Purchasing and staking Templar's standalone Alpha tokens is essentially a leveraged bet on the future model quality and computing power scale of the subnet.
4.2 The ecological closed loop of computing power, value, and network
In this free market game, model quality is deeply tied to economic returns. The better the models produced by Templar, the higher the value of its tokens; and high-value tokens, in turn, attract more and stronger computing power miners from around the world to migrate here. This mechanism creates a positive flywheel that tightly binds computing power, value, and network effects, forming an unbreakable ecological loop.
5.Ecological Status
From the current ecological operation perspective, the Templar project already possesses a considerable market scale. Its current market value is 88 million USD. In terms of participant structure, more than 70 independent and distributed peer nodes have substantially participated in the joint pre-training project of the 72B-level ultra-large model. As the Covenant AI team builds peripheral subnets such as SN39 (computing power leasing) and SN81 (reinforcement learning fine-tuning) around SN3, a complete upstream and downstream network system centered on decentralized large model production has initially begun to take shape.
As of April 7, the price of Templar's alpha token is approximately 0.071 TAO, with around 7,519 token-holding addresses, 253 miners, 3 validators, and an emission ratio of 2.58%. Meanwhile, in its liquidity pool, the TAO proportion is 6.71%, and the Alpha proportion is 93.29%. Based on price and token-holding numbers, Templar has established a certain user base and level of attention, but it is still in the early diffusion stage overall.
Data Resource：https://bittensormarketcap.com/subnets/3
6.Competitive Landscape and Advantages and Disadvantages
6.1 Industry Positioning and Core Competencies
In the context of highly centralized computing resources, Templar provides the industry with a technical solution capable of resisting single-point risks. Its core advantage lies in a paradigm shift driven by mechanism design: it achieves truly permissionless participation and utilizes federated learning to safeguard data sovereignty and privacy. By combining game theory mechanisms with distributed training, Templar greatly reduces developers' dependence on a single computing power supplier and builds computing channels that can circumvent censorship.
6.2 Continuously Evolving Challenges and Planning
Decentralized training faces long-standing challenges of network communication bottlenecks and hardware heterogeneity as it scales up. In response, Templar has planned continuous optimization and refinement at the technical level. On one hand, asynchronous training optimization is pursued through algorithm upgrades and community geek rewards, aiming to completely eliminate stuttering caused by network limitations. On the other hand, Templar is developing a new generation of heterogeneous architecture, designed to allow widely deployed consumer-grade graphics cards (such as the RTX 4090) to combine into virtual clusters, truly awakening the massive long-tail computing power in the hands of the public.
7.Future Outlook: Can Decentralized Industrial-Grade Large Models Be Established?
From the current stage, Templar (SN3) has successfully demonstrated the feasibility of delivering high-performance industrial-grade models through distributed computing power by delivering the Covenant-72B. Whether it can make further breakthroughs in the future depends on the establishment of a commercial closed loop and the introduction of liquidity.
In terms of commercial implementation, Templar plans to introduce a "token-gated" mechanism to provide token holders with exclusive access to high-end models, aiming to achieve a complete closed loop from R&D to commercial value capture. On a more macro financial level, as the compliance of the crypto industry accelerates (such as the approval of ETFs by TAO), the inflow of traditional institutional capital will directly increase its economic budget, thereby attracting higher-quality nodes to join in attempts to train next-generation large models.
In the long-term projection, the maturity of AI agent technology will bring massive demand, and Templar is expected to provide permissionless computing and payment infrastructure for it. Templar is not just a supplement to the Web2 model; it is also promoting AI technology toward open-source transparency, exploring a resilient path that can run parallel to centralized labs and facilitate global collaboration.
Reference
Templar Official Website: https://www.tplr.ai/
Templar Twitter: https://x.com/tplr_ai
Templar Discord: https://discord.com/invite/N5xgygBJ9r
Taostats K-line: https://taostats.io/subnets/3/chart
bittensormarketcap data：https://bittensormarketcap.com/subnets/3