🐰 Research Highlight — Hyperbolic

This week, we’re diving into Hyperbolic, an open-access AI cloud making waves.

Hyperbolic’s bold mission is to democratize AI by offering affordable compute power for inference.

But before that, let’s kick things off with what we found most interesting about Hyperbolic…

Hyperbolic’s Secret Sauce—Proof of Sampling

Hyperbolic is pushing boundaries by addressing one of the toughest challenges in AI: verifying that an output truly comes from the specified AI model

This issue is especially tricky with centralized, closed-source providers like OpenAI. When you ask GPT-4 for an output, how can you be sure you’re not getting shortchanged—say, by OpenAI running a cheaper GPT-3.5 model instead (at 1/20 of the price per token)?

Currently, this assurance relies on reputation, but Hyperbolic believes this should be handled in a trustless, decentralized way.

There are a few ways to do this currently:

• Optimistic Machine Learning (OpML): Assumes all transactions are valid unless challenged by a validator.

• Zero Knowledge Machine Learning (zkML): Uses ZK circuits to verify that computations were performed correctly.

However, both have limitations:

• OpML relies on validators to check results, leading to delayed finality due to dispute periods. Plus, there’s no inherent incentive to ensure validators act honestly.

• zkML is computationally extremely heavy, sometimes taking days to generate proof for large models with 70B+ parameters.

Hyperbolic aims to overcome these drawbacks with its Proof of Sampling (PoSP) protocol and Sampling Machine Learning (SpML). SpML leverages sampling and game theory to encourage honest behaviour without constant oversight.

It’s based on a game-theoretic concept known as the pure strategy Nash Equilibrium, where all participants have a clear incentive to act honestly because the costs of cheating outweigh the potential gains.

The easiest mental model for this is to think of it as a public bus ticketing system.

Ticket inspectors conduct random checks only, so you might think passengers would frequently risk not buying a ticket. Surprisingly, they don’t because the penalties are high enough to deter cheating. As long as the penalty far exceeds the ticket's cost, honesty prevails.

Hyperbolic’s SpML uses economic incentives to address the limitations of current verification mechanisms like OpML and zkML. It provides a balance of speed and security without a heavy computational burden.

The caveat? It assumes that everyone acts rationally, which may not always be the case.

If SpML works well in practice, it will be a game-changer for decentralized AI applications, making trustless, verified inference a reality.

Cheap and Scalable Compute

Training AI is expensive.

Electricity and access to compute are some of the biggest costs to companies and startups. The cost of the computational power required to train models doubles every nine months.

GPT-3 costs around $4M (2020), while GPT-4 (2023) costs a staggering $190M of computing to train.

Only well-resourced organizations can keep up at this rate. Smaller participants and retail enthusiasts are priced out. One Stanford postdoc was blocked from conducting his research because he couldn’t afford the thousands of GPUs needed.

One major challenge in decentralized compute networks is managing heterogeneous hardware—not just top-tier Nvidia chips but a broad spectrum of GPUs.

Hyperbolic’s Decentralized Operating System is the core of its compute network. It will seamlessly cluster resources with built-in auto-scaling and fault tolerance.

Hyperbolic’s breakthrough is in how it handles this complexity.

• They offer flexibility by optimizing tensor operations across diverse hardware, from Nvidia to AMD GPUs.

• Their compiling stack abstracts the intricacies, enabling developers to achieve high performance across different GPU setups without getting bogged down in deployment and configuration.

Other marketplaces may offer decentralized GPUs, but they often lack the sophisticated optimization Hyperbolic delivers, placing the burden of performance tuning on the user.

Hyperbolic streamlines this with an API that provides access to AI models optimized for various hardware, making global compute resources more accessible.

Hyperbolic released its limited alpha version of its GPU marketplace on August 15th, allowing 100 waitlisted members to trial GPU rental. You can sign up for their waitlist here.

The AI Services Layer

The next component of Hyperbolic’s AI ecosystem is the AI services layer, which offers capabilities like inference, model training, model evaluation, and Retrieval Augmented Generation (RAG).

Within the Hyperbolic app, you can easily run top open-source models like Llama 3.1 405B and Hermes 3 70B. To fine-tune outputs, you can adjust hyperparameters such as max tokens, temperature, and top P.

This setup, accessible via a straightforward API, simplifies the process for developers, making it as easy as using the OpenAI API but at a fraction of the cost.

Hyperbolic’s platform opens the door to innovative AI applications, including:

• Revenue Sharing for AI Agents: Tokenizing ownership of AI agents to redistribute revenue.

• AI-Powered DAOs: Leveraging AI for governance decisions.

• Fractionalized GPU Ownership: Enabling users to own and trade fractions of GPUs.

What Role does Crypto Play?

At the core of Hyperbolic’s infrastructure is its blockchain, underpinning the orchestration, services, and verification layers. The blockchain handles settlement and governance for Hyperbolic’s open-access AI cloud. It also powers the arbitration and verification mechanisms of the PoSP technology.

While details on the blockchain aspect are still sparse, you can expect Hyperbolic to reveal more about this soon.

🌈 Research-Level Alpha

Hyperbolic is still in the testnet phase. They raised $7m in a seed round led by Polychain Capital and Lightspeed Faction.

Interestingly, Hyperbolic is the sole provider of the Llama 3.1 405B base model.

Base models are the initial pre-trained versions of LLMs that haven’t undergone fine-tuning or reinforcement learning with human feedback (RLHF). It offers some advantages:

• A clean slate for fine-tuning on specific tasks

• A starting point for advanced AI techniques, such as synthetic data generation or model distillation

Beware: Base models are wild, though!

Users can sign up for the waitlist for GPU rental and supply here.

Hyperbolic also runs community missions with tasks that reward users with discord roles. You can join their discord here.

The Team

Dr. Jasper (Yue) Zhang is the CEO and co-founder of Hyperbolic Labs. He was previously a senior blockchain researcher at Ava Labs and a quantitative researcher at Citadel Securities. He completed his Ph.D. in math at UC Berkeley in two years and won gold medals at the Alibaba Global Math Competition and the Chinese Mathematical Olympiad.

Dr. Yuchen Jin is the CTO and co-founder of Hyperbolic Labs. He holds a Ph.D. in CS systems and networking from the University of Washington. He previously worked at OctoML, a company that delivers infrastructure to run, tune, and scale generative AI applications.

You can check out the rest of the team here.

Our Thoughts

• Overall, we’re pretty excited about Hyperbolic. They are definitely one of the top teams to watch in Crypto AI.

• Hyperbolic is more than just a compute provider. Innovations like PoSP and SpML add a new layer of trust and verification to decentralized AI.

• Experimenting with base models on Hyperbolic is quite fun, especially since they’re one of the few providers making this accessible today. We can definitely support their commitment to open-source AI.

• We wrote about Prime Intellect a few weeks ago. It remains to be seen whether Hyperbolic is as focused on distributed AI training as Prime Intellect.

• While we’ve noted that the demand side of compute has generally been sparse, that doesn’t seem to be the case for Hyperbolic. They’ve shown early traction with the research market, attracting significant interest from researchers and developers.

That’s it! If you have specific feedback or anything interesting you’d like to share, please just reply to this email. We read everything.

Cheers,

Teng Yan & Joshua

This newsletter is intended solely for educational purposes and does not constitute financial advice. It is not an endorsement to buy or sell assets or make financial decisions. Always conduct your own research and exercise caution when making investment choices.