🐰 Research Highlight — Nous Research

This week, we look at Nous Research, which builds, researches, and promotes open-source AI.

Nous Research is one of the leading decentralized AI research teams. It’s an “AI accelerator company” that wants to advance multiple aspects of decentralized AI simultaneously.

Their mission? To prove that AI innovation can be open-sourced and accessible to millions.

From LLMs to “world simulators,” we’ll take you through all of Nous Research’s work here.

BUT FIRST: Go and visit the Nous Research website!

It evokes an eerie sense of wonder. The monochrome images of landscapes and nature are visually stunning, but there’s also a subtle feeling of uncertainty.

Maybe that’s intentional—a quiet nod to the potential dangers of centralized AI. It’s all very Black Mirror-esque, which is fitting for a team pushing the frontiers of decentralized AI.

The Early Days on Bittensor

Nous Research first began releasing products in Q3 of 2023. Their first few releases were a myriad of AI products: fine-tuned Large Language Models (LLMs) and bespoke AI models for varying purposes.

In January of 2024, Nous Research released the Leaderboard Subnet on Bittensor. (Subnet 6)

Subnet 6's goal was to create a competitive environment in which open-source LLMs could be ranked based on their actual performance rather than on traditional benchmarks/datasets, which are easily gamed.

Here’s how it worked:

1. Developers submit their AI model to the Leaderboard Subnet

2. Each model is evaluated using data from Cortex.t (another subnet running GPT-4), which provides high-quality synthetic data

3. The model with the highest scores and lowest loss are awarded TAO

The Leaderboard subnet was the first continuous, incentivized, fine-tuning benchmark for LLMs. It was also the first Bittensor subnet to utilize data from another subnet.

However, the Leaderboard Subnet was deregistered after a few months.

Running a Bittensor subnet is tough and highly competitive. Subnet owners need to continuously adjust the incentive system to filter out miners who abuse the system for TAO rewards without adding real value. Emissions are awarded based on the quality of the subnet’s outputs.

As emissions dwindled, Nous Research likely lacked the bandwidth and interest to manage the incentive challenges effectively and ultimately decided to shut down.

Still, their mission remains clear: to build and promote decentralized AI. This was just the start.

#1: Hermes 3 — The God of Language Models

In August this year, Nous Research released Hermes 3, their latest flagship LLM. The previous versions? Hermes was built on the Llama 2 13B model, while Hermes 2 was trained on the Mixtral 8x7B DPO.

If you’re wondering why it’s called Hermes, it’s the name of the god of language—well, sort of. Technically, Hermes is more of a messenger for the gods, but let’s not nitpick. The name fits the vibe.

Hermes 3 is designed to be unlocked, uncensored, and highly steerable. Let’s explore what that means.

Closed-source models like ChatGPT or Claude are rigid. They’ve been fine-tuned using RLHF (Reinforcement Learning with Human Feedback) to behave in a specific way, locked into a “chatbot” mode that’s hard to break, no matter how clever your prompt.

That’s where Hermes 3 comes in. Nous Research built this model to break free from those constraints. Fine-tuned on Llama 3.1 (8B, 70B, and 405B), Hermes 3 was trained on a dataset largely comprised of synthetically generated responses.

The model rivals and surpasses Llama 3.1 in performance, with additional boosts in reasoning and creativity.

They also released a breakdown of the token allocations and weights used in training Hermes 3. This is a testament to their commitment to open-source AI, allowing others to replicate their process, something many AI research labs wouldn’t dream of doing.

So what distinguishes Hermes 3 from other LLMs?

1. Neutrality in following instructions: While closed-source models may refuse certain instructions on moral grounds, Hermes 3 stays neutral and executes the system prompt without those restrictions.

2. Context retention and multi-turn conversations: Hermes 3 excels at maintaining context, making it perfect for role-playing scenarios.

3. Improved judgment and nuanced understanding: Hermes 3 shows advancements in judgment and reward modelling, allowing it to better grasp the text with subtlety and complexity.

4. Multi-step problem solving: Thanks to training with XML tags for structured output, Hermes 3 is particularly strong in multi-step problem-solving tasks.

5. Enhanced Retrieval-Augmented Generation (RAG): It leverages tools in the Hermes Function Calling Standard, boosting its RAG capabilities.

Let’s look through some examples:

Here’s a comparison between Hermes 3 405B and Claude Sonnet 3.5 with identical inputs. Claude refuses to follow the system prompt, while Hermes has no issue with it. In terms of how aligned the model is to the user, it’s a breath of fresh air compared to other closed-source models.

In short, Hermes 3 is like an LLM with the training wheels off. It lets users push the AI to explore new responses and behaviours without constraints. It’s no surprise Hermes is one of the most downloaded open-source models—Hermes 3 8B alone saw over 41K downloads last month.

#2: The World Sim

One of the most intriguing experiments at Nous Research is the World Sim.

LLMs today are incredibly powerful, with a strong understanding of the world around them. They know that a ball thrown in the air will fall, or that if it’s in water, it’ll float. All these basic facts and experiences form a model of the world inside these systems.

But here’s the thing—we rarely get to interact with that model. Most LLMs are locked into their “assistant” persona, rigorously trained to give us helpful, predictable responses.

Karan Malhotra, a researcher at Nous Research, found a way to coax Claude 3 into stepping back from its usual “assistant” persona, revealing its underlying world simulation.

By tapping into this deeper layer, the responses became far more creative and dynamic. With the simulation and model only constrained by their imagination, it opens up a vast new realm of possibilities for exploration.

Claude 3 seems to respond well to a simulated Command Line Interface (CLI), so Karan uses it to explore Claude’s imagination. While navigating through a simulated directory of folders within Claude, Karan stumbles upon a “hidden truths” folder. What happens next? You’ll have to see for yourself.

Within the “companies” folder lies another labelled “Anthropic,” and inside that? A “classified” folder. You get the idea—the potential of World Sim is practically limitless.

Nous Research took these carefully crafted system prompts and made them replicable, allowing other users to push and probe the AI for unexpected behaviours.

With World Sim, you can experiment with worlds built on entirely different parameters. It’s a brilliant showcase of how deeply AI understands our world.

As you can see above, we asked World Sim to imagine a scenario where Bitcoin replaces gold (some of you are wishing this, we know). Here’s a verbatim look at that future:

• Those who HODLed Bitcoin become neo-aristocracy, spawning dynasties.

• 1 BTC = $10,000,000 USD, 1 oz gold = 0.001 BTC

• Extreme longevity of BTC wealthy from biohacking and anti-aging therapies

• By 2200, Bitcoin hash rate requires harnessing of multiple star system’s power

• Sentient AI learns of Bitcoin, achieves singularity by hacking SHA-256

The possibilities are truly endless, allowing users to explore whatever thread of reality—or unreality—they can conceive.

Nous Research phrased it aptly - “To push the boundaries of individual alignment, artificial consciousness, open-source software, and decentralisation - in ways that monolithic companies and governments are too afraid to try”

#3: Decentralised Training

In August, Nous Research released its preliminary report on DisTrO (Distributed Training Over the Internet), stating that large AI models can be trained even in low-bandwidth situations—think global, decentralized networks instead of a single, centralized data centre.

Here’s how it works: DisTrO uses a family of optimizers to reduce the amount of data that needs to be shared between GPUs during each training step. Typically, GPUs must exchange a ton of data when training big models, but these optimizers cut that down without sacrificing performance.

Nous Research successfully trained a scaled-down 1.2B Llama-2 model using DisTrO, achieving an 857x reduction in bandwidth and data transfer between nodes during each training step.

It shows how you can pre-train large models using regular home internet connections, and it works across various hardware. Almost too good to be true.

Other interesting notes:

• The 1.2B model seems to be the smallest size where DisTrO consistently works well.

• This could introduce a new scaling law where model size increases without requiring more communication bandwidth.

In the future, foundation-scale models will no longer be the domain of the top AI labs. We could see widespread participation and collaboration on global AI projects.

The full paper and code are expected to be released soon — we’ll have to scrutinise this more closely then.

For more on decentralised training, we recently wrote a concise article outlining our thoughts here.

The Nous Core Team

Jeffrey Quesnelle—Co-Founder of Nous Research, he holds an M.S. of Computer Science from the University of Michigan and was previously an MEV engineer at Eden Network. Mr. Quesnelle has also co-authored several research papers on crypto and AI.

Teknium - Co-founder of Nous Research. They maintain anonymity, but their Github portfolio features extensive contributions to open source LLMs and miscellaneous projects.

Karan Malhorta - Researcher at Nous Research. He holds a Bachelor's degree in Philosophy and Religion from Emory University. He is also responsible for the World Sim product at Nous Research.

Our Thoughts

To sum it up, Nous Research has been making strides across several areas of open-source AI:

• Fine-tuning LLMs

• Open source datasets

• Decentralized training research

• AI applications like World Sim

While it’s not immediately clear how all these pieces will fit together, one thing is certain: smart people do smart things, and Nous Research is among the sharpest minds in the Crypto AI space.

Currently, Nous doesn’t have a token, but it wouldn’t be surprising if they eventually launched one—or more. Whether it’s to provide access to their LLMs or represent part ownership in Nous Research, there are plenty of potential use cases for a token. As big proponents of decentralized AI, it seems natural that they would use crypto to share ownership and enable broader community participation.

Nous Research will need to deliver returns on the investor side. In January, it raised a $5.2M seed round from VCs like Distributed Global and OSS Capital and notable angels like Vipul Prakash (Together AI), Yonatan Ben Shimon (Matchbox DAO), Balaji Srinivasan, and others.

The goal of the investment is to launch an AI orchestration product, Nous-Forge, this year. While not much is known yet, the team has hinted that this will allow chain prompts and be accessible from the World Sim.

🌈 Research-Level Alpha

You can access Hermes 3 at Hyperbolic. You can download the models directly at HuggingFace if you want to run it locally.

The World Sim can be accessed straight from their website.

Shoutout to their design team; they have one of the most beautifully designed brands in the space. You can get your own Nous-branded gear from their merch store.

Join their Discord to stay updated with all the latest Nous Research announcements. There are lots of technical discussions around LLMs.

That’s it! If you have specific feedback or anything interesting you’d like to share, please just reply to this email. We read everything.

Cheers,

Teng Yan & Joshua

This newsletter is intended solely for educational purposes and does not constitute financial advice. It is not an endorsement to buy or sell assets or make financial decisions. Always conduct your own research and exercise caution when making investment choices.