Future Development Vision of Nostr

Introduction to Nostr#

Nostr is a very lightweight open protocol that has the potential (according to project documentation) to serve as a decentralized social media platform. The protocol specification is defined in NIPs (Nostr Improvement Proposals), which can be found here.

The foundation of the protocol is a WebSocket server (called nostr-relay) that handles and stores a very simple data structure known as Event. It looks like this:

{
"id": <32-bytes sha256 of the serialized event data>
"pubkey": <32-bytes hex-encoded public key of the event creator>,
"created_at": <unix timestamp in seconds>,
"kind": <integer>,
"tags": [
["e", <32-bytes hex of the id of another event>, <recommended relay URL>],
["p", <32-bytes hex of the key>, <recommended relay URL>],
... // other kinds of tags may be included later
]
"content": <arbitrary string>,
"sig": <64-bytes signature of the sha256 hash of the serialized event data, which is the same as the "id" field>,
}

Events are always signed (using Schnorr signatures) and they contain structured data that can have semantic meaning. The Schnorr type XOnlyPubkeys defined in BIP340 (currently used with Bitcoin Taproot) serves as "identity" throughout the protocol.

The nostr-client is a client that can communicate with nostr-relay and can use Subscription Filters. Events filters represent all nostr collections that the client is interested in.

Clients do not need to register or create accounts. Clients identify themselves with their public keys. Each time a client connects to a relay, it submits its subscription filters, and as long as the client is connected, the relay streams "interesting events" to the client.

Relays can cache client subscriptions, but it is not mandatory. Clients should handle everything on the "client" side, while relays can be as dumb as rocks.

Clients do not talk to each other. But relays can. This allows relays to fetch data for clients that they do not have. Clients can subscribe to events outside of the relays they are connected to.

At first glance, this gives the impression that Nostr is useless as a protocol (why not just sign and dump the raw JSON and let clients figure it out themselves?), but from a deeper perspective, the "dumb server, smart client" model reveals some significant engineering advantages, especially in decentralized protocol design.

This document outlines how these dumb servers, smart clients, and the Bitcoin network, along with end-to-end encryption, come together to address the problem of "decentralized social networks," or DSN (a buzzword I just thought of).

Problem Statement#

If you haven't been living under a rock for the past two years, you are aware of the current emergence and the strong demand in the market for a "Twitter alternative." A social media platform that does not compromise user motivations.

This demand has given rise to alternative social media platforms like Gab and Mastodon. A recent statement from the head of Ex Twitter has hinted that this will be the next big problem to solve. So, a disclaimer: I am not saying this is an easy problem to solve, nor do I think Nostr can solve all problems. But it at least "has a chance."

The core issue of creating a decentralized media platform is not technical, but social.

Creating social media (or chat applications) may be the most textbook challenge you face as a new software developer. The core structure of the system is quite simple.

A database to store things,
A network interface to communicate with clients,
Some filtering to retrieve query data as quickly as possible.

Of course, this is much more complicated in real life. But the key to this design is the same for all social media designs. So why can't we build it and get it done?

The problem is that it has to be a "decentralized" system, which can only succeed through "network effects" and an emerging engineering consensus among developers on a set of protocols. Otherwise, we will create the same problems we intend to solve.

This is where things get messy. If you built the perfect social media today, how would you convince other developers to build on top of it? If developers do not build features, why would users come? If users do not come, what is the point of the media platform?

The examples of Gab and Mastodon clearly show that simply open-sourcing the code is not enough. The standard building processes and designs must also be made public. Otherwise, one person will end up being a small group, with most voluntarily engaging in radical projects and ultimately becoming the "benevolent dictator" of the platform.

Because they must meet the real design constraints of such platforms while providing their products at scale, they ultimately create a small team that specializes in platform design. This makes it difficult for casual developers to create fun applications on the platform. At some point, they may decide to design their own little protocol, but ultimately, they will encounter the same barriers. No one is willing to voluntarily build on a platform designed for a specific niche market.

The cost of storing data is also high. "Server owners" require resources, maintenance, and time. All current owners of hosted Mastodon instances do so voluntarily, and users simply rely on their goodwill rather than shutting down instances. The long-standing issue of "knowledge sharing" arises.

So can we do better?

An Alternative Approach, the Dumb Nostr Way#

What if we don't build perfect social media, but just build the most basic Lego blocks needed to create such things and let developers naturally reach consensus on this basic standard unit of the puzzle?

That's what Nostr does.

To this end, it adopts the following approach.

It specifies the minimal unit of social data format (an Event) and lets developers naturally reach an agreement based on this. This defines the core of the protocol. Everyone needs to agree on the minimum backbone to be part of the network.

Nostr defines these protocol rules as NIPs. It mentions a set of mandatory NIPs. These are the rules that need to be implemented to discuss the Nostr protocol.

On top of these mandatory NIPs, anyone can define optional NIPs. Relays can freely choose the set of NIPs they support.

By defining more tag items in future NIPs, Events can expand data on the fly.

An Events can be viewed as a universal data store. The content that can be placed within it is unlimited.

Although it seems strange, such a simple protocol has garnered more developer attention compared to many "carefully designed" existing alternative social media platforms.

The project has sparked great interest among developers, and the community has quickly developed a rich ecosystem that includes libraries, applications, and relays. And this list is growing every day.

The Telegram chat room has about 400 members and is growing daily.

Why? "Because it's so simple."

This simplicity allows anyone interested to easily write JSON streams and start communicating with any existing relay using the protocol.

A live list of currently running Nostr relays can be found here.

Two ongoing Twitter-like applications, branle and NOSTR-twitter, already exist.

People are also regularly adding new additional details on top of the basic NIPs.

The simplicity of the protocol allows developers to quickly converge on open standards and place all complexity on the client side. The entire application experience will be handled by the client, while the relay will remain a dumb data server. This allows developers to move quickly and iterate on client applications while remaining compatible with any available relay.

This also increases client compatibility. There can be two different applications but still be able to see each other's posts. The core of the platform is decentralized, and clients are compatible with each other through a simple storage protocol. This is the cleverness of the "dumb server, smart client" model. Quickly reaching consensus on basic standards and iterating excellent client applications faster.

Complexity can be customized at the client layer, while interoperability is achieved at the relay layer.

Missing Pieces#

Once we understand what the core Lego blocks look like, the remaining pieces are DOS protection, relay incentives, and some methods for sharing nostr subscription data among users.

Putting Bitcoin into Nostr#

Thanks to Bitcoin, relay incentives and DOS protection can be addressed in one go.

If its infrastructure relies on a fragile "voluntarism" foundation, a robust social network cannot be built. As we know, "if the product is free, then you are the product." These future media platforms should be natively integrated with Bitcoin.

A one-stop solution to achieve this is to use BDK. A high-performance Bitcoin wallet library that is flexible enough to handle various Bitcoin interfaces and databases. Some new NIPs have been added to define payment request and payment response event types.

For each event they publish, payments can be a one-time on-chain transaction or a series of LN payments between the client and the relay. (BDK + LDK is actively being developed). Relays can set their rates in sats/byte, and if they want to "volunteer," they can choose to set it to 0.

This provides a good way for high-maintenance public relays to profit from their services while protecting them from DOS attacks.

Remember, Nostr relays are just dumps of simple JSON data. They are accessed via subscription filters. This makes nostr a universal data-sharing platform between clients. With Bitcoin, we are now talking about sharing Bitcoin scripts, descriptors, DLC contracts, and other Bitcoin DeFi information through the nostr relay network. But this information may be sensitive and should not be shared in plaintext on public platforms.

To address this, an encrypted nostr subscription sharing mechanism is needed. This could be another server that simply facilitates encrypted subscription data sharing among participants.

This can be achieved in the following way:

Encrypt [ subscription+ ] using a DH shared secret derived from the public key of the intended recipient. relay-address
Publish the encrypted data along with the recipient's public key to this server.
The recipient client is notified, downloads, and decrypts the data to obtain the subscription to fetch actual data from nostr.
The actual data is also encrypted ciphertext using the same shared secret, so the recipient knows how to decrypt it.

These servers can be very lightweight because they do not need to store all historical subscription data. They can periodically purge old data, and even clear it in real-time after knowing the recipient has downloaded it. This will keep their costs very low and eliminate the need to solve incentive issues.

These servers do not need to follow any common protocol. They can be implemented freely by any design. They just need a way to connect to clients and know when to notify them when something relevant happens.

They are also censorship-resistant like nostr relays. If one fails or stops working, anyone can start another. Since they do not have to retain history, switching from one server to another will not affect the overall information flow.

These servers also cannot exploit the data because all they see are random encrypted blobs, so they do not need to be highly secure.

The Final Picture#

So now, combining all of this—Nostr, Bitcoin, and encrypted subscription sharing—we have a very powerful and inherently private social network that can share data among participants using some very generic global protocols.

This allows the hidden social networks of people to selectively open their posts to specific trusted entities.

These posts could be DLC contracts, descriptors of multi-signatures among multiple parties, DLC oracles published only to subscribed members, and so on.

In this framework, the basic unit of "identity" is the public key. Public keys are akin to aliases in the real world. Anyone can have any number of aliases. If one alias is leaked, they can quickly create another, just like we create a new Bitcoin address for each payment.

Using public keys as aliases allows selective opening to your own private trusted network. You can associate one public key with your global alias (the well-known Twitter username) and then use any number of parallel aliases to communicate only among specific groups or use specific applications.

Data associated with all these public keys will remain completely unrelated and can be distributed across multiple nostr relays.

The final summarized model is:

A highly interoperable and extremely simple relay protocol, nostr.
A flexible framework for adding new relay features using optional upgrades that relays can choose to adopt.
An encrypted subscription sharing mechanism for passing nostr subscriptions.
Native integration with Bitcoin, facilitating both the "currency internet" and DOS protection.
A decentralized publishing layer for clients to publish public and private content.
Client-side complexity to interpret this content, with a local financial contract generation UI utilizing Bitcoin features.

Unlike everything in web3.0, this does not involve another "blockchain" (I know that's bad).

The Road Ahead#

Sounds good, but we are not there yet. And to realize these dreams, a lot of engineering design is needed. There are unknown problems to address ahead. The design decisions for these relays and clients need careful planning. Simply having a simple protocol is not enough.

These relays should be efficient, robust, rigorously peer-reviewed in public, and guaranteed to meet basic security levels. This work must be done openly, and the design of elements should be as flexible as possible to meet the needs of different client developers.

If this thing needs to scale to professional services that people can deploy on their servers and build significant products on top of, we need more than just hobby code and example applications.

What is needed is not another cool nostr application, but a thoughtfully designed infrastructure library that shadow super coders can use to build the next cool nostr application with "internal Bitcoin."

Introducing rust-nostr#

rust-nostr is a project in the conceptual phase aimed at addressing the above issues. The idea is to provide a complete, modular, easy-to-expand nostr infrastructure suite with strong security guarantees, traceability, and easy customization for developers based on their needs, and very easy to download, deploy, and manage on their servers.

The entire structure is still pending, but below is a rough outline of rust-nostr.

A binary crate producing nostrd. A lightweight and efficient Rust implementation of Nostr-Relay. nostrd will come with a set of supported NIPs. Basic NIPs can be included by default. Additional NIPs can be specified at build time via feature flags.
nostr-cli can serve as a server-side manager for nostrd. It can also communicate with any other relay using the nostr protocol and can serve as a CLI nostr client. Maintenance access can be provided to relays through basic or cookie authentication.
A rich nostr-API library. Included in the project, it can serve as a simple development tool for developers to build their nostr clients. These APIs can then be exposed to other languages via FFI, providing developers with a one-stop tool to build their cool Nostr clients.
A portal is an encrypted nostr subscription sharing server. The specification for the portal is not part of the project as it is already a solved problem. It is well understood in the cryptographic literature, and there are many candidate implementations in open source. The Signal App itself is an example of a portal, although it is difficult to use for this use case. A local team in India has been focusing on this issue to facilitate a specific use case of p2p Bitcoin transactions called CypherPost, which is already a very suitable portal implementation. Ultimately, a streamlined Rust candidate implementation will be added to the project repository. But people are free to develop and use their own portals while remaining compatible with the rest of the network.

All of this (except the portal) will be integrated with native Bitcoin + Lightning through BDK and LDK.

To ensure that all parts of the infrastructure are always synchronized with each other, they will undergo rigorous integration testing in the project's CI pipeline.

Once all of this is laid out, the rust-nostr suite can be used to propose various more complex clients that engage in Bitcoin DeFi with each other.

Conclusion#

So far, this is just a raw idea, and I don't even know what possible unknown challenges lie ahead. I expect many. As they say, "the devil is in the details." This seems like an ambitious project, but it is not.

By limiting the scope of the project to provide very specific building tools, it can almost be achieved with some enthusiastic Rust developers. Rust is also the most suitable language to build it because it allows us to strictly define protocol rules at the compiler level, reducing the room for error while generating very clean and easily auditable code.

By not trying to create a "product" but just solving Lego blocks, I believe this is achievable.

Then, the project can pave the way for Bitcoin entrepreneurs to propose various applications. The limits of the application space are only constrained by imagination.

So if you happen to care about this issue and want to lend a hand, or if you have any general suggestions or comments, please reach out to me on Twitter @rajarshimaitra. DMs are open.

Translation: Hoodrh

Original source