Multi-Chain SDK Vision

A quick brain-dump around what I would describe as an API for the "Polkadot Network".

This is mostly a summary of the PAPI discussions here and here, but I have posted the meat here as it might be relevant to all SDKs such as polkadot-js-api and paritytech/subxt, not just PAPI.

What is Polkadot Network

Going forward, the Polkadot network is not the Polkadot relay chain anymore, and it is not AssetHub. It a combination of system chains that might fluidly decide to scatter themselves further (example: Staking functionality moving to a different system parachain), or unify to achieve synchrony (example: Coretime chain merging into AH).

So, Polkadot Network is not the relay chain, and it is not a specific system chain. It is the combination thereof. In this view, a proper SDK is one that abstracts away over, not a pallet, not a single chain, but rather over the entire network, the red dashed circle below.

API For The Polkadot Network

With that definition in mind, let's consider:

The "Polkadot Network" (!= relay chain) as a whole always provides the following list of functionalities:

1. Query and transfer of the DOT token.
2. Staking the DOT token
3. Ability to vote on a referendum with the DOT token

The goal should be to provide an API that is abstract over "on which chain each of these functionalities reside", but rather the mere fact that "They exist somewhere in the Polkadot network".

dotApi = await new PolkadotNetwork(); 
{ transferrable, reserved } = dotApi.balance_of("kian");

// options for staking, such as who you want to supprot. 
opts = {}
result = dotApi.stake(transferrable / 2, opts);

Notice in the above, I do not specify a network, or a pallet. I merely ask the "Polkadot Network, please tell me how much DOT I have in your entirety.

Easier Said Than Done

The above basically suggests: Let's make life very very very hard for a few JS API teams, in return for all other builder teams having an easier time.

This is a worthwhile tradeoff, and is aligned with making Polkadot more accessible: As a novice Application developer, you are not thrown in the pitfalls of a multi-chain, asynchronous, XCM-riddled ecosystem.

Instead, you have the option to start with the "Polkadot Network API", and then graduate into using low level APIs that let you deal with chains and pallets from the get go.

That being said, one major technique that can be used while building this infrastructure to prevent it from becoming a maintenance snowball: Using RuntimeAPIs. That is, refraining from reading the direct pallet storage for as much as possible, and instead rely on more high level queries that the runtime exposes in the form of raw runtime-apis and/or view-function^[1].

This moves at least a part of the complexity of implementing the above into the runtime, which has a few advantages:

1. It is the responsibility of the polkadot-sdk maintainers to update them, not you. Of course, these APIs are meant to have a super stable interface, and only their implementation might change.
2. It remains backwards compatible and upgrade-able by nature: If the :code changes, the implementation might changes.