Layer 1 Sharding - Pros vs Cons

[No_One]

It has been said that Layer 1 sharding is not a good idea due to limited reliability and security for scalability. I’m no sharding expert, but I have to disagree based on what I thought I knew about sharding. If anyone feels this is true, please educate me on why this is.

I’ll break this down piece by piece based on what I’ve seen from some of the projects implementing layer sharding.

• I’ve tested SEELE the most extensively of any layer 1 sharding solution. They currently have 4 shards. I am impressed with the reliability within shards and between different shards from a transaction perspective. All of my transactions arrive, regardless of the shard I send from/receive on. When I increase the fee, it arrives faster.

• If we’re talking scaling from a smart contract perspective, this is where SEELE has a weakness; currently smart contracts between different shards isn’t possible on their network. So limited scalability in this perspective would be accurate. However, other sharding platforms such as MultiVac will have smart contract functionality between shards. Once they release their testnet 3.0 I will be able to do a legitamit test, but for now I will take their word.

• Security doesn’t seem to be limited or impacted at all on SEELE due to it being a layer 1 sharding platform. In fact, it seems security would be better since someone would have to take over multiple shards to control the network, which would seem more difficult than simply 51% attacking an individual shardless network.

Just curious on everyone’s thoughts on this. Maybe I’m missing something here.

[larrypang]

Moved to the Technology & Developers category!

Thanks for starting this great discussion topic. From my point of view, which is centered more on usability/practicality than theory, sharding is interesting but not as promising in the near-term as other approaches to scalability. There are two major camps in the scalability space – on-chain (e.g., sharding) and off-chain (e.g., state channels, sidechains). Take a look at this article on off-chain: https://hackernoon.com/difference-between-sidechains-and-state-channels-2f5dfbd10707

Things that we value in IoTeX Network, like instant finality, are difficult to achieve with sharding or other on-chain scaling techniques. But like you said, there is lots of great progress being made in both on-chain and off-chain scaling. In the long-term, I believe both on-chain/off-chain scaling can work together, but for now the more practical and reliable approach is off-chain scaling.

@raullen @qevan - please share your thoughts as well

[JoeGoodman]

Blochain current succes is based on balanced systems that are the slowest but securest and more decentralized ones. So their main use cases are value storing, transfrontier payments and trading speculation.

Future promises are based on a decentralized and secure world computer that build a new internet of value that will change human interactions. But for that we need scalability and social acceptance.

Iotex propones a strong solution for this two concepts: a governed multichained solution that grows “block by block” (chain by chain). It’s a balanced solution: enought scalability, with enought security for a world of governed chains. The adventage of this approach is that is “inclusive”: It invites people and groups to participate, as new chains. This concept needs to be realized, but the potential is there.

For me the technology debate have just starting, but I bet that the next wining horses will be balanced solutions that show actual growth on actual use cases, despite their approach to the scalability issue be one or another.

Note: Notice that I’m not using “performance”, or “sharding” but “scalability”: performance and sharding are tech features. What humans need is scalability: the right present and future performance for a given use case.

Note2: Iotex has the advantage that has also the privacy and “things oriented” features that also will be necessary

[raullen]

This boils down to how you see blockchain - do you see it more as a distributed system/infra or more as a one-of-a-kind global state machine. Of course, the real view is the hybrid of both.

Many ppl see blockchain more as the distributed system/infra and apply theories/technologies that are proved to be useful/valuable in the domain of distributed systems to blockchain. Sharding is a great example, which originated from database. Almost all popular databases (cassandra, gfs) that are currently in use are equipped with sharding as a feature which is great. It is a typical application of the idea of “divide and conquer” in CS. It would make a blockchain faster and has a higher throughput, perhaps a better user experience. The downside is complexity in terms of implementation. BTW, due to the decentralized nature of blockchain, FLP and CAP theories do apply, i.e., which is called impossible triangle by vitalk.

Some ppl see blockchain more as a one-of-a-kind global state machine that keeps perhaps most valuable information about this world about this generation of the human being. In that sense, tamper-proof (in general, security), anti-censorship (in generally, decentralization) demostratet the real value proposition of a public blockchain - something, such as database, distributed file systems, cannot provide tho they are fast, high performant and easy to use.

Since we are building a real public blockchain (i.e., IoTeX is not a pure distributed infra, at least the layer 1), we value security and decentralization much more than scalability, which can be addressed. Scalability would be useful only if the layer 1 is reliably and continuously providing security and decentralization, and scalability can be achieved in many ways IMO

• on chain - basically hierarchical structure, to have multi-chains running on the side with the layer 1
• off chain - ZKP with or without rollup, state channel, plasma (which only works for UTXO)

On the tech side, sharding itself is challenging:

• Adds complexity in the system: properly implementing a sharded architecture is a complex task. If not done correctly, there is a significant risk that the sharding process can lead to corrupted global states.
• Rebalancing data is non trivial: when a shard outgrows other shards and becomes unbalanced (a.k.a hotspot). In this case any benefits of sharding is canceled out. The ledger needs a re-sharding to allow for a more even data distribution. Moving data from one shard to another shard requires lot of complexity at protocol level (which hurts security potentially) and downtime (which hurts decentralization potentially).
• State sharding is largely an unsolved problem even by Ethereum 2.0, it is at the research stage and not practical right now.

In general, my view is “sharding is not the proper tech for layer 1”.

[No_One]

@raullen you definitely make some great points; particularly, highlighting the challenges of sharding. Side-chains seem best path for layer 1 & on-chain.