Constellation

Eng Update: Kryo, Algebird, metrics, and gossip


#1

Some brief release note style updates for upcoming next version of Constellation:

Created adaptors for Java crypto classes to make them compatible with Twitter’s Chill Scala Kryo serialization library in preparation for switching off Java serialization. We were temporarily using Java serializers for peer to peer communications (which is standard in Akka and works easily with our crypto libs) but that carries with it security risks. Kryo is much safer and faster but requires some custom modifications for public key serialization (for example).

Started preparing simple Algebird integrations (Sketch Map, HyperLogLog, MinHasher) for demonstrating approximate frequency estimations and set operations for transaction and id hashes. More generic refactoring / cleaning up of code. Added much higher transaction counts to the tests to start getting performance measurements and validation of internal buffering mechanisms. Tuning of Akka threading parameters. Added more API endpoints for simplifying some of the tests and pulling some simple metrics / data from the node.

Started dumping some metrics to a prototype of a node dashboard page (served from the same API with a simple Scalatags / Scala.js example). Some more work on refactoring peer authentication protocols. Added some reputation configuration endpoints and storage structures. Created the basic schema for our Fiber Bundle classes and some demonstrations of recursion traversals to extract transactions / ids / arbitrary data from bundles. Added some basic structure for handling different types of network conflict cases for preparations with integrations between gossip protocols and the new consensus updates. More schema updates oriented around votes, conflict detections, and proofs. Handled several complex Kryo serialization test cases to verify correctness.


#2

@ryle awesome update is this update to support Proof of Meme (PoM):grinning:?


#3

Great progress mate, excited for next weeks updates


#4

Yes, but it’s only a part of the reputation calculation. This was just to allow nodes to assign labels (trust scores) to other nodes (distinct from our deterministic reputation calculated from the gossip DAG, but still used during conflict resolution.)


#5

This was just to allow nodes to assign labels (trust scores) to other nodes (distinct from our deterministic reputation calculated from the gossip DAG, but still used during conflict resolution.)

@Eng excited to hear more updates like this!