Over on Twitter, there was an interesting discussion about the importance of client diversity for the beacon chain. As part of that cyber_hokie asks:

cyber_hokie – @cyber_hokie: As the majority client, isn’t it more likely that issues with Prysm would cause more severe financial penalties for minority clients voting on an alternative chain under accountable safety given Prysm validators likely hold the 2/3 threshold?

It’s important to note up front that this isn’t a Prysm specific issue. This isn’t a criticism of Prysm at all. All the major beacon chain clients are great – the issue here is any one client getting too large a share of validators. How much is too large? In an ideal world every client would have less than 1/3rd of validators. Then a bug in a single client doesn’t prevent the chain from continuing to finalise. More than 2/3rds majority is a really big problem because of the potential for those clients to finalise on an incorrect fork.

One of the most commonly overlooked steps when setting up a beacon node is to setup port forwarding on your router to ensure that other nodes can connect in to yours. This is often overlooked, or not done correctly, because there are so many different routers, all with different, fiddly interfaces that providing simple instructions is essentially impossible and because your node will generally work even without it by connecting outbound to other peers even though none can connect inbound.

I got the chance to talk with Pooja Ranjan and Alita Moore from the Ethereum Cat Herders as part of their PEEPanEIP series last week: PEEPanEIP #38: Altair in Teku with Adrian Sutton. It was a chance to talk through the actual implementation details required to support the Altair fork.

Mostly listing here for my own record, but if you haven’t seen it and you care at all about Ethereum hopefully it will be interesting.

Note: This post is deliberately high level. It doesn’t attempt to be an economic proof or go into the gory details of exactly how the difficulty adjustment works. The aim is just to see the high level effects of how the system pans out without getting lost in the nitty gritty details of what is ultimately a very complex system.

Pretty much every time the word “miner” is mentioned in an Ethereum discussion someone will claim that miners are paid too much and miners will respond saying they’re struggling to survive. Turns out, both can be simultaneously true and in fact it’s pretty much the expected case.

If you’re getting into staking ETH on the beacon chain, it’s important to know what you’re actually signing up for. There’s lots of articles about how easy it is and how awesome, intelligent and good looking stakers are, so here’s a few hard truths you should make sure you’re ok with…

Short Term Stuff

Most people know this stuff so I’m just going to skim over it.

No Withdrawals Yet

You can’t withdraw staked ETH or rewards yet. There isn’t a fixed timeline on when you will be able to.

Through the beacon chain testnets people have been gradually developing better ways to evaluate how well their validators are working. Jim McDonald made a big leap forward defining attestation effectiveness which focuses on the inclusion distance for attestations. That revealed a bunch of opportunities for clients to improve the attestation inclusion distance and attestation effectiveness scores improved quite dramatically until now scores are generally at or close to 100% pretty consistently.

But then beaconcha.in added a view of how much was earned for each attestation. Suddenly people were left wondering why some of their “rewards” were negative even though the attestation was included quickly. It became obvious that inclusion distance was only one part of attestation rewards.

For the beacon chain MainNet, I’ve setup an instance of chaind – a handle little utility that connects to a beacon node (like Teku) and dumps out the chain data to a database so its easy to run various queries against. While you can do this with various REST API queries and custom code, having it all accessible by SQL makes ad-hoc queries a lot easier and lets you add it as a datasource for Grafana to display lots of powerful metrics.

Teku requires a password file for each individual keystore, with the same relative path as the keystore but a .txtextension. If all the keystores have the same password, you can copy a single password.txtfile for all the keystores with the command below:

for keyFile in keys/*; do cp password.txt "secrets/$(echo "$keyFile" | sed -e 's/keys\/\(.*\)\.json/\1/').txt"; done

Note that the keys directory is specified twice in that command so if the keys are in a different directory name, make sure to update both places.

Last week I put together a video explaining attestation inclusion and delays in ETH2. This is one of the keys to maximising your validator rewards and the health of the network.

We run a bunch of hosts on EC2 and while most just have the default DNS name, they all have a Name tag to identify their purpose. While there’s lots of automation setup via ansible, it is often still necessary to SSH directly into a particular box, especially while debugging issues.

I find it annoying to have to go log into the AWS console just to find the DNS name of the particular box I want so I’ve written a little script that searches for hosts based on the name tag and can then SSH into them.

It’s like something out of a Sherlock novel – the doors are all locked, windows barred and yet somehow the jewels have been stolen. Or in this case, every block created forms a single chain and yet somehow the validator client has wound up on the wrong fork. Fortunately Sherlock, or rather Cem Ozer, was on the case. This is the story behind those very annoying “Produced invalid attestation” errors that Teku would sometimes generate.

Stumbled across ArchUnit today which looks useful. In particular I think there’s power in being able to assert that certain packages really never depend on each other. Although gradle/maven modules would probably be a better level to assert at. It’s depressingly common for code bases to be split into separate modules with the intention that they be a clear separation of concerns only for a web of dependencies to be added because someone wanted to reuse some class and didn’t refactor to a common module.