Living in a state of accord.

Introducing Pantheon

This week, the work I’ve been doing for the past 6 months, and that PegaSys has been working on for the past 18 months or so was released into the world. Specifically we’ve released Pantheon 0.8.1, our new MainNet compatible, Apache 2 licensed, Java-based Ethereum client. And it’s open source.

I’m pretty excited about it on a few fronts.  Firstly I think it’s a pretty important thing for the Ethereum community. To be a healthy ecosystem, Ethereum needs to have diversity in its clients to avoid a bug in one client taking out or accidentally hard forking the entire network. Currently though, Geth and Parity dominate the Ethereum client landscape.  Pantheon clearly won’t change that in the short term, but it is backed by significant engineering resources to help it keep up with the ever changing Ethereum landscape and be a dependable option.

I’m also really excited that Pantheon is released under the Apache 2.0 license. Both Parity and Geth along with most other clients are licensed under the GPL or LGPL. There are still a large number of enterprises that completely avoid the GPL and LGPL which has closed off Ethereum to them. Having Pantheon available under a permissive license and in a highly familiar language like Java will make it much easier for many enterprises to start using, building on and innovating with Ethereum.

Pantheon will also be building out functionality from the Enterprise Ethereum standard for things like privacy and permissioning to make private chains more powerful and flexible. Meanwhile we have a significant number of researches continuing to work on developing new ways to get the most out of Ethereum.

Finally I’m quite excited to be able to contribute to an open source project as my full time job. I’ve had the opportunity to do some open source for work in the past but only on a fairly small scale. It’s a bit daunting to have everything in the open and on the record, but I’m really looking forward to engaging with the community and being able to show exactly what I’ve been doing.

Debugging Ethereum Reference Tests

There’s an exceptionally valuable set of ethereum reference tests that are run by  most or all of the different Ethereum clients to ensure they actually implement the specifications in a compatible way.  They’re one of the most valuable resources for anyone developing an Ethereum client.

The Aleth project maintains the official test client called testeth but it’s a little cryptic to work out how to actually run things with it and then use that to debug failures happening in the client you’re actually developing.  So this is what I’ve found useful:

Firstly, you need to checkout Aleth and compile it following their instructions. The one catch here is that you need to enable EVM tracing by adding the -D VMTRACE=1 flag when doing the initial cmake .. Once the whole build is finished you should find testeth in aleth/build/test/testeth Copy that to somewhere on your PATH.

Then you need to check out the actual reference tests and set ETHEREUM_TEST_PATH to point to where you checked it out.

Now you should be able to start running tests.  Running a single test with loads of debug looks like:

./testeth -t GeneralStateTests/stCreate2 -- --singletest create2collisionStorage --verbosity 3 --vmtrace --statediff | less -R

The param after -t needs to point to a directory containing the group of tests you want to run.  It could be as high level as GenerateStateTests or BlockchainTests or point to one of the directories under that.  The –singletest option lets you run just one file underneath that directory. The name is the filename without the .json extension.

–verbosity 3 –vmtrace –statediff turns on pretty comprehensive debugging information so you get a full EVM trace, plus a record of the changes to state that were made (it appears to show each change as well as the final state though I’m not sure I’ve fully understood the format yet).

And then finally piping to less -R allows you to scroll up and down and search through the output easily.  The -R tells less to render terminal control characters so you still get the pretty colours.

Now you have a reference client running the test and providing a ton of information about what the test expects to happen, go run the test in the client your developing, compare outputs, beat your head against a wall for a while and eventually find and fix your bug.

And a massive thanks to everyone who’s been involved in building up these reference tests. If you ever have to pay for your own beer at DevCon the community is doing something very wrong.

Obscuring Presence of Browser Plugins with window.postMessage

There are a number of browser plugins which inject additional JavaScript APIs into the DOM so websites can take advantage of the plugin functionality.  One example of that is MetaMask which “brings Ethereum to your browser”. This allows any website the user visits to detect that the plugin is installed by checking for the presence of those APIs which may aid them in targeting attacks such as the recent spate of phishing attacks against MetaMask users. So there’s a proposal in place to require websites to get specific authorisation from the user before the APIs will be injected.  And since injecting an API to allow the website to request access would defeat the point, it uses window.postMessage:
Dapps MUST request the web3 API by sending a message using window.postMessage API. This message MUST be sent with a payload object containing a type property with a value of “WEB3_API_REQUEST” and an optional id property corresponding to an identifier of a specific wallet provider, such as “METAMASK”.
If the plugin is installed it will prompt the user for access and if granted inject the APIs into the DOM.  If the plugin isn’t installed or if the user refuses access, the website simply receives no response. Clever.

Bitcoin Redux: crypto crime, and how to tackle it | Light Blue Touchpaper

Interesting review of the regulatory landscape around crypto-currencies. There are a lot of echo’s of issues with the over-the-counter nature of most FX trading, albeit with even less enforced regulation and uncertainty.
Bitcoin Redux explains what’s going wrong in the world of cryptocurrencies. The bitcoin exchanges are developing into a shadow banking system, which do not give their customers actual bitcoin but rather display a “balance” and allow them to transact with others. However if Alice sends Bob a bitcoin, and they’re both customers of the same exchange, it just adjusts their balances rather than doing anything on the blockchain. This is an e-money service, according to European law, but is the law enforced? Not where it matters. We’ve been looking at the details.
Source: Bitcoin Redux: crypto crime, and how to tackle it | Light Blue Touchpaper Also interesting to note is that most of the regulation required is already in place and just needs to be enforced. In most cases there isn’t any need for radical rethinking of laws, just apply the current laws about treating consumers fairly and Know-Your-Customer to this new technology.

The Great Bug Hunt – Allen Pike

A fun thing about programming is that most days, you make progress. Maybe you fix some issues, maybe you add a feature, maybe you build towards something bigger. Your code moves ever forward. Until it doesn’t. On occasion, you will hit a Bug. Not a mundane bug, some trifle you can fix in an hour, or even a day. This is a true Bug. One that defies reason. One that evokes a “that’s not possible,” a “how could this even happen?”, or most dreadfully, a “could there be a bug in the compiler?” Hold on kids, we’re going hunting.

Source: The Great Bug Hunt – Allen Pike

Quite an impressive and entertaining bug hunt story really. And the parting words are oh so true:

Whether the Bug is in your code, a 3rd party library, or the thermal expansion of prototype hardware in the morning sun, the only solution is science. And maybe a little whisky.