Chasing the DAO Attacker’s Wake

This post is a follow-up to my previous post on HackingDistributed covering the mechanics and timeline of the DAO exploit, and contains a few things that cropped up during investigation that were not relevant, timely, or concrete enough to make it in the original post.

A second exploit

In the original writeup, we covered the recursive send exploit on the DAO in withdrawRewardFor, which is where much of my focus has lied (as well as the researchers and media have been focused on for the last few days).

In my previous post, I described an amplification attack whereby the attacker managed to infinitely repeat his 30X amplification attack (see Step 4A).

Thanks goes to Joey Krug and Martin Köppelmann for working tirelessly to uncover the full impact of what I initially dismissed as an implementation detail.  As Martin points out in his brief summary of the attack, there were actually two independent exploits here that build a third potent exploit:

  1. A recursive attack on splitDAO from withdrawRewardFor, allowing you to withdraw ~30X as many DAO tokens as you should have access to, but only once.
  2. A reentrant but non-recursive attack on splitDAO from withdrawRewardFor through transfer, allowing you to double your DAO tokens with every split.
  3. A combination of (1) and (2), allowing you to withdraw ~30X as many DAO tokens as you should have access to, as many times as you want.

The attacker employed (3), as we’ve already discussed.  But let’s take a look at (2).  To do so, we can create a contract where (1) and (3) no longer exists as vulnerabilities.  Let’s take a look at a contract where withdrawRewardFor was written perfectly, with no recursive reentrancy vulnerabilities.  Recall from the previous post that the reason withdrawRewardFor was vulnerable to causing reentrancy in splitDAO in DAO 1.1, even after it was supposedly fixed, was a problem with the first line of the function:

function withdrawRewardFor(address _account) noEther internal returns (bool _success) {
  if ((balanceOf(_account) * rewardAccount.accumulatedInput()) / totalSupply < paidOut[_account])
    throw;

  uint reward =
    (balanceOf(_account) * rewardAccount.accumulatedInput()) / totalSupply - paidOut[_account];

  reward = rewardAccount.balance < reward ? rewardAccount.balance : reward;

  paidOut[_account] += reward;
  if (!rewardAccount.payOut(_account, reward))
    throw;

  return true;
}

Remember that even if there is no money in the rewards account, and the amount of money to be paid out to the user is 0, the payOut call still runs and still calls arbitrary code in the recipient’s contract:

function payOut(address _recipient, uint _amount) returns (bool) {
  if (msg.sender != owner || msg.value > 0 || (payOwnerOnly && _recipient != owner))
      throw;
  if (_recipient.call.value(_amount)()) { // vulnerable
      ...
}

the first time getRewardFor runs, it will pay out the legitimate reward. The second time it runs it will pay out 0.  The third time it will pay out 0.  Etc.  But it will still pay out, and it will still allow the user to call back into splitDAO 30X more times than they should be able to.

Note that even the fixed withdrawRewardFor in DAO 1.1 is vulnerable to reentrancy.  This reentrancy won’t threaten the balance of the rewards account, since the second (and third and fourth) time you run the function it will pay out 0, but it is vulnerable to reentrancy nonetheless.

But what if we fix the function in DAO 1.2, and we write a perfect withdrawRewardFor function, that is not vulnerable to reentrancy at all?  That function would look like this:

function withdrawRewardFor(address _account) noEther internal returns (bool _success) {
  if ((balanceOf(_account) * rewardAccount.accumulatedInput()) / totalSupply <= paidOut[_account])
    return false; // Stop all splitDAO calls from failing, orthogonal change

The only change is bolded and underlined, a simple < to <= change.  Why?  The second time this function runs, paidOut[_account] will be equal to the left side of the expression.  It gets explicitly set to that in the code right before the reentrancy is triggered:

  uint reward =
    (balanceOf(_account) * rewardAccount.accumulatedInput()) / totalSupply - paidOut[_account];
  paidOut[_account] += reward;

(paidOut account will cancel out, being subtracted from the first line and added in the second).

So this is what DAO 1.1’s withdrawFor should have looked like.  It is no longer vulnerable to infinite reentrancy.  It follows exactly the best practices for sending value that were outlined when the original antipattern was made public.  The code is simple and easily reviewed.

On top of this, it is no longer vulnerable to the infinite amplification of splitDAO I described in Section 4A of the old post, which the attacker ended up using: because the payOut call never runs when the amount to be paid out is 0, repeated runs of this function are useless to an attacker.

But, as Joey Krug pointed out in the comments of my old blog post, it doesn’t matter.  We can still drain a DAO 1.2 with a perfect withdrawRewardFunction using the same exploit we described in 4A.  The key is that withdrawRewardFor calls arbitrary code once in this new case.  And once is enough for an exploit.

Simply apply the (4A) amplification exploit by itself: call the DAO’s transfer function while your reward is getting withdrawn, and your balance will be zeroed after tokens were transferred to your child DAO, with you getting to keep the tokens in a new account.  The code walkthrough is in (4A) of the previous post, and involves splitDAO and the transfer function both modifying the same balance array non-atomically (in this case, the array is modified mid-split).

So any state influencing the logic of any procedure non-atomically is vulnerable to attack if interrupted by a call to an arbitrary contract.  This goes far beyond the “write functions that are reentrant” suggestion: instead write functions that either don’t call out to arbitrary contracts or make no assumptions about their control flow or state after doing so.

This exploit has some interesting characteristics.  Firstly, it’s very slow (duh).  You essentially double your tokens with each invocation.  But this doesn’t really matter: it’s fast enough to be effective, and fast enough to mean that nobody can stop you.

The second is that, in our fixed version of the DAO 1.1 code that’s really not vulnerable to reentrancy, this exploit requires the reward account to have a balance.  If the rewardAccount.accumulatedInput() (total sum of Ether sent to rewards account) returns 0, and the user has never been paid out, our modified function will always return false and never execute the potentially malicious code.   In the published version of the DAO 1.1 code with the still-vulnerable withdrawRewardFor, the reward account is not required to have a balance.

In practice, this doesn’t matter.  The reward account for the DAO was funded; perhaps by accident, perhaps on purpose.  That means that the attacker actually was withdrawing a share of his reward each time, and would have been able to exploit even a reentrant withdrawRewardFor function in a totally fixed DAO 1.2!

A fundamental flaw in Solidity

There’s absolutely no doubt in my mind that virtually everyone in the community would have missed exploit (2) in my exploit list if exploit (1) did not exist.  While exploit (1) was an instance of a known antipattern that we’ve beaten to death at this point, exploit (2) was a much more subtle instance of this antipattern: no reentrancy into the original function was called for, merely reentrancy into the original contract.

The conclusions of this are as follows:

  1. If you use the call construct in Solidity on arbitrary external contracts, and if you have any externally-callable functions in your own contract that modify state, you cannot assume anything about the state of your contract after the external call is executed.
  2. The above was not a known programming practice at the time of the DAO exploit.  Check out the Solidity documentation on call.  It’s missing any hints as to this massive security exploit, and misleads developers into a false sense of normalcy and safety when using this construct.
  3. As a recommendation, do not call arbitrary contract code in your contract using Solidity’s call construct, ever if you can avoid it.  If you can’t, do it last and understand that you lose guarantees as to the program flow of your contract at that point.

The above imply to me that this was actually not only a flaw or exploit in the DAO contract itself: technically the EVM was operating as intended, but Solidity was introducing security flaws into contracts that were not only missed by the community, but missed by the designers of the language themselves.

I would lay at least 50% of the blame for this exploit squarely at the feet of the design of the Solidity language.  This may bolster the case for certain types of corrective action.  I refuse to lay the blame exclusively on a poorly coded contract when the contract, even if coded using best practices and following the language documentation exactly, would have remained vulnerable to attack.

A weird request for help

One of the curiosities I stumbled on in writing these posts was the DAO’s rewards account; why was the attacker collecting rewards, and who funded the account?  Hell, architecturally, why was the withdrawRewardFor function running at all in this stage of the DAO?

Let’s take a look at the DAO’s reward address. The DAO accounting documentation from Slockit pegs this address as 0xd2e16a20dd7b1ae54fb0312209784478d069c7b0. Check that account’s transactions and you see a pattern: 200 pages of .00000002 ETH transactions to 0xf835a0247b0063c04ef22006ebe57c5f11977cc4 and 0xc0ee9db1a9e07ca63e4ff0d5fb6f86bf68d47b89, the attacker’s two malicious contracts.

But check out where those coins are coming from, on the last page of the account’s transaction log: a single funding transaction, sending 0.001 ETH from 0xe76563eb8413ede9b4a1a3c1f3280a95c4b60a33 right around the end of the DAO’s creation period.

That same address also later invests in the main DAO, and holds DigixDAO tokens.

My theory: it was probably a noob who wanted to buy into the DAO crowdsale, tried to do so programatically, and failed. But how could you miss the main DAO address?  It was plastered all over the frontpage of daohub.org, and the rewards address only mentioned in some obscure depths of the DAO wiki.  So why send coins there?  Was it just someone playing with the DAO structure and seeing what they could accomplish?

Interestingly enough the only Google result for account is an Ethereum forum post asking for help with the Javascript API in sending a transaction there.  The account that posted that question never logged in again; it was their only post.

Perhaps an interesting and curious bit of blockchain history we’ll never know the answer to.  If you’re the holder of that account, I’d be curious to know your thoughts on this though.

(Please don’t start a witch hunt; this is only a curiosity and did not affect the DAO attack in any way.  Given the timeframe of the payment it is improbable this had anything to do with the latest attack.)

The attacker was not alone

One of the reasons I was able to quickly reconstruct and analyze the paths the attacker was looking at in the previous post is because I was actually working on constructing such an attack myself.

About a week ago I got a mail from Emin Gün Sirer:

(reproduced with permission, please do not bother the sender.  it’s not customary to publicize near misses in academia and I don’t know how much he wants made public.)

On Sat, 11 Jun 2016 17:42:37 -0400 Emin G Sirer <> wrote
Hi guys,
I'm pretty sure I know how to empty out The DAO.

and began several hours spent poking around the DAO for this exact vulnerability.  After a few weekend hours eliminating the potential for the exploit in several places in the code in v1.1, Sirer replied to my analyses:

On Sun, 12 Jun 2016 13:34:09 -0400 Emin Gün Sirer <> wrote
Oddly, I was one of the first people to find out about the
potential problem...
I still think that splitDAO may have a vulnerability. It violates
the withdraw pattern by not zeroing the balances[] field until after
the call. So I think it may be possible to have it move rewardTokens
to a splitting DAO multiple times. This is happening on lines 640 to
666 (hah!) of DAO.sol. Am I wrong?
In any case, this is indeed a statically analyzable vulnerability.
I'll ask Andrew Miller if he has a static analysis for this problem.

So I spent a few hours going through splitDAO, and I concluded that there was no way to trigger the recursive send vulnerability.  I did a short writeup and replied by saying it was likely not possible.  I cited timing issues, and an inability to trigger a recursive send from inside either splitDAO or createTokenProxy. I provided a few scenarios where an exploit might be credibly developed, and then I stepped away to tend to work obligations.

There were several other people in Sirer’s group working on the same, and while we did get scooped, it’s interesting to note that the attacker was not the only one who knew where to look or how to trigger the exploit.

This attack on the DAO was inevitable.  While I present this exchange more as a curiosity than an attempt to prove a point, it does give me some hope in the power of open review that we were not completely taken by surprise here.

Stepping out briefly

Thanks to everyone who read these first two posts in the series, I hope they were somewhat illuminating.

Unfortunately due to a work deadline, I need to take off of writing about the DAO until later this week (at least Wednesday if not later).  That means you won’t see much substance out of me, except perhaps a few brief Tweets.

There are still so many unexplored leads here: we still need to reconstruct the full blockchain-based picture of the attack, including where the attacker’s tokens ended up and why (if someone has already done this point me to it and I’ll be glad to edit).  We still need to search the public testnet and see whether the attacker didn’t test an attack there first.  We still need to painstakingly decompile and reconstruct his original Solidity code, for archival and analysis purposes.

We still need to come up with a list of lessons and best practices (a few thoughts: scale these contracts up slowly rather than jumping to a huge bounty, and stop using the call construct in Solidity without documented and rigorous reasoning as to its safety).  And as a community we still need to pave a path forward, whether that involves a fork or not.

The only thing I can share on all this is that the science and intrigue of smart contracts is stronger now than ever.  This is a great opportunity to emerge as a stronger, better guided, more directioned and principled community than we were last week.

I’d like to thank the attacker for the countless hours they undoubtedly spend poring over developing and testing this exploit.  Hats off to you sir, you beat us all this time.  Next time I don’t think you’ll be so lucky.

14 Responses to “Chasing the DAO Attacker’s Wake”

  1. RJ says:

    Hi there. I’m a Solidity dev. I think a better suggestion is simply to not use external calls with contracts that you cannot trust/cannot reliably know the code underneath. Most things should be handled with send(). Yes, there are some things solidity should do to better handle situations like these. We are currently working on documenting it out. The exploit around reentrance has been known for some time, but I don’t think two and two were put together until the DAO with a recursive attack…which is that deadly. Either way, we will be documenting these cases and also doing more to default enable more secure code. That I can guarantee you. The criticism is much appreciated and very constructive.

    • phil says:

      Thanks for chiming in!

      Agreed, the call construct is OK if you know what code is running. If you don’t though, as I said you can make no assumptions about your program’s state or control flow when that call completes.

    • Josh says:

      It’s safer to check and set a flag on all entry points, and clear it on exit. If you merely do the calls last you have to worry about the arguments to the call being corrupted.

      • Josh says:

        PS. Just commenting on the blog post. I didn’t mean to reply to RJ.

      • phil says:

        Sort of. If you call A->B, A sends value to a malicious address, then B sends value based on a potentially altered state. Unless you somehow encapsulate the relevant state and store it until the method invocation. But that introduces a lot of complexity.

  2. Corbin says:

    And this is why I’m shocked that, of the concepts borrowed from the object-capability literature, Ethereum borrowed smart contracts but did not follow up with capability-safe language and VM design.

  3. Joshuad31 says:

    I cannot thank you enough for this. The whole community spending many hours to dig into the details of this attack really has provided a good understanding for the non technical enthusiasts.

  4. The only thing I can share on all this is that the science and intrigue of smart contracts is stronger now than ever. This is a great opportunity to emerge as a stronger, better guided, more directioned and principled community bla bla bla blabla blabla blabla blabla blabla blabla blabla blabla bla.

    By all means, keep inflating the bubble. For one thing, USG.MIT & friends have to bleed every last coin they got before they’ll go the fuck away. For the other thing, rape’s no fun if the meat’s not at the very least squiggling and the etherape’s no different.

    • phil says:

      Don’t know if you’re actually Popescu, but if you are I have some choice words for you:

      Du-te in pizda mati, cretinule.

      I’ll approve this comment because it’s not automated spam, but it disgusts me to have this kind of garbage on my blog.

      • (Removed: Off Topic)

        • phil says:

          Please stay on topic. I reserve the right to remove all future off topic discussions as spam. Please also limit yourself to one top-level comment per article and one reply per reply (these are commenting rules for everyone on my blog and the first public announcement of such).

          I encourage you to freely state whatever on-topic opinion you have. If you’d like to contact me on any other topic, you can find me on Twitter at @phildaian. Cheers.

  5. I’m not sure that it’s the case that “you cannot assume anything about the state of your contract after the external call is executed”.

    Surely you can at least assume that the state of the contract can only change to the extent that a externally-callable function allows it to. If you’ve got a variable in your smart contract that isn’t changed by any externally-callable function, then you can at least assume that it won’t change.

    • phil says:

      You are correct, as long as you consider all externally callable functions and internal functions they call. For complex contracts it’s probably easier to make no assumptions unless you have rigorous reasoning behind them.

Leave a Reply to Mustafa Al-Bassam