du | sort fails

I would like to sort the output of du using the sort command.  But
this fails:

du <dir> | sort

because du prints the sizes out like this:

123
12
1

rather than this: (prefixing with spaces)

123
 12
  1

Which would work.