test ~ expand

given : 
in ~joe a file named a.txt
and the next script :

------------
#!/bin/bash
$a=joe
test -e ~$a/a.txt && echo "yes"
------------

it does not output anything.
WHY NOT ????????

and if we do : 
test -e ~joe/a.txt && echo "yes"

it writes at stdin "yes"

WHY ????????????