The backpack hostel / guesthouse there is very cheap, on average I believe USD10-20 is already not bad.
For transportation, if the distance is quite far, then you can consider taking overnight train / bus which you can save the travel time and also the overnight accommodation costs.
In general, I would say Vietnam is a bit more expensive comparing with Cambodia / Laos. But still it's a cheap place to travel in Hong Kong standard.