A budget one wouldn't have the key cards (mainly for side-decking) such as Thunder King Rai-oh, Doomcaliber Knight and Banisher of the Radiance. A budget one may also not be able to get 3 structure decks, missing out on 3 Gearframe and 3 fortress.
It may also not contain staples such as Solemn Warning, Solemn Judgment, Mirror Force and Forbidden Lance.