My retaining wall shifted after I skimped on soil prep
I was putting up a small garden wall and didn't pack down the dirt well enough. After some rain, the whole section moved and bricks came loose. Had to tear it all out and begin again from scratch. Don't cut corners on base work like I did. A little extra time there saves a huge headache later.
Question if it's really that big a deal. My neighbor's wall has been crooked for ages and it still holds back the dirt. You might be overthinking the base work. A few loose bricks don't mean you have to start over. Sometimes a quick fix saves more time than redoing everything.