Video blackjack question

I am having trouble understanding why card counting won't work against the Shuffle Master video blackjack (6 dk, deal after 2/3 shoe, but shuffle not announced). Since you don't know when there is a shuffle, why can't one just randomly choose a table, start counting the first round as if it was the start of a new shuffle and continue counting for 4 decks then restart the count? Is it the fact that because you don't know when a shuffle has occurred that makes the count meaningless? (So, half the time the count is correct and half the time it is not; so, the average would be 0.)