My Guess as to why
I don't recall that quote specifically, although I have read similar quotes in other books.
As to the limited number of players for single deck, you are right when you say to get more rounds in per shuffle, and thus a lower percentage of hands being dealt off the top, when the player is at a disadvantage. I believe one of Stanford Wong's books, Professional Blackjack or Blackjack Secrets, says to play single deck with no more than two other players. Also, I seem to recall an article in Blackjack Insider that claimed the cut card effect on a single deck game has a big negative consequence in you expected win when 4 players are at the table and the deck is being dealt to 31-35 cards. Again, the cause is a higher percentage of hands being dealt from a full deck.
As far as playing with good players, it does make a difference if the you are playing in a face down game, and you are playing after the others (that is you are at third base or the other players are to your right).
In that circumstance you can use a technique called "counting with inference" to resolve close calls with your playing strategy decisions. For example, lets say you know the player to you right is a good player, and you see him stand with his two down cards only when the dealer has a 7 through Ace up. In this instance you can go on the assumption that the player probably holds at least 17, (after all he stood) and thus his hand probably includes one 10 card. Thus if you otherwise had a close call, say a 16 versus a dealer 1 with a true count of 0 (HiLo method), or a 9 versus a 7 with a true count of +4 (also HiLo), your adding in this one extra piece of information would change you play from stand to hit in the first instance, or from double down to hit in the second instance--because you would adjust your running count by -1 due to the infered cards. But, if you had doubts about the other players ability, you could not make that assumption safely as that player just might be standing on a stiff hand.
As you can see, counting with inference is a fine point of the game. It is limited to face down games with players you know have playing ability and they are seated to your right. Also, it weighs in only when your hand is a close call, and consequently, the gain from it is not large and it occurs infrequently. Still, the nature of blackjack advantage play requires you try to accumulate as many small edges as possible to get the upper hand while playing.
Bryce Carlson, in Blackjack for Blood, discusses another circumstance when you can infer what the other players downcards are...when the player hits to get exposed cards summing to 10 or more and he does not bust. In this instance, you can assume the player is holding small cards, and thus that increases your likelihood of getting a high card. Your adjustment is to add to your running count +1 before making your playing decision.
I think Lance Humble's book "Worlds Greatest Blackjack Book" may also discuss this modification to you play. By the way, Humble's book relates some of the author's encounters with Mr. Revere, and it per Humble's account, Mr. Revere was a colorful character.
Hope this helps. Sorry if I overwhelmed you.