The full letter from Seneca to Lucilius (really good stuff in bold):

Do you ask me what you should regard as especially to be avoided? I say, crowds; for as yet you cannot trust yourself to them with safety. I shall admit my own weakness, at any rate; for I never bring back home the same character that I took abroad with me.  Something of that which I have forced to be calm within me is disturbed; some of the foes that I have routed return again.  Just as the sick man, who has been weak for a long time, is in such a condition that he cannot be taken out of the the house without suffering a relapse, so we ourselves are affected when our souls are recovering from a lingering disease.  To consort with the crowd is harmful; there is no person who does not make some vice attractive to us, or stamp it upon us, or taint us unconsciously therewith.  Certainly, the greater the mob with which we mingle, the greater the danger.

But nothing is so damaging to good character as the habit of lounging at the games; for then it is that vice steals subtly upon one through the avenue of pleasure.  What do you think I mean? I mean that I come home more greedy, more ambitious, more voluptuous, and even more cruel and inhuman, because I have been among human beings.  By chance I attended a mid-day exhibition, expecting some fun, wit, and relaxation, – an exhibition at which men’s eyes have respite from the slaughter of their fellow-men.  But it was quite the reverse.  The previous combats were the essence of compassion; but now all the trifling is put aside and it is pure murder.  The men have no defensive armour.  They are exposed to blows at all points, and no one ever strikes in vain.  Many persons prefer this programme to the usual pairs and to the bouts “by request.” Of course they do; there is no helmet or shield to deflect the weapon.  What is the need of defensive armour, or of skill?  All these mean delaying death.  In the morning they throw men to the lions and the bears; at noon, they throw them to the spectators.  The spectators demand that the slayer shall face the man who is to slay him in his turn; and they always reserve the latest conqueror for another butchering.  The outcome of every fight is death, and the means are fire and sword.  This sort of thing goes on while the arena is empty.  You may retort: “But he was a highway robber; he killed a man!” And what of it?  Granted that, as a murderer, he deserved this punishment, what crime have you committed, poor fellow, that you should deserve to sit and see this show?  In the morning they cried “Kill him!  Lash him!  Burn him; Why does he meet the sword in so cowardly a way?  Why does he strike so feebly? Why doesn’t he die game?  Whip him to meet his wounds!  Let them receive blow for blow, with chests bare and exposed to the stroke!” And when the games stop for the intermission, they announce: “A little throatcutting in the meantime, so that there may still be something going on!
Come now; do you/a not understand even this truth, that a bad example. reacts on the agent?  Thank the immortal gods that you are teaching cruelty to a person who cannot learn to be cruel. The young character, which cannot hold fast to righteousness, must be rescued from the mob; it is too easy to side with the majority.  Even Socrates, Cato, and Laelius might have been shaken in their moral strength by a crowd that was unlike them; so true it is that none of us, no matter how much he cultivates his abilities, can withstand the shock of faults that approach, as it were, with so great a retinue.  Much harm is done by a single case of indulgence or greed; the familiar friend, if he be luxurious, weakens and softens us imperceptibly; the neighbour, if he be rich, rouses our covetousness; the companion, if he be slanderous, rubs off some of his rust upon us, even though we be spotless and sincere.  What then do you think the effect will be on character, when the world at large assaults it!  You must either imitate or loathe the world.

But both courses are to be avoided; you should not copy the bad simply because they are many, nor should you hate the many because they are unlike you.  Withdraw into yourself, as far as you can, Associate with those who will make a better man of you.  Welcome those whom you yourself can improve. The process is mutual; for men learn while they teach.  There is no reason why pride in advertising your abilities should lure you into publicity, so that you should desire to recite or harangue before the general public. Of course I should be willing for you to do so if you had a stock-in-trade that suited such a mob; as it is, there is not a man of them who can understand you.  One or two individuals will perhaps come in your way, but even these will have to be molded and trained by you so that they will understand you.  You may say: “For what purpose did I learn all these things?” But you need not fear that you have wasted your efforts; it was for yourself that you learned them.

In order, however, that I may not today have learned exclusively for myself, I shall share with you three excellent sayings, of the same general purport, which have come to my attention. This letter will give you one of them as payment of my debt; the other two you may accept as a contribution in advance.  Democritus says: “One man means as much to me as a multitude, and a multitude only as much as one man.” The following also was nobly spoken by someone or other, for it is doubtful who the author was; they asked him what was the object of all this study applied to an art that would reach but very few.  He replied: “I am content with few, content with one, content with none at all.”
Written to one of the partners of his studies: “I write this not for the many, but for you; each of us is enough of an audience for the other.” Lay these words to heart, Lucilius, that you may scorn the pleasure which comes from the applause of the majority.  Many men praise you; but have you any reason for being pleased with yourself, if you are a person whom the many can understand?  Your good qualities should face inwards. 

Farewell.