No. When seen from that perspective, don’t drink that milk which is procured by harming cows. Swamiji believes in a non-diary vegan diet. This according to him is ahimsa diet.

My grandfather reared cows. The cows would feed their calves and then the excess milk would simply leak from their udders. My grandmother would place a bowl beneath the udders of the cows and fetch that milk. We would drink that milk. More importantly, after the cow has fed its calf, the excess milk needs to be milked gently. If it is not milked, the cow suffers. It is for the well-being of the cow that it is milked. Drinking that milk is not harmful. Despite that if you do not want to drink it, give it to someone who needs. No harm.

But in my opinion, by drinking factory milk you can never accumulate sin. This is for the simple reason that what you drink is not milk at all. It is not real milk. It is reconstituted using artificial substances.

In the past there were many who strictly followed vegetarian diet and refrained from consuming meat. Did they not live for thousands of years? It is foolish to believe that without animal protein and without eggs a person cannot get adequate nutrition. It is wrong to believe that animal diet helps us live longer. An elephant has a huge body. It is heavy weight and is also an intelligent animal. But it eats only vegetarian diet.

(Q&A Session 13 Dec 2015)