Is it unhealthy to include lots of er and other dairy products in your diet?

im a vegetarian ( because of religious reasons. though i have chicken like once in two weeks. im east indian by the way). and im also a bit of a picky eater. we usually have a lot of vegetables and rice in our diet ( no spicy food. my family uses a lot of herbs while cooking ). i really love dairy products and i use er and cream liberally in almost every dish i eat even though i don't really like sweet stuff. but recently i read articles on how dairy products cause way lot of infections and can worsen sinusitis ( which im diagnosed with) and can even cause cholestrol and weight gain ( im not really fat though im 5'9and i weigh 50-52 kgs and i've been having lots of dairy products since i was a kid). do i need to change my diet because of problems it might cause in future?