Dear Nileesh,
you can restrict the fields at various levels.
1.Backend while defining the Material type you can make field with tcode OMSR and OMS9.
2.you can remove them from Governance process in MDG.
3.you can do it at CR level but it will not be material specific.
regards
shankar