If your broccoli leaves are deformed and curled, and upon inspection you can see colonies of grey or green soft-bodied insects with or without ants, then your plant likely has aphids.
The first approach to controlling aphids is to spray them off your plants with a strong spray of water from the hose. There are other measures that can be taken as well. Read more about controlling aphids.
Irregular holes in broccoli leaves or chewed leaves are probably caused by some kind of caterpillar. Common caterpillars that attack broccoli, cabbage, and similar plants include cabbage loopers and armyworms. Effective control measures include handpicking them off, using a bacterial-based insecticide such as Bt or Spinosad, or using a row cover to prevent moths and butterflies from laying their eggs on your plants.
Read more about controlling caterpillars in the garden.
Snails and Slugs
Snails and slugs can cause significant damage in the garden. They destroy young plants and chew holes in leaves and flowers. Generally they leave a silvery mucous trail. Control measures require several different approaches.
Read more about controlling snails and slugs.