Totally up to your comfort and desire to record stuff.
IMO, for most people, It's probably not necessary if you're sticking with broccoli, asparagus, and cruciferous vegetables... unless you're eating a crazy amount, or you have a heavy carb restriction, or your already very lean. Like 5 CUPS of broccoli is what, 100-120 cals maybe? That's not going to change anything for you if you're chubby. And if you feel the need to subtract fibrous carbs, it'll mean even less. But if 100 cals makes or breaks your diet, then yes, you should.
If you stray into vegetables like peas, corn, soybeans, potatoes... then yes. you probably should write them down.
The calories reported by food companies and not exactly accurate anyways (1). Personally, I eat 6 times a day and eat about 4 cups of broccoli a day. I write down everything because its just easier.... but on the whole, I can't measure accurately enough to where I'm TRULY counting calories down to within 16-20 cals per meal. I'm ok with being mathematically off in my calories every day, as long as I'm consistently off throughout the week. I think I'm eating about 2800 cals/day but in reality its probably a range of 2500-3000.... but it doesn't matter. Even if your strictly talking "calories" and you're off by 1000 a day, just be consistent.
Keep it simple,
C-
Source:
(1)
Can You Trust Calorie Counts on Food Labels? | Berkeley Wellness