Is there a way to exclude caretakers from "select idle constructors"?

should be able to find it at Settings -> HUD Panel -> Quick Selection Bar -> Track idle caretakers, remember to turn off simple settings at the bottom right