try (one's) patience
To frustrate or annoy one by continued unwanted behavior; to test the limits of one's patience. His tangential questions are clearly trying the professor's patience, who asked that all questions be held until the end of the lecture. Will you take the kids to the playground for an hour? They're really trying my patience.
try somebody’s ˈpatience
make somebody feel impatient: Jim’s constant complaining was really beginning to try her patience.