Improve opentracing annotations for Notifier (#10111)

The existing tracing reports an error each time there is a timeout, which isn't
really representative.

Additionally, we log things about the way `wait_for_events` works
(eg, the result of the callback) to the *parent* span, which is confusing.

changelog.d/10111.misc

@@ -0,0 +1 @@
+Improve opentracing annotations for `Notifier`.

synapse/notifier.py

@@ -485,6 +485,7 @@ class Notifier:
             end_time = self.clock.time_msec() + timeout
 
             while not result:
+                with start_active_span("wait_for_events"):
                     try:
                         now = self.clock.time_msec()
                         if end_time <= now:
@@ -499,7 +500,6 @@ class Notifier:
                             self.hs.get_reactor(),
                         )
 
-                    with start_active_span("wait_for_events.deferred"):
                         log_kv(
                             {
                                 "wait_for_events": "sleep",