@renchap poking at the new #otel support in 4.3.0, and there's a bunch of neat data here. yay!
-
@renchap poking at the new #otel support in 4.3.0, and there's a bunch of neat data here. yay!
is there any value in keeping these repeated ZSCAN traces? they're the vast majority of our current event usage, and they don't seem to be terribly meaningful. usually (maybe exclusively) just a span that says ZSCAN.
safe to drop these? i don't think we'll lose any meaningful observability by doing so?
#hachyderm #hachyinfra #MastoInfra #opentelemetry
-
@esk Are they part of a trace, and/or do they have a parent span?
I think they are Sidekiq workers polling Redis and trying to see if there are any jobs to process, in which case they can be filtered (but maybe not all ZSCANs?). If thats the case, we should probably filter them out directly in our instrumentation, or better sample them heavily.
For mastodon.social we only keep 1% of traces, except for errors where we keep 100% of them. -
@renchap they seem to be the full trace - no parent/children that I can find (granted, i'm a big honeycomb noob).
we plan to turn on sampling for things here shortly to make the total number of traces less... overwhelming.
-
@esk So this can probably be dropped, and if you can open a PR on the Mastodon repo that either heavily sample or maybe drop those single calls, it will probably get merged as I dont really see how those would be useful.
