Transform v1 sanitizers into processors or something else

[yurishkuro]

In the v1 collector all received spans are passed through the sanitizers framework cmd/collector/app/sanitizer/. This is currently missing in jaeger-v2. Since the jaegerexporter in v2 operates on OTLP data, the existing sanitizers won't work directly, and will need to be rewritten to operate on OTLP format directly.

There are two options how the sanitizers can be invoked. One is to invoke them automatically in the exporter. Another is by defining them as span processors in the OTEL collector pipeline. This allows users to cherry-pick which sanitizers they want to use, but also moves the burden of configuration onto the user, whereas in collector-v1 the default set of sanitizers was always used automatically. I would suggest invoking them automatically in the exporter - in the future if configurability is desirable it can be implemented.

The main requirements are

1. bad data should be sanitized so that it's now "good", e.g. no malformed utf8 strings present anywhere, since they can break transmission
2. the original bad data should still be preserved in some encoded (safe) form for debugging purposes.

[varshith257]

Automatically invoking the sanitizers in the exporter seems like a good approach, ensuring consistent data sanitization without burdening users with additional configuration. This approach maintains simplicity while addressing the immediate need for sanitization in the v2 collector.

[arujjval]

@yurishkuro So for now, we have to work on rewriting existing sanitizers for operating on OTLP right?

[yurishkuro]

Yes.

[mahadzaryab1]

@yurishkuro what's the status of this? is there anything I can help with?

[yurishkuro]

Yes , I think this stalled

[mahadzaryab1]

@yurishkuro what can I help with? are there subtasks for this issue?

[yurishkuro]

@mahadzaryab1 see comments in #5551 - it's a good direction, but needs completion.

[mahadzaryab1]

@yurishkuro do you know what the reasoning behind https://github.com/varshith257/jaeger/blob/v2-sanitizer/cmd/collector/app/sanitizer/utf8_sanitizer.go#L76-L78 is? if the value isn't a valid utf-8 string why is it added to the map? isn't this just a no-op?

[mahadzaryab1]

@yurishkuro do you know what the reasoning behind https://github.com/varshith257/jaeger/blob/v2-sanitizer/cmd/collector/app/sanitizer/utf8_sanitizer.go#L76-L78 is? if the value isn't a valid utf-8 string why is it added to the map? isn't this just a no-op?

Oh I see. They get converted to bytes. Do we want the same behaviour for the v2 sanitizer ?

[yurishkuro]

Yes. The motivation is to make the data well formed, but also to preserve original invalid data for debugging.