Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send JSON encoded payloads without re-marshalling them? #127

Open
grandstairs opened this issue Jun 8, 2024 · 0 comments
Open

Send JSON encoded payloads without re-marshalling them? #127

grandstairs opened this issue Jun 8, 2024 · 0 comments

Comments

@grandstairs
Copy link

I would like to use this library as the basis for multiple logging libraries. However, the only way I was able to figure was to create an io.WriteCloser that wraps this library to do the .Post(tag, data) with the actual log line.

type FluentWriter struct {
	Fluent *fluent.Fluent
	Tag    string
}

func (w *FluentWriter) Close() error {
	return w.Fluent.Close()
}

func (f *FluentWriter) Write(p []byte) (n int, err error) {
	// this is wasteful since the logger just spent time to json.Encode this very payload.
	msg := map[string]interface{}{}
	json.Unmarshal(p, &msg)
	err = f.Fluent.Post(f.Tag, msg)
	return len(p), err
}

The issue is that Post() and EncodeAndPostData() both expect an non-encoded map/struct/interface which isn't the b []byte that an io.Writer provides.

f, err := fluent.New(fluent.Config{});
logSink := &logger.FluentWriter{Fluent: f, Tag: "ApplicationLogs"}
logger := zerolog.New(logSink).With().Timestamp().Logger()

Is there either 1) a better way to wrap this library so I can avoid this encode/decode/encode process or 2) is there a way to send a raw b []byte payload via msgpack without fluent-logger-golang needing to mess with it?

As-is, there really isn't much performance savings going with zerolog or zap over using something like slog or logrus because there will be a ton of extra memory allocations running the JSON encoder/decoder over the payloads.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant