Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

can not get specific log from records #44

Open
wexgjduv opened this issue Jul 7, 2021 · 1 comment
Open

can not get specific log from records #44

wexgjduv opened this issue Jul 7, 2021 · 1 comment

Comments

@wexgjduv
Copy link

wexgjduv commented Jul 7, 2021

I have fluent-bit deployed in k8s with the go plugin, but we noticed some specific logs are lost.

Here is the pod's log:

{"level":"debug","ts":1625627314150.1467,"caller":"xservice/callback.go:179","msg":"api call done","app":"test","appBuildHost":"runner-x","appBuildTime":"2021-07-06 10:14:06","appGoVersion":"go1.15","appGitTag":"v0.6.0_2021-07-06_10-14-06","appGitVersion":"30333326","Resp":"task_id:1628888882647513000 data:{x_id:6108888888888787 xList:{rn:"rn:xt:x/64888888889070" timestamp:{seconds:1625627313 nanos:21238000} x_id:2020092191 status:FINISHED amount:{amount:1} x_list:{amount:{amount:1} x:{rn:"rn:xx:y/59578888888885733" x_owner:{user_rn:"rn:user:legacy-user/1688850" xx_qq_rn:"rn:xx:xx/58033951876352146:qq/59600811175680000" display_name:{language:"zh-CN" text:"张"}} type:EXTERNAL display_name:{language:"zh-CN" text:"测试 Mall"}} external_xx_info:"{\"Data\":{\"XInCent\":1,\"CreateTime\":1625627313,\"Cent\":1,\"ETime\":1625627313,\"ErrorCode\":null,\"ErrorDescription\":null,\"Status\":\"Success\"},\"ID\":\"73b7cb2ad93f\",\"NotifyTime\":1625627313,\"NotifyType\":\"XResult\",\"XServiceID\":1712788888888888889,\"XNumber\":46766078888888888875}"}}}","x-trace-id":"da488888888888758893429","x-span-id":"7188888888833","x-parent-id":"388888888886"}

In FLBPluginFlushCtx, I tried to print all records but found no messages like api call done in fluent-bit's output.

//export FLBPluginFlushCtx
func FLBPluginFlushCtx(ctx, data unsafe.Pointer, length C.int, tag *C.char) int {
    // Type assert context back into the original type for the Go variable
    // cfgContext := output.FLBPluginGetContext(ctx).(map[string]string)

    dec := output.NewDecoder(data, int(length))

    messages := make(map[string][]string)

    for {
        ret, _, record := output.GetRecord(dec)
        if ret != 0 {
            break
        }

        fbTag := fmt.Sprintf("%s", C.GoString(tag))

        for k, v := range record {
            fmt.Printf("tag: %s, record: \"%s\": %v, \n", fbTag, k, v)
        }
    }

    return output.FLB_OK
}
@wexgjduv
Copy link
Author

wexgjduv commented Jul 7, 2021

After inspecting the file contents in running pod's log file, some logs had wrong encoding like \u0008˵\ufffd or \ufffd<90>\ufffd\u or \ufffdʁ\ufffd\. I disabled outputting of these logs, other logs were collected by our plugin successfully.
I think the encoding of the pod's log may affect the collectiong of other logs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant