Before specifying version, we were just reading json logs from buildOutput.Body, but now that only serves the Successfully tagged message at the end.
Could someone help understand how to visualize the builderkit build logs like you would the v1 builder logs?
","upvoteCount":3,"answerCount":3,"acceptedAnswer":{"@type":"Answer","text":"This is what I have figured out so far:
\nYou need to use the jsonmessage package (github.com/docker/docker/pkg/jsonmessage`) to read the response body like you did with V1.
I think there is a helper function in that package you can use. I use a custom function that looks like this:
\nfunc DockerMessages(in io.Reader, auxCallback func(jsonmessage.JSONMessage)) error {\n\tdec := json.NewDecoder(in)\n\n\tfor {\n\t\tvar jm jsonmessage.JSONMessage\n\t\tif err := dec.Decode(&jm); err != nil {\n\t\t\tif err == io.EOF {\n\t\t\t\tbreak\n\t\t\t}\n\t\t\treturn fmt.Errorf(\"decode JSON message: %w\", err)\n\t\t}\n\n\t\t// If the message contains an error, return it to the caller\n\t\t// after writing. This is how Docker communicates things like\n\t\t// image builds failing.\n\t\tif jm.Error != nil {\n\t\t\treturn jm.Error\n\t\t}\n\n\t\t// Check for out of band data (like image IDs, digests, etc.)\n\t\tif jm.Aux != nil {\n\t\t\tif auxCallback != nil {\n\t\t\t\tauxCallback(jm)\n\t\t\t}\n\t\t\tcontinue // (docker ignores the rest if there is aux data)\n\t\t}\n\t\t\n\t\t// ...With Buildkit all of the data is on JSONMessage.Aux. You can switch on JSONMessage.ID to guess what the Aux type will be. For buildkit output from an image build it will be moby.buildkit.trace.
The buildkit trace is a binary encoded protobuf message which is then JSON encoded, so it's a base64 JSON string. You can decode it like this:
\nimport controlapi \"github.com/moby/buildkit/api/services/control\"\n\nvar bytes []byte\nif err := json.Unmarshal(*jm.Aux, &bytes); err != nil {\n\tpanic(err)\n}\nresp := controlapi.StatusResponse{}\nif err := proto.Unmarshal(bytes, &resp); err != nil {\n\tpanic(err)\n}The proto is defined here.
\nEdit: here's a more complete example that just prints the output to the terminal: https://gist.github.com/matt-allan/9d5419abd2c234011de175e891be48fb
\nIt uses the \"github.com/moby/buildkit/util/progress/progressui\" package to print the output.
","upvoteCount":2,"url":"https://github.com/moby/moby/discussions/43788#discussioncomment-13291612"}}}How to Read BuildKit Build Logs #43788
-
|
Howdy. I'm working on a cli tool that builds docker images as part of its workflow It was relatively straightforward serving the logs with the v1 builder, but I'm having a hard time figuring out how to serve the logs to the end user when using builderkit. Before specifying version, we were just reading json logs from Could someone help understand how to visualize the builderkit build logs like you would the v1 builder logs? |
Beta Was this translation helpful? Give feedback.
Replies: 3 comments 1 reply
-
|
BuildKit client is a bit more involved than the classic builder; the code in the Docker CLI could be a starting point; https://github.com/docker/cli/blob/v20.10.17/cli/command/image/build_buildkit.go However, for the next release, build with buildkit will be backed by builds, which implements more of the BuildKit features; code can be found here; https://github.com/docker/buildx/blob/5b452b72a2bf9945bb7d062cf28842430f910777/commands/build.go#L90 |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for the info! Apologies in advance for my ignorance. Regarding the next release: does this mean we'll be able to simply call client.ImageBuild, and read from the output's body for build logs as we can without buildkit, or will we need to hook into other api calls or something like that? |
Beta Was this translation helpful? Give feedback.
-
|
This is what I have figured out so far: You need to use the jsonmessage I think there is a helper function in that package you can use. I use a custom function that looks like this: func DockerMessages(in io.Reader, auxCallback func(jsonmessage.JSONMessage)) error {
dec := json.NewDecoder(in)
for {
var jm jsonmessage.JSONMessage
if err := dec.Decode(&jm); err != nil {
if err == io.EOF {
break
}
return fmt.Errorf("decode JSON message: %w", err)
}
// If the message contains an error, return it to the caller
// after writing. This is how Docker communicates things like
// image builds failing.
if jm.Error != nil {
return jm.Error
}
// Check for out of band data (like image IDs, digests, etc.)
if jm.Aux != nil {
if auxCallback != nil {
auxCallback(jm)
}
continue // (docker ignores the rest if there is aux data)
}
// ...With Buildkit all of the data is on The buildkit trace is a binary encoded protobuf message which is then JSON encoded, so it's a base64 JSON string. You can decode it like this: import controlapi "github.com/moby/buildkit/api/services/control"
var bytes []byte
if err := json.Unmarshal(*jm.Aux, &bytes); err != nil {
panic(err)
}
resp := controlapi.StatusResponse{}
if err := proto.Unmarshal(bytes, &resp); err != nil {
panic(err)
}The proto is defined here. Edit: here's a more complete example that just prints the output to the terminal: https://gist.github.com/matt-allan/9d5419abd2c234011de175e891be48fb It uses the "github.com/moby/buildkit/util/progress/progressui" package to print the output. |
Beta Was this translation helpful? Give feedback.
This comment was marked as spam.
This comment was marked as spam.
This is what I have figured out so far:
You need to use the jsonmessage
package (github.com/docker/docker/pkg/jsonmessage`) to read the response body like you did with V1.I think there is a helper function in that package you can use. I use a custom function that looks like this: