Asking for help with hardware decoding to convert into JPG images.

[DarkiT]

Thanks for the great work you have done with this library!

I am using the hardware_decoding example for hardware decoding, and I have obtained Frame.Data(). However, I keep encountering errors when trying to save the frame as a JPG image, which prevents me from proceeding.

I plan to take timed screenshots with my camera and send them to WeChat, but due to the aforementioned errors, I can't continue. Could someone help me? Here is the decoder information for my camera's VLC streaming.

Can you provide a demo for saving a frame to an image?

[asticode]

Could you share the code and the error you're getting?

Anyway check out this part of the Demuxing/Decoding example on how to properly generate a go image with the data of a frame. Once you get an image.Image, you can use jpeg.Encode or png.Encode to save it to a file. Beware as you may need to convert to proper RGBA in order to get good result.

[DarkiT]

This is my streaming information.

Input #0, rtsp, from 'rtsp://192.168.1.175':
  Metadata:
    title           : Media Presentation
  Duration: N/A, start: 0.040000, bitrate: N/A
  Stream #0:0: Video: hevc (Main), yuv420p(tv), 2560x1440, 25 fps, 25 tbr, 90k tbn

Now it's working properly, but the conversion efficiency is too low. How can I use RKMPP hardware acceleration to achieve image transcoding?

if s.inputStream.CodecParameters().MediaType() == astiav.MediaTypeVideo {

	fd := finalFrame.Data()

	var err error
	if i, err = fd.GuessImageFormat(); err == nil {
		if err := fd.ToImage(i); err != nil {
			log.Error("guessing image format failed: %v", err)
		}
	} else {
		nv12, err := fd.Bytes(1)
		if err != nil {
			log.Error("copying frame data to the image failed: %v", err)
			return
		}
		i = NV12ToRGBA(nv12, finalFrame.Width(), finalFrame.Height())
	}
}
				
// NV12ToRGBA converts NV12 formatted data to an RGBA image.
func NV12ToRGBA(nv12 []byte, width, height int) *image.RGBA {
	// Create an RGBA image
	rgba := image.NewRGBA(image.Rect(0, 0, width, height))

	// Iterate through each pixel in the NV12 data and convert it to RGBA
	for y := 0; y < height; y++ {
		for x := 0; x < width; x++ {
			// Calculate indices for Y, U, and V components
			yIndex := y*width + x
			uvIndex := width*height + (y/2)*width + (x/2)*2

			// Extract Y, U, and V components
			yComp := float64(nv12[yIndex])
			uComp := float64(nv12[uvIndex])
			vComp := float64(nv12[uvIndex+1])

			// Convert YUV to RGB
			r := clamp(yComp + 1.402*(vComp-128))
			g := clamp(yComp - 0.344136*(uComp-128) - 0.714136*(vComp-128))
			b := clamp(yComp + 1.772*(uComp-128))

			// Set the RGBA values
			rgba.Set(x, y, color.RGBA{R: r, G: g, B: b, A: 255})
		}
	}

	return rgba
}

// Helper function to clamp a value to the range [0, 255]
func clamp(v float64) uint8 {
	if v < 0 {
		return 0
	}
	if v > 255 {
		return 255
	}
	return uint8(v)
}

[asticode]

You could use a filter instead (checkout format and possibly hwdownload as well as the example) and if your goal is to transcode checkout this example

[Tryanks]

If your target platform is RKMPP, you may need to use go-astiav/bump/n6.1 (see issues #27 and #57). For hardware encoding and decoding, consider using ffmpeg-rockchip. The mainline FFmpeg does not support Rockchip Mpp.

[asticode]

Closing this issue, but feel free to reopen it if needed 👍