[BUG] Frame Rate Drop and Stuttering in Egress Output at 1080p 30fps

[ozgur-d]

Describe the bug
When attempting to stream at 1920x1080 30fps with a 6000 bitrate, the stream looks smooth via the egress layout URL but experiences significant stuttering (as if at a low framerate) in both the file output and RTMP stream. CPU usage on the server remains around 50%. Reducing the resolution to 1280x720 at 30fps improves the stream, but it’s still not flawless. Changing the bitrate does not affect the result. As motion in the video increases, the quality doesn’t degrade significantly, but the framerate drops further, leading to more noticeable stuttering. I’m also sharing OBS-captured output at 30fps and the RTMP stream output for comparison.

Obs Record From Layout: https://www.youtube.com/watch?v=QH4Ugk4nJB8
Stream Record: https://www.youtube.com/watch?v=nv8UauKl41g
Egress Version
1.8.2

Egress Request
The request is as follows (TypeScript backend):

const encodingOptions: EncodingOptions = {
  width: 1920,
  height: 1080,
  framerate: 30,
  videoBitrate: 6000,
  videoCodec: VideoCodec.H264_MAIN,
  audioBitrate: 128,
  audioCodec: AudioCodec.OPUS,
  depth: 24,
  audioFrequency: 44100,
};

const options: RoomCompositeOptions = {
  layout: roomSettings.streamLayout,
  customBaseUrl: process.env.EGRESS_LAYOUT_URL,
  encodingOptions: encodingOptions,
};

await this.egressClient.startRoomCompositeEgress(
  roomMember.room.uid,
  rtmpOutput,
  options,
);

Additional context
My egress server has 8 CPU cores. I have tried different settings for videoCodec and audioCodec, but the issue persists. The problem seems tied to high resolutions and framerate inconsistencies. The server load seems moderate at around 50% CPU usage during streams. I also tried overriding the following values, but the result remained the same:

cpu_cost: # optionally override cpu cost estimation, used when accepting or denying requests
  room_composite_cpu_cost: 8.0
  web_cpu_cost: 8.0
  track_composite_cpu_cost: 6.0
  track_cpu_cost: 3.0

I have tested with the following options for frontend typescript:

<LiveKitRoom
options={{
  adaptiveStream: true (also false),
  dynacast: false,
  videoCaptureDefaults: {
    resolution: {
      width: 1280,
      height: 720,
      aspectRatio: 1280 / 720,
      frameRate: 30,
    },
  },
}}

Logs

2024-09-16T15:13:08.730Z        INFO    egress  redis/redis.go:142      connecting to redis     {"nodeID": "NE_Rm6y42thEsMs", "clusterID": "", "simple": true, "addr": "90px-live.redis.cache.windows.net:6379"}
2024-09-16T15:13:08.753Z        INFO    egress  stats/monitor.go:169    cpu available: 8.000000 max cost: 4.000000      {"nodeID": "NE_Rm6y42thEsMs", "clusterID": ""}
2024-09-16T15:13:08.753Z        DEBUG   egress  service/service_debug.go:38     debug handler disabled  {"nodeID": "NE_Rm6y42thEsMs", "clusterID": ""}
2024-09-16T15:13:08.753Z        DEBUG   egress  service/service.go:137  starting service        {"nodeID": "NE_Rm6y42thEsMs", "clusterID": "", "version": "1.8.2"}
2024-09-16T15:13:08.753Z        INFO    egress  service/service.go:143  service ready   {"nodeID": "NE_Rm6y42thEsMs", "clusterID": ""}
2024-09-16T15:13:08.753Z        DEBUG   egress  service/service.go:125  starting template server on address localhost:7980      {"nodeID": "NE_Rm6y42thEsMs", "clusterID": ""}
2024-09-17T13:29:32.877Z        DEBUG   egress  stats/monitor.go:306    cpu check       {"nodeID": "NE_Rm6y42thEsMs", "clusterID": "", "total": 8, "pending": 0, "used": 0, "required": 4, "available": 8, "activeRequests": 0, "canAccept": true}
2024-09-17T13:29:33.409Z        DEBUG   egress  stats/monitor.go:306    cpu check       {"nodeID": "NE_Rm6y42thEsMs", "clusterID": "", "total": 8, "pending": 0, "used": 0, "required": 4, "available": 8, "activeRequests": 0, "canAccept": true}
2024-09-17T13:29:33.409Z        INFO    egress  service/service_rpc.go:45       request received        {"nodeID": "NE_Rm6y42thEsMs", "clusterID": "", "egressID": "EG_VVWiJwdLCZii"}
2024-09-17T13:29:33.421Z        INFO    egress  service/service_rpc.go:60       request validated       {"nodeID": "NE_Rm6y42thEsMs", "clusterID": "", "egressID": "EG_VVWiJwdLCZii", "requestType": "room_composite", "outputType": "stream", "room": "6c123486-3ea5-48a3-9cdf-a3a888076f5c", "request": {"RoomComposite":{"room_name":"6c123486-3ea5-48a3-9cdf-a3a888076f5c","layout":"screenshare","custom_base_url":"https://egresslayout-azure.test.net","Output":{"Stream":{"protocol":1,"urls":["rtmp://cdn-static.testnet/LiveApp/{jfW...865}","rtmp://20.218.136.230:1934/live/{6c1...863}"]}},"Options":{"Advanced":{"width":1920,"height":1080,"depth":24,"framerate":30,"audio_codec":1,"audio_bitrate":128,"audio_frequency":44100,"video_codec":2,"video_bitrate":6000}}}}}
2024-09-17T13:29:33.443Z        DEBUG   egress  server/main.go:188      handler launched        {"nodeID": "NE_Rm6y42thEsMs", "handlerID": "EGH_HQTgqiYgufrQ", "clusterID": "", "egressID": "EG_VVWiJwdLCZii"}
2024-09-17T13:29:33.444Z        INFO    egress  redis/redis.go:142      connecting to redis     {"nodeID": "NE_Rm6y42thEsMs", "handlerID": "EGH_HQTgqiYgufrQ", "clusterID": "", "egressID": "EG_VVWiJwdLCZii", "simple": true, "addr": "90px-live.redis.cache.windows.net:6379"}
2024-09-17T13:29:33.459Z        DEBUG   egress  source/web.go:149       creating pulse sink     {"nodeID": "NE_Rm6y42thEsMs", "handlerID": "EGH_HQTgqiYgufrQ", "clusterID": "", "egressID": "EG_VVWiJwdLCZii"}
2024-09-17T13:29:33.470Z        DEBUG   egress  source/web.go:173       creating X display      {"nodeID": "NE_Rm6y42thEsMs", "handlerID": "EGH_HQTgqiYgufrQ", "clusterID": "", "egressID": "EG_VVWiJwdLCZii", "display": ":1028125055", "dims": "1920x1080x24"}
2024-09-17T13:29:33.471Z        DEBUG   egress  source/web.go:204       launching chrome        {"nodeID": "NE_Rm6y42thEsMs", "handlerID": "EGH_HQTgqiYgufrQ", "clusterID": "", "egressID": "EG_VVWiJwdLCZii", "url": "https://egresslayout-azure.test.net?layout=screenshare&token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE3MjY2NjYxNzMsImlzcyI6IjZlbTIzZmtnZnIzc25iZDIiLCJraW5kIjoiZWdyZXNzIiwibmJmIjoxNzI2NTc5NzczLCJzdWIiOiJFR19WVldpSndkTENaaWkiLCJ2aWRlbyI6eyJjYW5QdWJsaXNoIjpmYWxzZSwiY2FuUHVibGlzaERhdGEiOmZhbHNlLCJjYW5TdWJzY3JpYmUiOnRydWUsImhpZGRlbiI6dHJ1ZSwicmVjb3JkZXIiOnRydWUsInJvb20iOiI2YzEyMzQ4Ni0zZWE1LTQ4YTMtOWNkZi1hM2E4ODgwNzZmNWMiLCJyb29tSm9pbiI6dHJ1ZX19.0_Yp-bZSMVzj3yAzszpIR7MrwDG-oWTDwKYw5QnuSda&url=wss%3A%2F%2Flive-azure.test.net%3A7880", "sandbox": true, "insecure": true}
0:00:00.209906420    39 0x648c4e374f90 WARN                 default gstsfelement.c:97:gst_sf_create_audio_template_caps: format 0x120000: 'AVR (Audio Visual Research)' is not mapped
0:00:00.209940021    39 0x648c4e374f90 WARN                 default gstsfelement.c:97:gst_sf_create_audio_template_caps: format 0x180000: 'CAF (Apple Core Audio File)' is not mapped
0:00:00.209947321    39 0x648c4e374f90 WARN                 default gstsfelement.c:97:gst_sf_create_audio_template_caps: format 0x100000: 'HTK (HMM Tool Kit)' is not mapped
0:00:00.209955421    39 0x648c4e374f90 WARN                 default gstsfelement.c:97:gst_sf_create_audio_template_caps: format 0xc0000: 'MAT4 (GNU Octave 2.0 / Matlab 4.2)' is not mapped
0:00:00.209961121    39 0x648c4e374f90 WARN                 default gstsfelement.c:97:gst_sf_create_audio_template_caps: format 0xd0000: 'MAT5 (GNU Octave 2.1 / Matlab 5.0)' is not mapped
0:00:00.209967621    39 0x648c4e374f90 WARN                 default gstsfelement.c:97:gst_sf_create_audio_template_caps: format 0x210000: 'MPC (Akai MPC 2k)' is not mapped
0:00:00.209974421    39 0x648c4e374f90 WARN                 default gstsfelement.c:97:gst_sf_create_audio_template_caps: format 0x230000: 'MPEG-1/2 Audio' is not mapped
0:00:00.209982322    39 0x648c4e374f90 WARN                 default gstsfelement.c:97:gst_sf_create_audio_template_caps: format 0xe0000: 'PVF (Portable Voice Format)' is not mapped
0:00:00.209990122    39 0x648c4e374f90 WARN                 default gstsfelement.c:97:gst_sf_create_audio_template_caps: format 0x160000: 'SD2 (Sound Designer II)' is not mapped
0:00:00.209999822    39 0x648c4e374f90 WARN                 default gstsfelement.c:97:gst_sf_create_audio_template_caps: format 0x190000: 'WVE (Psion Series 3)' is not mapped
0:00:00.339319130    39 0x648c4e374f90 WARN              cudaloader gstcudaloader.c:169:gst_cuda_load_library: Could not open library libcuda.so.1, libcuda.so.1: cannot open shared object file: No such file or directory
0:00:00.339341030    39 0x648c4e374f90 WARN                 nvcodec plugin.c:94:plugin_init: Failed to load cuda library
2024-09-17T13:29:34.951Z        DEBUG   egress  gstreamer/bin.go:64     adding src audio to pipeline    {"nodeID": "NE_Rm6y42thEsMs", "handlerID": "EGH_HQTgqiYgufrQ", "clusterID": "", "egressID": "EG_VVWiJwdLCZii"}
2024-09-17T13:29:34.952Z        INFO    egress  source/web.go:275       chrome: START_RECORDING {"nodeID": "NE_Rm6y42thEsMs", "handlerID": "EGH_HQTgqiYgufrQ", "clusterID": "", "egressID": "EG_VVWiJwdLCZii"}
2024-09-17T13:29:34.953Z        DEBUG   egress  gstreamer/bin.go:64     adding src video to pipeline    {"nodeID": "NE_Rm6y42thEsMs", "handlerID": "EGH_HQTgqiYgufrQ", "clusterID": "", "egressID": "EG_VVWiJwdLCZii"}
2024-09-17T13:29:34.954Z        DEBUG   egress  gstreamer/bin.go:70     adding sink iUhZ3K3L53XA to stream      {"nodeID": "NE_Rm6y42thEsMs", "handlerID": "EGH_HQTgqiYgufrQ", "clusterID": "", "egressID": "EG_VVWiJwdLCZii"}
2024-09-17T13:29:34.954Z        DEBUG   egress  gstreamer/bin.go:70     adding sink NYro9vpwUnrj to stream      {"nodeID": "NE_Rm6y42thEsMs", "handlerID": "EGH_HQTgqiYgufrQ", "clusterID": "", "egressID": "EG_VVWiJwdLCZii"}
2024-09-17T13:29:34.954Z        DEBUG   egress  gstreamer/bin.go:70     adding sink stream to pipeline  {"nodeID": "NE_Rm6y42thEsMs", "handlerID": "EGH_HQTgqiYgufrQ", "clusterID": "", "egressID": "EG_VVWiJwdLCZii"}
2024-09-17T13:29:34.958Z        DEBUG   egress  pipeline/controller.go:211      waiting for start signal        {"nodeID": "NE_Rm6y42thEsMs", "handlerID": "EGH_HQTgqiYgufrQ", "clusterID": "", "egressID": "EG_VVWiJwdLCZii"}
2024-09-17T13:29:34.958Z        DEBUG   egress  gstreamer/state.go:75   pipeline state building -> starting     {"nodeID": "NE_Rm6y42thEsMs", "handlerID": "EGH_HQTgqiYgufrQ", "clusterID": "", "egressID": "EG_VVWiJwdLCZii"}
2024-09-17T13:29:34.964Z        DEBUG   egress  gstreamer/state.go:75   pipeline state starting -> running      {"nodeID": "NE_Rm6y42thEsMs", "handlerID": "EGH_HQTgqiYgufrQ", "clusterID": "", "egressID": "EG_VVWiJwdLCZii"}
2024-09-17T13:29:34.965Z        INFO    egress  pipeline/watch.go:235   pipeline playing        {"nodeID": "NE_Rm6y42thEsMs", "handlerID": "EGH_HQTgqiYgufrQ", "clusterID": "", "egressID": "EG_VVWiJwdLCZii"}
2024-09-17T13:29:36.349Z        DEBUG   egress  [gst fixme] gst_rtmp_connection_handle_protocol_control: set peer bandwidth: 10000000, 2        {"nodeID": "NE_Rm6y42thEsMs", "handlerID": "EGH_HQTgqiYgufrQ", "clusterID": "", "egressID": "EG_VVWiJwdLCZii", "caller": "../gst/rtmp2/rtmp/rtmpconnection.c:867"}
2024-09-17T13:29:36.350Z        DEBUG   egress  [gst warning] parse_ecma_array: Expected array with 1 elements, but read 2      {"nodeID": "NE_Rm6y42thEsMs", "handlerID": "EGH_HQTgqiYgufrQ", "clusterID": "", "egressID": "EG_VVWiJwdLCZii", "caller": "../gst/rtmp2/rtmp/amf.c:802"}
2024-09-17T13:29:36.363Z        DEBUG   egress  [gst fixme] gst_rtmp_connection_handle_protocol_control: set peer bandwidth: 5000000, 2 {"nodeID": "NE_Rm6y42thEsMs", "handlerID": "EGH_HQTgqiYgufrQ", "clusterID": "", "egressID": "EG_VVWiJwdLCZii", "caller": "../gst/rtmp2/rtmp/rtmpconnection.c:867"}
2024-09-17T13:29:36.398Z        DEBUG   egress  [gst warning] gst_rtmp_connection_handle_cm: Server sent response "_result" without transaction {"nodeID": "NE_Rm6y42thEsMs", "handlerID": "EGH_HQTgqiYgufrQ", "clusterID": "", "egressID": "EG_VVWiJwdLCZii", "caller": "../gst/rtmp2/rtmp/rtmpconnection.c:1074"}
2024-09-17T13:29:36.398Z        DEBUG   egress  [gst warning] gst_rtmp_connection_handle_cm: Server sent response "_result" without transaction {"nodeID": "NE_Rm6y42thEsMs", "handlerID": "EGH_HQTgqiYgufrQ", "clusterID": "", "egressID": "EG_VVWiJwdLCZii", "caller": "../gst/rtmp2/rtmp/rtmpconnection.c:1074"}

[cagrisungur]

Any update on this?

Same problem is going on.