Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Connect to RDP session causes crash on hackintosh with RX580 #21

Open
laggykiller opened this issue Sep 1, 2023 · 8 comments · Fixed by team-unstablers/ulalaca-xrdp#13
Assignees

Comments

@laggykiller
Copy link

laggykiller commented Sep 1, 2023

I am testing on a hackintosh with i7-12700F, RX580, running Ventura. xrdp immediately crash after logging in the session. The session screen remains blue. On rare occasion I can move the mouse cursor for a bit before the crash.

I installed following this guide: https://teamunstablers.notion.site/xrdp-Ulalaca-Getting-started-f82b0c55f0b540a6ac277cc5902361b1

The specs and the EFI is given here: https://github.com/laggykiller/Hackintosh_MSI_B660M-A_WIFI_DDR4

Note that I also have a RTX2070 inserted into motherboard, but disabled by EFI. Not sure if this is the culprit?

Here is the crash report:

xrdp log
-------------------------------------
Translated Report (Full Report Below)
-------------------------------------

Process:               xrdp [682]
Path:                  /usr/local/Cellar/xrdp-git/HEAD-fc31b39/sbin/xrdp
Identifier:            xrdp
Version:               ???
Code Type:             X86-64 (Native)
Parent Process:        xrdp [292]
Responsible:           xrdp [292]
User ID:               0

Date/Time:             2023-09-01 17:12:10.9127 +0800
OS Version:            macOS 13.5.1 (22G90)
Report Version:        12
Anonymous UUID:        D177EA49-7E28-DE3D-E7E5-EBB64D6AA5B9


Time Awake Since Boot: 110 seconds

System Integrity Protection: enabled

Crashed Thread:        3

Exception Type:        EXC_ARITHMETIC (SIGFPE)
Exception Codes:       0x0000000000000001, 0x0000000000000000

Termination Reason:    Namespace SIGNAL, Code 8 Floating point exception: 
8
Terminating Process:   exc handler [682]

Application Specific Information:
crashed on child side of fork pre-exec


Thread 0::  Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	    0x7ff8160e22ae poll + 10
1   libcommon.0.dylib             	       0x1059884ca g_fd_can_read + 
38
2   xrdp                          	       0x1058f7531 
xrdp_process_main_loop + 687
3   xrdp                          	       0x1058ec5fa 
xrdp_listen_main_loop + 560
4   xrdp                          	       0x1058e6e85 main + 2213
5   dyld                          	    0x7ff815dc041f start + 1903

Thread 1:
0   libsystem_kernel.dylib        	    0x7ff8160e22ae poll + 10
1   libulalaca.0.dylib            	       0x105b357fc 
UnixSocketBase::poll(short, int) + 52
2   libulalaca.0.dylib            	       0x105b37535 
IPCConnection::workerLoop() + 141
3   libulalaca.0.dylib            	       0x105b387a7 void* 
std::__1::__thread_proxy[abi:v15006]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, 
std::__1::default_delete<std::__1::__thread_struct>>, void 
(IPCConnection::*)(), IPCConnection*>>(void*) + 62
4   libsystem_pthread.dylib       	    0x7ff81611a1d3 _pthread_start 
+ 125
5   libsystem_pthread.dylib       	    0x7ff816115bd3 thread_start + 
15

Thread 2:
0   libsystem_kernel.dylib        	    0x7ff8160de0fe __psynch_cvwait 
+ 10
1   libsystem_pthread.dylib       	    0x7ff81611a758 
_pthread_cond_wait + 1242
2   libc++.1.dylib                	    0x7ff8160571e2 
std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) 
+ 18
3   libc++.1.dylib                	    0x7ff816057a51 
std::__1::__assoc_sub_state::__sub_wait(std::__1::unique_lock<std::__1::mutex>&) 
+ 45
4   libulalaca.0.dylib            	       0x105b3a517 
std::__1::__assoc_state<std::__1::unique_ptr<unsigned char, 
std::__1::default_delete<unsigned char>>>::move() + 51
5   libulalaca.0.dylib            	       0x105b39c34 
std::__1::future<std::__1::unique_ptr<unsigned char, 
std::__1::default_delete<unsigned char>>>::get() + 28
6   libulalaca.0.dylib            	       0x105b37981 
std::__1::unique_ptr<ULIPCHeader, std::__1::function<void (void*)>> 
IPCConnection::read<ULIPCHeader>(unsigned long) + 163
7   libulalaca.0.dylib            	       0x105b3789b 
IPCConnection::nextHeader() + 45
8   libulalaca.0.dylib            	       0x105b3a841 
ProjectorClient::mainLoop() + 99
9   libulalaca.0.dylib            	       0x105b3b6dd void* 
std::__1::__thread_proxy[abi:v15006]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, 
std::__1::default_delete<std::__1::__thread_struct>>, void 
(ProjectorClient::*)(), ProjectorClient*>>(void*) + 62
10  libsystem_pthread.dylib       	    0x7ff81611a1d3 _pthread_start 
+ 125
11  libsystem_pthread.dylib       	    0x7ff816115bd3 thread_start + 
15

Thread 3 Crashed:
0   libulalaca.0.dylib            	       0x105b3c447 
XrdpUlalacaPrivate::createCopyRects(std::__1::vector<ULIPCRect, 
std::__1::allocator<ULIPCRect>>&, int) const + 117
1   libulalaca.0.dylib            	       0x105b3c1bd 
XrdpUlalacaPrivate::updateThreadLoop() + 1095
2   libulalaca.0.dylib            	       0x105b3e071 void* 
std::__1::__thread_proxy[abi:v15006]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, 
std::__1::default_delete<std::__1::__thread_struct>>, void 
(XrdpUlalacaPrivate::*)(), XrdpUlalacaPrivate*>>(void*) + 62
3   libsystem_pthread.dylib       	    0x7ff81611a1d3 _pthread_start 
+ 125
4   libsystem_pthread.dylib       	    0x7ff816115bd3 thread_start + 
15


Thread 3 crashed with X86 Thread State (64-bit):
  rax: 0x00000000004c2600  rbx: 0x0000000000000000  rcx: 
0x0000000000000000  rdx: 0x0000000000000000
  rdi: 0x000000010596b080  rsi: 0x0000000000000003  rbp: 
0x000070000f19ee80  rsp: 0x000070000f19edd0
   r8: 0x0000000000000ffe   r9: 0x00007f9d4bf00000  r10: 
0x0000000000000192  r11: 0x00007f9d4bf04080
  r12: 0x000070000f19eeb0  r13: 0x00007f9d4bf05630  r14: 
0x00007f9d4c853a00  r15: 0x00007f9d4bf05648
  rip: 0x0000000105b3c447  rfl: 0x0000000000010206  cr2: 
0x0000000112337000
  
Logical CPU:     4
Error Code:      0x00000000 
Trap Number:     0

Thread 3 instruction stream:
  cb 48 89 55 c8 49 89 f6-49 89 fc bf 30 00 00 00  .H.U.I..I...0...
  e8 44 30 00 00 49 89 c5-0f 57 c0 0f 11 40 08 48  [email protected]
  8b 05 fb 4c 00 00 48 83-c0 10 49 89 45 00 4d 8d  ...L..H...I.E.M.
  7d 18 41 0f 11 45 18 49-c7 45 28 00 00 00 00 4c  }.A..E.I.E(....L
  89 bd 70 ff ff ff 4c 89-ad 78 ff ff ff 41 0f bf  ..p...L..x...A..
  4e 19 41 0f bf 46 1b 0f-af c1 89 d9 0f af cb 99  N.A..F..........
 [f7]f9 48 63 f0 4c 89 ff-e8 fa 03 00 00 85 db 4c  ..Hc.L.........L	
<==
  89 7d a0 0f 84 3e 03 00-00 4c 89 6d 88 4c 89 a5  .}...>...L.m.L..
  68 ff ff ff 41 0f bf 46-19 89 45 90 0f 57 c0 f3  h...A..F..E..W..
  0f 2a c0 89 5d ac f3 0f-2a d3 f3 0f 5e c2 66 0f  .*..]...*...^.f.
  3a 0a c0 0a 41 0f bf 46-1b 89 45 98 f3 0f 2a c8  :...A..F..E...*.
  f3 0f 2c c8 f3 0f 11 55-b0 f3 0f 5e ca 0f 57 c0  ..,....U...^..W.

Binary Images:
       0x1058e1000 -        0x10590cfff xrdp (*) 
<d4030975-c4cf-38b1-9f26-5b171eeec96a> 
/usr/local/Cellar/xrdp-git/HEAD-fc31b39/sbin/xrdp
       0x105982000 -        0x105991fff libcommon.0.dylib (*) 
<4107ae6e-3ccd-3cbe-a0a2-3d9f422616dd> 
/usr/local/Cellar/xrdp-git/HEAD-fc31b39/lib/xrdp/libcommon.0.dylib
       0x1059a2000 -        0x1059a9fff libipm.0.dylib (*) 
<191ffd40-c22c-3db9-9bf7-1342f1adf53a> 
/usr/local/Cellar/xrdp-git/HEAD-fc31b39/lib/xrdp/libipm.0.dylib
       0x1059e7000 -        0x105a0afff libxrdp.0.dylib (*) 
<eab43044-1201-3936-a699-bf70cafb224b> 
/usr/local/Cellar/xrdp-git/HEAD-fc31b39/lib/xrdp/libxrdp.0.dylib
       0x105aa5000 -        0x105b00fff libssl.3.dylib (*) 
<ffa75e91-86e1-3f3e-8bdf-37ed77cc6176> 
/usr/local/Cellar/openssl@3/3.1.2/lib/libssl.3.dylib
       0x105f5a000 -        0x106279fff libcrypto.3.dylib (*) 
<1fcd44a2-e972-3d3d-95d1-4b7bf8459e95> 
/usr/local/Cellar/openssl@3/3.1.2/lib/libcrypto.3.dylib
       0x1059b6000 -        0x1059bdfff libtoml.1.dylib (*) 
<c9819b99-ddd0-3ccd-ac46-2ef287b05017> 
/usr/local/Cellar/xrdp-git/HEAD-fc31b39/lib/xrdp/libtoml.1.dylib
       0x105b31000 -        0x105b40fff libulalaca.0.dylib (*) 
<6c6f50a5-c545-3c52-bb99-e7d8df016668> 
/usr/local/Cellar/xrdp-git/HEAD-fc31b39/lib/xrdp/libulalaca.0.dylib
    0x7ff8160da000 -     0x7ff816113fff libsystem_kernel.dylib (*) 
<642737ce-21cf-3d87-a7b6-7eea2c080a7e> 
/usr/lib/system/libsystem_kernel.dylib
    0x7ff815dba000 -     0x7ff815e525ef dyld (*) 
<31e1c182-e611-3883-97a3-4334a21c90bd> /usr/lib/dyld
    0x7ff816114000 -     0x7ff81611ffff libsystem_pthread.dylib (*) 
<3a3fa85c-f1fe-336c-bf68-924fc947ade3> 
/usr/lib/system/libsystem_pthread.dylib
    0x7ff816049000 -     0x7ff8160c4ff7 libc++.1.dylib (*) 
<8b258faf-4392-3385-a019-d80f49c5af31> /usr/lib/libc++.1.dylib

External Modification Summary:
  Calls made by other processes targeting this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by all processes on this machine:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0

VM Region Summary:
ReadOnly portion of Libraries: Total=181.2M resident=0K(0%) 
swapped_out_or_unallocated=181.2M(100%)
Writable regions: Total=1.1G written=0K(0%) resident=0K(0%) 
swapped_out=0K(0%) unallocated=1.1G(100%)

                                VIRTUAL   REGION 
REGION TYPE                        SIZE    COUNT (non-coalesced) 
===========                     =======  ======= 
Activity Tracing                   256K        1 
Kernel Alloc Once                    8K        1 
MALLOC                           278.3M       31 
MALLOC guard page                   24K        6 
MALLOC_MEDIUM (reserved)         480.0M        4         reserved VM 
address space (unallocated)
MALLOC_NANO (reserved)           384.0M        1         reserved VM 
address space (unallocated)
STACK GUARD                       56.0M        4 
Stack                             9752K        4 
VM_ALLOCATE                          4K        1 
__DATA                             344K       45 
__DATA_CONST                       726K       41 
__DATA_DIRTY                        67K       24 
__LINKEDIT                       171.9M       10 
__OBJC_RO                         66.3M        1 
__OBJC_RW                         2013K        2 
__TEXT                            9488K       49 
dyld private memory                260K        2 
shared memory                       20K        3 
===========                     =======  ======= 
TOTAL                              1.4G      230 
TOTAL, minus reserved VM space   595.0M      230 



-----------
Full Report
-----------

{"app_name":"xrdp","timestamp":"2023-09-01 17:12:10.00 
+0800","app_version":"","slice_uuid":"d4030975-c4cf-38b1-9f26-5b171eeec96a","build_version":"","platform":1,"share_with_app_devs":0,"is_first_party":1,"bug_type":"309","os_version":"macOS 
13.5.1 
(22G90)","roots_installed":0,"incident_id":"F16EB652-61D0-4651-87C4-4C378A7964C3","name":"xrdp"}
{
  "uptime" : 110,
  "procRole" : "Unspecified",
  "version" : 2,
  "userID" : 0,
  "deployVersion" : 210,
  "modelCode" : "MacPro7,1",
  "coalitionID" : 283,
  "osVersion" : {
    "train" : "macOS 13.5.1",
    "build" : "22G90",
    "releaseType" : "User"
  },
  "captureTime" : "2023-09-01 17:12:10.9127 +0800",
  "incident" : "F16EB652-61D0-4651-87C4-4C378A7964C3",
  "pid" : 682,
  "cpuType" : "X86-64",
  "roots_installed" : 0,
  "bug_type" : "309",
  "procLaunch" : "2023-09-01 17:12:07.0412 +0800",
  "procStartAbsTime" : 110891407191,
  "procExitAbsTime" : 114762014961,
  "procName" : "xrdp",
  "procPath" : "\/usr\/local\/Cellar\/xrdp-git\/HEAD-fc31b39\/sbin\/xrdp",
  "parentProc" : "xrdp",
  "parentPid" : 292,
  "coalitionName" : "homebrew.mxcl.xrdp-git",
  "crashReporterKey" : "D177EA49-7E28-DE3D-E7E5-EBB64D6AA5B9",
  "responsiblePid" : 292,
  "responsibleProc" : "xrdp",
  "codeSigningID" : "",
  "codeSigningTeamID" : "",
  "codeSigningValidationCategory" : 0,
  "codeSigningTrustLevel" : 0,
  "sip" : "enabled",
  "exception" : {"codes":"0x0000000000000001, 
0x0000000000000000","rawCodes":[1,0],"type":"EXC_ARITHMETIC","signal":"SIGFPE"},
  "termination" : 
{"flags":0,"code":8,"namespace":"SIGNAL","indicator":"Floating point 
exception: 8","byProc":"exc handler","byPid":682},
  "asi" : {"libsystem_c.dylib":["crashed on child side of fork 
pre-exec"]},
  "extMods" : 
{"caller":{"thread_create":0,"thread_set_state":0,"task_for_pid":0},"system":{"thread_create":0,"thread_set_state":0,"task_for_pid":0},"targeted":{"thread_create":0,"thread_set_state":0,"task_for_pid":0},"warnings":0},
  "faultingThread" : 3,
  "threads" : 
[{"id":6156,"queue":"com.apple.main-thread","frames":[{"imageOffset":33454,"symbol":"poll","symbolLocation":10,"imageIndex":8},{"imageOffset":25802,"symbol":"g_fd_can_read","symbolLocation":38,"imageIndex":1},{"imageOffset":91441,"symbol":"xrdp_process_main_loop","symbolLocation":687,"imageIndex":0},{"imageOffset":46586,"symbol":"xrdp_listen_main_loop","symbolLocation":560,"imageIndex":0},{"imageOffset":24197,"symbol":"main","symbolLocation":2213,"imageIndex":0},{"imageOffset":25631,"symbol":"start","symbolLocation":1903,"imageIndex":9}]},{"id":6173,"frames":[{"imageOffset":33454,"symbol":"poll","symbolLocation":10,"imageIndex":8},{"imageOffset":18428,"symbol":"UnixSocketBase::poll(short, 
int)","symbolLocation":52,"imageIndex":7},{"imageOffset":25909,"symbol":"IPCConnection::workerLoop()","symbolLocation":141,"imageIndex":7},{"imageOffset":30631,"symbol":"void* 
std::__1::__thread_proxy[abi:v15006]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, 
std::__1::default_delete<std::__1::__thread_struct>>, void 
(IPCConnection::*)(), 
IPCConnection*>>(void*)","symbolLocation":62,"imageIndex":7},{"imageOffset":25043,"symbol":"_pthread_start","symbolLocation":125,"imageIndex":10},{"imageOffset":7123,"symbol":"thread_start","symbolLocation":15,"imageIndex":10}]},{"id":6174,"frames":[{"imageOffset":16638,"symbol":"__psynch_cvwait","symbolLocation":10,"imageIndex":8},{"imageOffset":26456,"symbol":"_pthread_cond_wait","symbolLocation":1242,"imageIndex":10},{"imageOffset":57826,"symbol":"std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&)","symbolLocation":18,"imageIndex":11},{"imageOffset":59985,"symbol":"std::__1::__assoc_sub_state::__sub_wait(std::__1::unique_lock<std::__1::mutex>&)","symbolLocation":45,"imageIndex":11},{"imageOffset":38167,"symbol":"std::__1::__assoc_state<std::__1::unique_ptr<unsigned 
char, std::__1::default_delete<unsigned 
char>>>::move()","symbolLocation":51,"imageIndex":7},{"imageOffset":35892,"symbol":"std::__1::future<std::__1::unique_ptr<unsigned 
char, std::__1::default_delete<unsigned 
char>>>::get()","symbolLocation":28,"imageIndex":7},{"imageOffset":27009,"symbol":"std::__1::unique_ptr<ULIPCHeader, 
std::__1::function<void (void*)>> 
IPCConnection::read<ULIPCHeader>(unsigned 
long)","symbolLocation":163,"imageIndex":7},{"imageOffset":26779,"symbol":"IPCConnection::nextHeader()","symbolLocation":45,"imageIndex":7},{"imageOffset":38977,"symbol":"ProjectorClient::mainLoop()","symbolLocation":99,"imageIndex":7},{"imageOffset":42717,"symbol":"void* 
std::__1::__thread_proxy[abi:v15006]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, 
std::__1::default_delete<std::__1::__thread_struct>>, void 
(ProjectorClient::*)(), 
ProjectorClient*>>(void*)","symbolLocation":62,"imageIndex":7},{"imageOffset":25043,"symbol":"_pthread_start","symbolLocation":125,"imageIndex":10},{"imageOffset":7123,"symbol":"thread_start","symbolLocation":15,"imageIndex":10}]},{"triggered":true,"id":6175,"instructionState":{"instructionStream":{"bytes":[203,72,137,85,200,73,137,246,73,137,252,191,48,0,0,0,232,68,48,0,0,73,137,197,15,87,192,15,17,64,8,72,139,5,251,76,0,0,72,131,192,16,73,137,69,0,77,141,125,24,65,15,17,69,24,73,199,69,40,0,0,0,0,76,137,189,112,255,255,255,76,137,173,120,255,255,255,65,15,191,78,25,65,15,191,70,27,15,175,193,137,217,15,175,203,153,247,249,72,99,240,76,137,255,232,250,3,0,0,133,219,76,137,125,160,15,132,62,3,0,0,76,137,109,136,76,137,165,104,255,255,255,65,15,191,70,25,137,69,144,15,87,192,243,15,42,192,137,93,172,243,15,42,211,243,15,94,194,102,15,58,10,192,10,65,15,191,70,27,137,69,152,243,15,42,200,243,15,44,200,243,15,17,85,176,243,15,94,202,15,87,192],"offset":96}},"threadState":{"r13":{"value":140313560634928},"rax":{"value":4990464},"rflags":{"value":66054},"cpu":{"value":4},"r14":{"value":140313570392576},"rsi":{"value":3},"r8":{"value":4094},"cr2":{"value":4600328192},"rdx":{"value":0},"r10":{"value":402},"r9":{"value":140313560612864},"r15":{"value":140313560634952},"rbx":{"value":0},"trap":{"value":0},"err":{"value":0},"r11":{"value":140313560629376},"rip":{"value":4390634567,"matchesCrashFrame":1},"rbp":{"value":123145555668608},"rsp":{"value":123145555668432},"r12":{"value":123145555668656},"rcx":{"value":0},"flavor":"x86_THREAD_STATE","rdi":{"value":4388728960}},"frames":[{"imageOffset":46151,"symbol":"XrdpUlalacaPrivate::createCopyRects(std::__1::vector<ULIPCRect, 
std::__1::allocator<ULIPCRect>>&, int) 
const","symbolLocation":117,"imageIndex":7},{"imageOffset":45501,"symbol":"XrdpUlalacaPrivate::updateThreadLoop()","symbolLocation":1095,"imageIndex":7},{"imageOffset":53361,"symbol":"void* 
std::__1::__thread_proxy[abi:v15006]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, 
std::__1::default_delete<std::__1::__thread_struct>>, void 
(XrdpUlalacaPrivate::*)(), 
XrdpUlalacaPrivate*>>(void*)","symbolLocation":62,"imageIndex":7},{"imageOffset":25043,"symbol":"_pthread_start","symbolLocation":125,"imageIndex":10},{"imageOffset":7123,"symbol":"thread_start","symbolLocation":15,"imageIndex":10}]}],
  "usedImages" : [
  {
    "source" : "P",
    "arch" : "x86_64",
    "base" : 4388163584,
    "size" : 180224,
    "uuid" : "d4030975-c4cf-38b1-9f26-5b171eeec96a",
    "path" : "\/usr\/local\/Cellar\/xrdp-git\/HEAD-fc31b39\/sbin\/xrdp",
    "name" : "xrdp"
  },
  {
    "source" : "P",
    "arch" : "x86_64",
    "base" : 4388823040,
    "size" : 65536,
    "uuid" : "4107ae6e-3ccd-3cbe-a0a2-3d9f422616dd",
    "path" : 
"\/usr\/local\/Cellar\/xrdp-git\/HEAD-fc31b39\/lib\/xrdp\/libcommon.0.dylib",
    "name" : "libcommon.0.dylib"
  },
  {
    "source" : "P",
    "arch" : "x86_64",
    "base" : 4388954112,
    "size" : 32768,
    "uuid" : "191ffd40-c22c-3db9-9bf7-1342f1adf53a",
    "path" : 
"\/usr\/local\/Cellar\/xrdp-git\/HEAD-fc31b39\/lib\/xrdp\/libipm.0.dylib",
    "name" : "libipm.0.dylib"
  },
  {
    "source" : "P",
    "arch" : "x86_64",
    "base" : 4389236736,
    "size" : 147456,
    "uuid" : "eab43044-1201-3936-a699-bf70cafb224b",
    "path" : 
"\/usr\/local\/Cellar\/xrdp-git\/HEAD-fc31b39\/lib\/xrdp\/libxrdp.0.dylib",
    "name" : "libxrdp.0.dylib"
  },
  {
    "source" : "P",
    "arch" : "x86_64",
    "base" : 4390014976,
    "size" : 376832,
    "uuid" : "ffa75e91-86e1-3f3e-8bdf-37ed77cc6176",
    "path" : 
"\/usr\/local\/Cellar\/openssl@3\/3.1.2\/lib\/libssl.3.dylib",
    "name" : "libssl.3.dylib"
  },
  {
    "source" : "P",
    "arch" : "x86_64",
    "base" : 4394950656,
    "size" : 3276800,
    "uuid" : "1fcd44a2-e972-3d3d-95d1-4b7bf8459e95",
    "path" : 
"\/usr\/local\/Cellar\/openssl@3\/3.1.2\/lib\/libcrypto.3.dylib",
    "name" : "libcrypto.3.dylib"
  },
  {
    "source" : "P",
    "arch" : "x86_64",
    "base" : 4389036032,
    "size" : 32768,
    "uuid" : "c9819b99-ddd0-3ccd-ac46-2ef287b05017",
    "path" : 
"\/usr\/local\/Cellar\/xrdp-git\/HEAD-fc31b39\/lib\/xrdp\/libtoml.1.dylib",
    "name" : "libtoml.1.dylib"
  },
  {
    "source" : "P",
    "arch" : "x86_64",
    "base" : 4390588416,
    "size" : 65536,
    "uuid" : "6c6f50a5-c545-3c52-bb99-e7d8df016668",
    "path" : 
"\/usr\/local\/Cellar\/xrdp-git\/HEAD-fc31b39\/lib\/xrdp\/libulalaca.0.dylib",
    "name" : "libulalaca.0.dylib"
  },
  {
    "source" : "P",
    "arch" : "x86_64",
    "base" : 140703498608640,
    "size" : 237568,
    "uuid" : "642737ce-21cf-3d87-a7b6-7eea2c080a7e",
    "path" : "\/usr\/lib\/system\/libsystem_kernel.dylib",
    "name" : "libsystem_kernel.dylib"
  },
  {
    "source" : "P",
    "arch" : "x86_64",
    "base" : 140703495331840,
    "size" : 624112,
    "uuid" : "31e1c182-e611-3883-97a3-4334a21c90bd",
    "path" : "\/usr\/lib\/dyld",
    "name" : "dyld"
  },
  {
    "source" : "P",
    "arch" : "x86_64",
    "base" : 140703498846208,
    "size" : 49152,
    "uuid" : "3a3fa85c-f1fe-336c-bf68-924fc947ade3",
    "path" : "\/usr\/lib\/system\/libsystem_pthread.dylib",
    "name" : "libsystem_pthread.dylib"
  },
  {
    "source" : "P",
    "arch" : "x86_64",
    "base" : 140703498014720,
    "size" : 507896,
    "uuid" : "8b258faf-4392-3385-a019-d80f49c5af31",
    "path" : "\/usr\/lib\/libc++.1.dylib",
    "name" : "libc++.1.dylib"
  }
],
  "sharedCache" : {
  "base" : 140703494709248,
  "size" : 21474836480,
  "uuid" : "08a60487-01b2-3cd1-a713-ab9ca254667b"
},
  "vmSummary" : "ReadOnly portion of Libraries: Total=181.2M 
resident=0K(0%) swapped_out_or_unallocated=181.2M(100%)\nWritable regions: 
Total=1.1G written=0K(0%) resident=0K(0%) swapped_out=0K(0%) 
unallocated=1.1G(100%)\n\n                                VIRTUAL   REGION 
\nREGION TYPE                        SIZE    COUNT (non-coalesced) 
\n===========                     =======  ======= \nActivity Tracing                   
256K        1 \nKernel Alloc Once                    8K        1 \nMALLOC                           
278.3M       31 \nMALLOC guard page                   24K        6 
\nMALLOC_MEDIUM (reserved)         480.0M        4         reserved VM 
address space (unallocated)\nMALLOC_NANO (reserved)           384.0M        
1         reserved VM address space (unallocated)\nSTACK GUARD                       
56.0M        4 \nStack                             9752K        4 
\nVM_ALLOCATE                          4K        1 \n__DATA                             
344K       45 \n__DATA_CONST                       726K       41 
\n__DATA_DIRTY                        67K       24 \n__LINKEDIT                       
171.9M       10 \n__OBJC_RO                         66.3M        1 
\n__OBJC_RW                         2013K        2 \n__TEXT                            
9488K       49 \ndyld private memory                260K        2 \nshared 
memory                       20K        3 \n===========                     
=======  ======= \nTOTAL                              1.4G      230 
\nTOTAL, minus reserved VM space   595.0M      230 \n",
  "legacyInfo" : {
  "threadTriggered" : {

  }
},
  "logWritingSignature" : "c6d121640235f01fffb2e2236b6f06a9b9fad9e9"
}

Btw I also tested with Sonoma. After logging in, xrdp does not crash, but the rdp screen stays at blue screen. The mouse cursor is teleported to left upper corner, and I cannot move it in the RDP session. However if I left click in RDP session, I can see that it went through and responded to my left click.

@laggykiller
Copy link
Author

I have captured log from running sudo xrdp -n

log.txt

The crash was due to floating point exception

Seems like there is no error before the crash.

@laggykiller
Copy link
Author

laggykiller commented Sep 2, 2023

In https://github.com/team-unstablers/ulalaca-xrdp/blob/c8c31c86219a432b2df8b8cb774d3e96be3bb6b7/XrdpUlalacaPrivate.cpp#L96

std::shared_ptr<std::vector<ULIPCRect>> XrdpUlalacaPrivate::createCopyRects(
    std::vector<ULIPCRect> &dirtyRects,
    int rectSize
) const {
    auto blocks = std::make_shared<std::vector<ULIPCRect>>();
    blocks->reserve((_sessionSize.width * _sessionSize.height) / (rectSize * rectSize));

    if (rectSize == RECT_SIZE_BYPASS_CREATE) {
        std::copy(dirtyRects.begin(), dirtyRects.end(), std::back_insert_iterator(*blocks));
        return std::move(blocks);
    }

    int mapWidth  = std::ceil((float) _sessionSize.width / (float) rectSize);
    int mapHeight = std::ceil((float) _sessionSize.height / (float) rectSize);

RECT_SIZE_BYPASS_CREATE is defined as 0 (https://github.com/team-unstablers/ulalaca-xrdp/blob/c8c31c86219a432b2df8b8cb774d3e96be3bb6b7/XrdpUlalacaPrivate.hpp#L47)

I guess the exception occurs when rectSize is 0. Seems like there is a condition check if (rectSize == RECT_SIZE_BYPASS_CREATE), which should be triggered when rectSize equal to 0, but it occurs before blocks->reserve((_sessionSize.width * _sessionSize.height) / (rectSize * rectSize));, so division of 0 occured? However, I am not sure how much to reserve() for blocks when rectSize is 0, so I am not sure how should the code be fixed...

Also, note that rectSize is set to 0 in the first place, possibly due to https://github.com/team-unstablers/ulalaca-xrdp/blob/c8c31c86219a432b2df8b8cb774d3e96be3bb6b7/XrdpUlalacaPrivate.cpp#L83:

int XrdpUlalacaPrivate::decideCopyRectSize() const {
    if (isRFXCodec()) {
        return 64;
    }

    if (isNSCodec() || isH264Codec() || isGFXH264Codec()) {
        // ??? FIXME
        return RECT_SIZE_BYPASS_CREATE;
    }

    return RECT_SIZE_BYPASS_CREATE;
}

The isRFXCodec() possibly returned False, so it fallback to returning 0 (RECT_SIZE_BYPASS_CREATE).

@laggykiller
Copy link
Author

I compiled and run as x86_64 on M1 macmini using Rosetta. floating point exception also occured at XrdpUlalacaPrivate::createCopyRects:
stdout.txt
crash_report.txt

@unstabler
Copy link
Collaborator

RECT_SIZE_BYPASS_CREATE is defined as 0 (https://github.com/team-unstablers/ulalaca-xrdp/blob/c8c31c86219a432b2df8b8cb774d3e96be3bb6b7/XrdpUlalacaPrivate.hpp#L47)

I guess the exception occurs when rectSize is 0. Seems like there is a condition check if (rectSize == RECT_SIZE_BYPASS_CREATE), which should be triggered when rectSize equal to 0, but it occurs before blocks->reserve((_sessionSize.width * _sessionSize.height) / (rectSize * rectSize));, so division of 0 occured? However, I am not sure how much to reserve() for blocks when rectSize is 0, so I am not sure how should the code be fixed...

Sorry for the late reply.
I will fix it as soon as I finish work today.

@jhay06
Copy link

jhay06 commented Sep 5, 2023

i have same issue , hope for a fix for this , thank you

@laggykiller
Copy link
Author

Currently I don't have access to that hackintosh with RX580, but just now I have done testing on another hackintosh with Intel Coffee Lake CPU and internal GPU (Specs: https://github.com/laggykiller/hackintosh-gmk-nucbox2). Without the patch, the behavior is the same as the hackintosh with RX580, immediately crash after logging in the session.

Using the patched version, the screen remains blue, and I can control the mouse cursor for a while, but the session crashes after a short while. Unfortunately I think more work has to be done to solve this issue...

btw Can you recreate the issue on your side?

@unstabler
Copy link
Collaborator

unstabler commented Sep 9, 2023

@laggykiller I completely forgot that sessionprojector.app performs os_log() and fputs() to STDERR for logging. Maybe you can check the logs of sessionprojector.app in Console.app. (+ If you don't mind, could you please attach the logs?)

@laggykiller
Copy link
Author

@unstabler seems nothing useful in log...

麗 -Ulalaca-: sessionbroker version 0.2.2

This software is open source software licensed under the Apache License 2.0.
You may obtain a copy of the source code for this software at https://github.com/team-unstablers/Ulalaca

(c) 2022-2023 team unstablers Inc.
    https://unstabler.pl

[9/11/2023, 8:09 PM][sessionbroker:INFO] Starting broker..
[9/11/2023, 8:09 PM][sessionbroker:INFO] Starting sesman..
[9/11/2023, 8:11 PM][SessionManagerServer:INFO] received announcement from pid 658; user laggykiller: SESSION_CREATED
[9/11/2023, 8:13 PM][SessionBrokerServer:INFO] [laggykiller]: authentication request from user
[9/11/2023, 8:14 PM][SessionBrokerServer:INFO] [laggykiller]: authenticated

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants