Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Build universal native libraries on macOS [2 XMR] #843

Open
woodser opened this issue Mar 26, 2024 · 25 comments
Open

Build universal native libraries on macOS [2 XMR] #843

woodser opened this issue Mar 26, 2024 · 25 comments
Labels
💰bounty There is a bounty on this issue

Comments

@woodser
Copy link
Contributor

woodser commented Mar 26, 2024 •
edited
Loading

Haveno can use native libraries instead of monero-wallet-rpc for wallet functionality using the monero-java project.

Currently, the native libraries built for macOS are platform-dependent, so they must be built for x86_64 and ARM64.

This issue requests making the native libraries universal on macOS, so multiple builds are not necessary.

This issue shares a bounty with the related issue: woodser/monero-java#81
@woodser woodser added the 💰bounty There is a bounty on this issue label Mar 26, 2024
@github-actions GitHub Actions
Copy link

There is a bounty on this issue, the amount is in the title. The reward will be awarded to the first person or group of people who resolves this issue.

If you are starting to work on this bounty, please write a comment, so that we can assign the issue to you. We expect contributors to provide a PR in a reasonable time frame or, in case of an extensive work, updates on their progresses. We will unassign the issue if we feel the assignee is not responsive or has abandoned the task.

Read the full conditions and details of our bounty system.

@woodser woodser changed the title Build universal native libraries on macOS [1.5 XMR] Build universal native libraries on macOS [2 XMR] Mar 27, 2024
@preland
Copy link
Contributor

preland commented Mar 28, 2024

I’ll look into this

@woodser
Copy link
Contributor Author

woodser commented Mar 28, 2024

Excellent, happy to have your help. Thanks.

@niyid
Copy link
Contributor

niyid commented Apr 2, 2024

I'm curious; is the goal to avoid multiple builds or to have a universal native library on Mac?

@woodser
Copy link
Contributor Author

woodser commented Apr 2, 2024

Universal native library for macOS x86_64 and ARM64.

@niyid
Copy link
Contributor

niyid commented Apr 2, 2024

Universal native library for macOS x86_64 and ARM64.

What if the universal native library is generated from already built x86_64 and ARM64 libraries? Then that involves multiple builds. That is why I ask.

@woodser
Copy link
Contributor Author

woodser commented Apr 2, 2024 •
edited
Loading

That could be one way to do it, but I was expecting to use cmake configuration and toolchains like monero-project, to avoid multiple builds and be able to do it from one machine: https://github.com/monero-project/monero/tree/master/cmake

@niyid
Copy link
Contributor

niyid commented Apr 2, 2024 via email

@niyid
Copy link
Contributor

niyid commented Apr 2, 2024 via email

@woodser
Copy link
Contributor Author

woodser commented Apr 2, 2024

I'm reading that multiple flags can be passed to the build command, e.g. CXXFLAGS="-arch x86_64 -arch arm64 -mmacosx-version-min=11.0 -std=c++11"

But I haven't looked into it deeply.

@niyid
Copy link
Contributor

niyid commented Apr 2, 2024 via email

@woodser
Copy link
Contributor Author

woodser commented Apr 2, 2024

Nope, but sounds worth trying.

@niyid
Copy link
Contributor

niyid commented Apr 2, 2024 via email

@niyid
Copy link
Contributor

niyid commented Apr 5, 2024 via email

@woodser
Copy link
Contributor Author

woodser commented Apr 7, 2024

set(CMAKE_OSX_ARCHITECTURES "x86_64;arm64") did not work by adding it to CMakeLists.txt in monero-cpp and monero-java; the resulting libs work on x86_64 but not arm64.

@niyid
Copy link
Contributor

niyid commented Apr 7, 2024

...and you added the lipo command?

I have not tested this yet (no Mac):

cmake_minimum_required(VERSION 3.4.1)

project(monero-java-jni)

# Set architectures for Apple platforms
if (APPLE)
    set(ARCHITECTURES x86_64 arm64)
endif()

# For other platforms, use the original script
if (WIN32)
    add_definitions( "-D_GLIBCXX_USE_NANOSLEEP=1" )
    add_definitions( "-DWIN32_LEAN_AND_MEAN" )
    SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wa,-mbig-obj -O2 -fPIC -std=c++14 -F/Library/Frameworks -pthread -lcrypto -lcrypt32") 
else()
    SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fPIC -std=c++14 -F/Library/Frameworks -pthread")
endif()

#SET(CMAKE_C_COMPILER /path/to/c/compiler)
#SET(CMAKE_CXX_COMPILER /path/to/cpp/compiler)

#############
# System
#############

set(MONERO_CPP "${CMAKE_SOURCE_DIR}/external/monero-cpp")
message(STATUS MONERO_CPP : ${MONERO_CPP} : ${MONERO_CPP})

set(MONERO_CPP_SRC "${MONERO_CPP}/src")
set(MONERO_PROJECT ${MONERO_CPP}/external/monero-project)
set(MONERO_PROJECT_SRC "${MONERO_PROJECT}/src")

# java header includes
if(NOT DEFINED ENV{JAVA_HOME} OR "$ENV{JAVA_HOME}" STREQUAL "")
  message(FATAL_ERROR "JAVA_HOME variable not set, for example: export JAVA_HOME=/path/to/jdk")
else()
  include_directories("$ENV{JAVA_HOME}")
  include_directories("$ENV{JAVA_HOME}/include")
  if (APPLE)
    include_directories("$ENV{JAVA_HOME}/include/darwin")
  elseif (WIN32)
    include_directories("$ENV{JAVA_HOME}/include/win32")
  else()
    include_directories("$ENV{JAVA_HOME}/include/linux")
  endif()
endif()

# monero-project header includes
include_directories("${MONERO_CPP}/external/libsodium/include/sodium")
include_directories("${MONERO_CPP}/external/openssl-sdk/include")
include_directories("${MONERO_CPP_SRC}/")
include_directories("${MONERO_PROJECT}/contrib/epee/include")
include_directories("${MONERO_PROJECT}/external/")
include_directories("${MONERO_PROJECT}/external/easylogging++")
include_directories("${MONERO_PROJECT}/external/rapidjson/include")
include_directories("${MONERO_PROJECT_SRC}/")
include_directories("${MONERO_PROJECT_SRC}/crypto")
include_directories("${MONERO_PROJECT_SRC}/crypto/crypto_ops_builder/include/")
include_directories("${MONERO_PROJECT_SRC}/wallet")
include_directories("${MONERO_PROJECT_SRC}/wallet/api")

# TODO: remove TRUEs, how are APPLE, DEPENDS, etc initialized?
if (TRUE OR HIDAPI_FOUND OR LibUSB_COMPILE_TEST_PASSED)
  if (APPLE)
    if(TRUE OR DEPENDS)
      list(APPEND EXTRA_LIBRARIES "-framework Foundation -framework IOKit -framework AppKit")
    else()
      find_library(COREFOUNDATION CoreFoundation)
      find_library(IOKIT IOKit)
      find_library(APPKIT AppKit)
      list(APPEND EXTRA_LIBRARIES ${IOKIT})
      list(APPEND EXTRA_LIBRARIES ${COREFOUNDATION})
      list(APPEND EXTRA_LIBRARIES ${APPKIT})
    endif()
  endif()
  if (WIN32)
    list(APPEND EXTRA_LIBRARIES setupapi)
  endif()
endif()

message(STATUS EXTRA_LIBRARIES: ${EXTRA_LIBRARIES})

############
# Boost
############

set(Boost_NO_BOOST_CMAKE 1)
set(Boost_USE_MULTITHREADED ON)
find_package(Boost 1.58 QUIET REQUIRED COMPONENTS chrono date_time filesystem program_options regex serialization wserialization system thread)
message(STATUS "Using Boost include dir at ${Boost_INCLUDE_DIR}")
include_directories(${Boost_INCLUDE_DIR})

############
# OpenSSL
############

if (APPLE AND NOT IOS)
  set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fvisibility=default -std=c++14")
  if (NOT OPENSSL_ROOT_DIR)
      EXECUTE_PROCESS(COMMAND brew --prefix openssl
        OUTPUT_VARIABLE OPENSSL_ROOT_DIR
        OUTPUT_STRIP_TRAILING_WHITESPACE)
    message(STATUS "Using OpenSSL found at ${OPENSSL_ROOT_DIR}")
  endif()
endif()

find_package(OpenSSL REQUIRED)
message(STATUS "Using OpenSSL include dir at ${OPENSSL_INCLUDE_DIR}")
include_directories(${OPENSSL_INCLUDE_DIR})

if(STATIC AND NOT IOS)
  if(UNIX)
    set(OPENSSL_LIBRARIES "${OPENSSL_LIBRARIES};${CMAKE_DL_LIBS};${CMAKE_THREAD_LIBS_INIT}")
  endif()
endif()

if (WIN32)
  list(APPEND OPENSSL_LIBRARIES ws2_32 crypt32)
endif()

######################
# monero-cpp
######################

add_library(monero-cpp SHARED IMPORTED)

# import shared c++ library
if (APPLE)
  set_target_properties(monero-cpp PROPERTIES IMPORTED_LOCATION ./libmonero-cpp.dylib)
elseif (WIN32)
  set_target_properties(monero-cpp PROPERTIES IMPORTED_LOCATION ./libmonero-cpp.dll)
  set_target_properties(monero-cpp PROPERTIES IMPORTED_IMPLIB ./libmonero-cpp.dll.a)
else()
  set_target_properties(monero-cpp PROPERTIES IMPORTED_LOCATION ./libmonero-cpp.so)
endif()

# Check if building for macOS to combine builds into a universal library using lipo
if(APPLE)
    # Add flags and configurations for each architecture
    foreach(arch IN ITEMS ${ARCHITECTURES})
        set(CMAKE_C_FLAGS_${arch} "${CMAKE_C_FLAGS} -arch ${arch}")
        set(CMAKE_CXX_FLAGS_${arch} "${CMAKE_CXX_FLAGS} -arch ${arch}")
        set(CMAKE_LIBRARY_OUTPUT_DIRECTORY_${arch} ${CMAKE_BINARY_DIR}/${arch})
        add_library(monero-java-${arch} SHARED ${MONERO_JNI_SRC_FILES})
        target_link_libraries(monero-java-${arch}
            monero-cpp
            ${Boost_LIBRARIES}
            ${OPENSSL_LIBRARIES}
            ${EXTRA_LIBRARIES}
        )
    endforeach()

    add_custom_command(TARGET monero-java-x86_64 monero-java-arm64
        POST_BUILD COMMAND
        lipo -create
        $<TARGET_FILE:monero-java-x86_64>
        $<TARGET_FILE:monero-java-arm64>
        -output ${CMAKE_LIBRARY_OUTPUT_DIRECTORY}/libmonero-java.dylib
    )

    # Install the combined library
    INSTALL(TARGETS monero-java-x86_64
        RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR} COMPONENT Runtime
        LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR} COMPONENT Runtime
        ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR} COMPONENT Development
    )
endif()

@NamesCode
Copy link

I can test this for you, I have an M2 mac and I can emulate an x86-64 one using UTM

@preland
Copy link
Contributor

preland commented Jun 2, 2024

I can test this for you, I have an M2 mac and I can emulate an x86-64 one using UTM

The main issue (sorry, I kinda dropped the ball on this issue, was working on other things) was/is getting monero to properly static compile for both x86_64 and arm64 on the same system. I was able to get that far in GitHub workflows, but I am still unable to properly test if lipo will create a proper universal binary (especially the cctools Linux port, which seems to hate GitHub workflows with a burning passion)

the most recent workflow run I did is here; if you could run lipo on the artifacts and see if it works properly, that would be highly appreciated. It should, but honestly it wouldn’t shock me at this point if it found a way to mess things up

@NamesCode
Copy link

NamesCode commented Jun 2, 2024 •
edited
Loading

I can test this for you, I have an M2 mac and I can emulate an x86-64 one using UTM

The main issue (sorry, I kinda dropped the ball on this issue, was working on other things) was/is getting monero to properly static compile for both x86_64 and arm64 on the same system. I was able to get that far in GitHub workflows, but I am still unable to properly test if lipo will create a proper universal binary (especially the cctools Linux port, which seems to hate GitHub workflows with a burning passion)

the most recent workflow run I did is here; if you could run lipo on the artifacts and see if it works properly, that would be highly appreciated. It should, but honestly it wouldn’t shock me at this point if it found a way to mess things up

Heyo! Good news!! after running lipo create ~/Downloads/Cross-Mac-x86_64/{Binary name here} ~/Downloads/Cross-Mac-aarch64/{Binary name here} -output ./{Binary name here} and chmod +x'ing the resulting binaries both worked perfectly on aarch64. No warnings or issues. Had a bit of a surprise when monerod started syncing too

The binaries themselves work in practice so you may be able to just run the above command in the final linker step?
I havent taken a look at the build.yml myself to see where it is erroring.

Proof it works too ^^:

image image

@niyid
Copy link
Contributor

niyid commented Jun 2, 2024

@ all, thank you all for your contribution and proving the idea works.

@NamesCode
Copy link

If there's anyway I can help further I'm more than happy too ^^

I'll see if I can take a look at the GitHub workflows tomorrow if you'd like @preland unless it's not going to be used in the final build?

@preland
Copy link
Contributor

preland commented Jun 2, 2024

If there's anyway I can help further I'm more than happy too ^^

I'll see if I can take a look at the GitHub workflows tomorrow if you'd like @preland unless it's not going to be used in the final build?

That would be appreciated; once that is working properly, then the path to getting this issue resolved will be as “simple” as monero->monero-cpp->monero-java->haveno

@NamesCode
Copy link

If there's anyway I can help further I'm more than happy too ^^
I'll see if I can take a look at the GitHub workflows tomorrow if you'd like @preland unless it's not going to be used in the final build?

That would be appreciated; once that is working properly, then the path to getting this issue resolved will be as “simple” as monero->monero-cpp->monero-java->haveno

It seems the issue is that the build-universal-macos-app action requires that the artifacts be pushed to a release first before it can create them.

The problem with this is Monero uses gitian for releases which includes signings, hashes etc. This means the fatbinary action needs gitian to be done first before it itself can be pushed into that release, making a chicken or the egg problem.

The two ideas that come to mind at the moment is either:

A)

Haveno bundles the fat binaries itself, using the github action your using to pull moneros official down and merge them.
A benefit of this is its generally easier and simpler to do, however a major downide of this though is that it is not officially signed by monero. For most people this isn't an issue as no one is really checking the file signature and we can sign it ourselves as if they are using haveno, they most likely already trust the binaries we provide.

B)

We hand roll a custom lipo maker thats able to use the previous artifacts somehow, possible a bash snippet post macos build in a macos env or creating a custom program and dockerizing it.
A benefit of this is it can be merged into monero upstream but there are quite a few more cons as, it may be more convoluted to pull the artifacts and lipo them and we would have to hand roll a program just to do so if we dont go the bash way which requires a macos vm.
Im not too well versed in github actions however so take this option with a pinch of salt since I dont fully know all the options available with it, we may just be able to pass the artifacts directly to amalgam (The name of the program the universal-macos-releaser uses under the hood) and have it produce a fat binary which gets uploaded to artifacts and included in the gitian release. If this is the case then this'd be ideal.

With my limited knowledge I'd want to say option A may be better but if we can just pull an artifact and put it into amalgam on the monero upstream then option B would be the best case.

I'll take a look into option B after the 18th since I have exams over the next few weeks making me unable to work on things.

@preland
Copy link
Contributor

preland commented Jun 7, 2024

Thanks for looking into it :)

Part of the issue is that, with the current dependency chain, the best solution would be to create the brand new lipo target within the main monero repo, and then have the dependency get fixed that way.

In terms of getting that done: we can figure out how to get the action working, and then wait for the PR to get merged; in the meantime we would just directly use the PR commit for monero-cpp

I noticed that there was a single pre-existing GitHub workflow that would use lipo to create a universal binary, but it wasn’t working when I tried it out. Maybe we could make a custom one? Tbh if it has to rely on a MacOS runner that shouldn’t be too much of a concern at this point (since the lipo step isn’t really in scope for Monero static builds, as it is rly little more than packaging. At most we may need to add some extra information in build documentation)

@preland preland mentioned this issue Jun 12, 2024
Merged
@preland preland mentioned this issue Jul 26, 2024
Open
@preland
Copy link
Contributor

preland commented Jul 28, 2024

Ignore my previous comment; it is wrong.

Looking into it deeply, the easiest way to do this is to build the entire libraries for both x86 and Arm separately, then combine using lipo.

I currently have PRs open in monero-cpp and monero-java that do just that. I can confirm that it works on a Debian 12 system. I have yet to test it on a macOS host (Windows won't work out of the box currently as I didn't update portions of the toolchain to allow for it to be used).

My only concern currently is whether or not the built artifact in monero-java will currently be accessible via jitpack.io (ie what we use to access gradle/maven/whatever artifacts from repositories). If that concern is addressed, then this will be a fully functioning universal macOS build pipeline (that can be compiled statically on other platforms).

One current pitfall is that the bash scripts will build the entire project from scratch. I can start looking into other changes once necessary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
💰bounty There is a bounty on this issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@woodser @niyid @NamesCode @preland