Modules will not work with Android

[peardox]

This one adds to the discoveries made in #9

Some Goolgling indicates the following to be a rather huge problem.

While Modules can be "installed" on Android they invariably contain a "*.so" library file. These files get installed into a directory which is marked as noexec resulting in the library existing but not being loadable.

This is the reason that the deployment of libpython[ver].[subver].so is required in addition to the Python system files (which include 'noexec' another copy of the same file)

While extensions without a library should install and function properly those with a library can't run with the current deployment method (as they'll be noexec).

One solution to this issue would be to deploy 'extra copies' of the module libraries along with the main APK and use those instead of the module's library. Problematically this fragments the deployment strategies depending on OS, CPU and Architecture.

One particularly annoying issue is that this does rather preclude the ability to deploy ad-hoc libraries to Android as you'll need to package all libraries along with the main APK.

Research continues...

[peardox]

I had a look around and there's a product in the Play Store called PyDroid so I got it (well, the 3 day trial) to see what it does - try it yourself.

See nrecording Screen recording 2022-08-12 1.21.35 PM.webm

The app has a 'plugin' for Pip. What this appears to do is download to local and then install from there.

If you install torch and psutil the code in the grab will do as shown (including missing Numpy out)

The upshot of this is that Pip can install in Android - just not sure how...

[lmbelo]

That's why I said you need to deploy the Python interpreter shared library and executable to the same folder as your app. It also address to extension modules or any other package that uses executables.
However, this is a known issue.

[lmbelo]

I can install any package distributed with source from PIP without any issue on my Android device. I only need to place the Python shared library and executable in the correct folder.

[peardox]

The specific project that's not doing installs is https://github.com/peardox/P4D-Tests - the FMX folder. That creates a EPyModuleInstallError every time. That one is installed with the PYTHONENVIRONMENTDIR set and does have a functioning Python Interpreter - it's checked out half way through install ATM just to be sure.

After that it does selectable (via IFDEFs package installs) - this part it where it fails ATM

Can you try installing that FMX app on Android?

Is there anything important about the order things are done in? Does Enable Python + Select Version have to be done before a PythonEnvironment and modules are added to a form?

II've downloaded Trial into a VM for Mac M1 tests (dunno what happens after 30 days - wipe VM and get another Trial under a different email?). I will get around to checking that out properly as well this week on the free M1 credit AWS have given me for a while (32 days of use til end of year).

Note : When FMX version installs on everything (subject to Delphi version) I'll simplify the hell out of my code and publish a Tutorial + stuff

[peardox]

Hmm - just tried a cloning HelloCPUFMX into a fresh directory. It won't even compile - keeps complaining about missing @drawable/splash_image

Looking that one up it appears CE is broken in regard to Android - https://en.delphipraxis.net/topic/5763-android-project-deployment-messed-up/

I'm trying a SDK Manager update to see if that helps (as it compiled before + after I re-installed delphi)

Also trying D11 in VM (paused that while Android SDK updates)

[lmbelo]

The only requirement is vendors must make their packages Android compliant.

I just installed debugpy on my device using PIP. Check it out:

FYI, many packages have dependencies that would be installed before them. You also need to make sure its dependencies are Android compliant.

You only need to figure out how to make it work xP

[peardox]

FYI I'm re-re-re-re-installing the whole set - things are going very well.

We must have had some small version discrepancies as this time everything is installing without issue (no hassle for Mac / Android)

I noticed that I had CV2_Contrib still in my set while you have OpenCVContrib - that indicated some potential mis-matches.

Hopefully this will resolve getting HelloCPUFMX working

[lmbelo]

I will try to give you all required steps in a while...

[lmbelo]

PSUtil, PyTorch and NumPy are not Android compliant, you can't install it directly from PIP.

Are you enabling the AUTOSCAN option of the PyEmbeddableEnvironment? Are you placing the GetPipAddOn on you form? First of all you need to install PIP, right?!

[peardox]

Not mentioned anywhere in the documentation :)

BTW everything just installed perfectly - just finished the re-re-re.... install

So, for Android specifically add GetPipAddon and set AUTOSCAN. Not required for others (so $IFDEF ANDROID the section?)

This should prove interesting in my more advanced project - no components on form, everything done at runtime...

Quick test - Yup, Main project still works on Win64 (sorta important)

[peardox]

Are you enabling the AUTOSCAN option of the PyEmbeddableEnvironment?

There's no AUTOSCAN option in the 3.9 packaged env - switching to PyEmbeddableEnvironment

It was ALMOST working without AUTOSCAN (well, still failed just not as badly)

[peardox]

Looking at this....
Screen recording 2022-08-14 12.06.35 AM.webm

Numpy, Torch and PSUtil will work on Android - possibly specific versions, maybe specially built

The PyDroid guy must be rich...

[lmbelo]

Does it requires any shared library? Send it with your app all would work good.
Do I have mentioned that there is a step-by-step guide showing how to compile PyTorch for Android on theirs official website?
By the way, your recording is not playing.

[peardox]

Oh, stupid Chromebook - basically it was showing Numpy, Pytorch and PSUtil running on Android via PyDroid

Yeah, build Torch myself is my next step - trying simpler options first

This looks interesting https://github.com/pypa/manylinux

I tried installing the bad packaged with Platform set to 'manylinux2014_aarch64' - which resulted in Command Error 2 (what's that one? - PIP documentation is all over the place too)

I did try extracting the shared libs from my nvidia Nano (it's aarch 64 so should be fine) but that didn't work. Then again nothing worked when I tried and things are 'less broken' now so I'll give it a go again.

Why are some modules not Android compliant? It would be desirable to have Android versions of everything in our Data Science set. First job is, I guess, to work out which ones are bad most knowing my luck) then work out the fixes and apply those to the packages somehow (which is hopefully just a deploy addition).

Personally I need the three I've been stuck on and a few more (SciPy, TorchVision, OnnxRuntime and Boto3) - might as well find out if any of those work as-is, start with SciPy (a important one) I guess...

[peardox]

Installed 9 - 8 failed (Boto3 works)

The ChromeBook is running through the same test. I's producing more output and it's very interesting. It's showing why something couldn't install (it's all down to missing libs and/or toolchains so far)

I'll post the full report later in Discord

[peardox]

Logfile
https://discord.com/channels/989230637342933042/989230639796588556/1008185984871059468

Interesting permission problems immediately stand out

[Walid-alg]

المتطلب الوحيد هو أن يقوم البائعون بجعل حزمهم متوافقة مع نظام Android.

لقد قمت للتو بتثبيت debugpy على جهازي باستخدام PIP. تحقق من ذلك:

للعلم، هناك العديد من الحزم التي تحتوي على تبعيات سيتم تثبيتها قبلها. كما يجب عليك التأكد من أن التبعيات متوافقة مع Android.

كل ما عليك فعله هو معرفة كيفية جعله يعمل xP

How you do it ?
How to install this library with pip ?
Can you show how,
Thank's