How can I build an android custom app for offline deployment

[tim-moody]

Users who have no internet connection can not get to the Google playstore, so Internet in a Box provides kiwix apks with embedded zims that can be loaded to an android phone directly from the server. In previous versions of the custom app builder the json file could be passed as an argument and this json file could have embed_zim: true. It could also have other parameters set to custom values. It looks like this is no longer supported.

The two wiki pages for this repo describe building apks for the Google playstore either with Travis or without using the json files in https://github.com/kiwix/kiwix-android-custom, all of which have embed_zim false.

What are the parameters to pass to build_custom_app.py to get an embedded zim, either built locally or built on Travis for upload to a public site or for download and not to upload to Google? Or is there an alternate method of obtaining an apk with embedded zim?

[tim-moody]

After studying this a bit it looks to me like building on travis is a good approach if we could have some additional parameters: --embed_zim (default false and take from json in if not supplied) and

something to allow the built apk to be uploaded somewhere other than google play store, say with scp. This might well require a private key to be supplied as well. I would rather not have to fork kiwix-build in order to override https://github.com/kiwix/kiwix-build/blob/master/travis/deploy_apk.sh

ideas?

[mgautierfr]

Hum... The functionality is not supported as an easy to use option.
The build without travis and the embedded zim file should be added to build_custom_app.py but this is not the case for now 😞

This is a bit complex for now and there is a lot of commands that will be not necessary to do once everything will be integrated in build_custom_app.py, but here what you have to do :

Clone the kiwix-android-custom repository using kiwix-build.

The easy way to do it is to launch : ./kiwix-build.py --target-platform android_arm kiwix-custom-app --android-custom-app foo --zim_file_size 0
(The options have no importance, but they need to be there)
After the [PREPARE] step is done, you can ctrl-c to stop kiwix-build.py (it will fail anyway)

Use the patch for kiwix-android

Be sure to use the path from this PR : kiwix/kiwix-android#257
Either it as been merged or you will have to switch to branch abs_path_embeded_zim in the git repository SOURCE/kiwix-android

Create your own directory and info.json for you customapp

The source of kiwix-android-custom will be in the SOURCE/kiwix-android-custom directory.
There are already different directories for different apps. Create your own directory with info.json.
In your info.json, set embed_zim to true and zim_file to the absolute path of the zim.

Set the variables VERSION_NAME and ZIM_SIZE

Follow the instructions from https://github.com/kiwix/kiwix-build/wiki/Build-and-publish-a-custom-app-locally-(without-travis)#get-zim-file and set the VERSION_NAME for whatever you want.

You will also need to have a VERSION_CODE and CONTENT_VERSION_CODE (https://github.com/kiwix/kiwix-build/wiki/Build-and-publish-a-custom-app-locally-(without-travis)#build-custom-app-apk)

Create the apk

Then everything is in place to build the custom apk.

Be sure that VERSION_NAME, VERSION_CODE and CONTENT_VERSION_CODE are defined and exported and run:

./kiwix-build.py --target-platform android_<your_arch> --android-custom-app <your_app> --zim-file-size $ZIM_SIZE kiwix-android-custom --skip-source-prepare

<your_app> is the directory name you've created in SOURCE/kiwix-android-custom

The apks will be generated in the directory : BUILD_android_<your_arch>/kiwix-android-custom_<your_app>/app/build/outputs/apk

[mgautierfr]

About your last comment I've missed (build on Travis) :

• We should extend build_custom_app.py to support local build (not on travis) and embedded zims.
• I'm not sure that we (Kiwix project) want to allow anyone to build custom app on travis using the kiwix-build repository of the kiwix organization. You will have to fork kiwix-build in your own organization.
• We could make the deploy_apk.sh script more configurable to allow configuration from a build_custom_app.py options. But it is a bit tricky as this is not simple options but also (encrypted) files that are used (ssh_key, keystore, ..). Ideas are welcomed :)

[tim-moody]

Thanks for all this information. Since using kiwix-build needs a patch from kiwix-android, when do I use kiwix-build and when do I use kiwix-android? Which represents the future?

[tim-moody]

In terms of using travis, I am now thinking I need to create my own repo with a modified android-custom-app.py which launches on my repo (with my credentials) but then clones kiwix-build to do the work and uses a modified deploy_apk.sh to upload the file to a server. I'd rather not have to continually patch a forked kiwix-build.

[tim-moody]

If there are planned changes to this whole workflow that you can share, I would work towards them.

[mgautierfr]

kiwix-build is just a set of scripts (kiwix-build.py, build_custom_app.py, potentially other...) that reuse the specific build system of each sub-projects.

You can use directly kiwix-android build system with gradle (and script in kiwix-android-custom, and build-system of kiwix-lib and dependencies, and ...) or use kiwix-build and don't care of the option to pass to those (sub) build-systems.

Both are the future. They will evolve in parallel.

All those operation are a bit complex because they are not integrated in kiwix-build scripts. But at the end, kiwix-build will just run those commands for you.

[mgautierfr]

Patch for kiwix-android has been merged.

[kelson42]

As long as kiwix/kiwix-android#56 is not properly fixed, Kiwix does not encouraged to use this kind of APK because the installation process might simply die on low-end devices and that is why the documentation is not ready and also why we do not provide directly the APKs at http://download.kiwix.org/

[nataren]

Hi @mgautierfr Thanks for the instructions, they are very helpful. I followed them and got much further on the compilation but it still failed with a Java Heap Space error.

I modified JAVA_TOOL_OPTIONS and JAVA_OPTS environment variables with export JAVA_TOOL_OPTIONS=-Xms1025m -Xmx10240m but it didn't make a difference.

I also tried modifying kiwix-android/app/build.gradle with

  dexOptions {
    javaMaxHeapSize "30g"
  }

but it didn't make a difference.

This what I am doing:

export VERSION_NAME=2017-08
export VERSION_CODE=172530
export CONTENT_VERSION_CODE=172530

python ./kiwix-build.py --verbose --target-platform android_arm --android-custom-app wikimedar kiwix-android-custom  --zim-file-size 1043986445 --skip-source-prepare

The extract of the error is:

Putting task artifact state for task ':app:validateSigningWikimedarCoverage' into context took 0.0 secs.
Executing task ':app:validateSigningWikimedarCoverage' (up-to-date check took 0.0 secs) due to:
  Task has not declared any outputs.
:app:validateSigningWikimedarCoverage (Thread[Daemon worker Thread 13,5,main]) completed. Took 0.0 secs.
:app:packageWikimedarCoverage (Thread[Daemon worker Thread 13,5,main]) started.
:app:packageWikimedarCoverage
Putting task artifact state for task ':app:packageWikimedarCoverage' into context took 0.0 secs.
Task :app:packageWikimedarCoverage class loader hash: 1312d3927f9f8cde8eb435e39623f786
Task :app:packageWikimedarCoverage actions class loader hash: 9c76010b8194cbd85e17a68745fd983a
Executing task ':app:packageWikimedarCoverage' (up-to-date check took 0.035 secs) due to:
  Output property 'incrementalFolder' file /home/anon/mybuild/kiwix-build/BUILD_android_arm/kiwix-android-custom_wikimedar/app/build/intermediates/incremental/packageWikimedarCoverage/dex-renamer-state.txt has changed.
  Output property 'incrementalFolder' file /home/anon/mybuild/kiwix-build/BUILD_android_arm/kiwix-android-custom_wikimedar/app/build/intermediates/incremental/packageWikimedarCoverage/file-input-save-data.txt has been removed.
  Output property 'incrementalFolder' file /home/anon/mybuild/kiwix-build/BUILD_android_arm/kiwix-android-custom_wikimedar/app/build/intermediates/incremental/packageWikimedarCoverage/zip-cache/v50W0REoiuq27ijCm0OPinhEqRw= has been removed.
All input files are considered out-of-date for incremental task ':app:packageWikimedarCoverage'.
Unable do incremental execution: full task run
:app:packageWikimedarCoverage FAILED
:app:packageWikimedarCoverage (Thread[Daemon worker Thread 13,5,main]) completed. Took 1.101 secs.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:packageWikimedarCoverage'.
> Java heap space

* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output.

BUILD FAILED

Total time: 9.867 secs
Stopped 0 worker daemon(s).
Received result Failure[value=org.gradle.initialization.ReportedException: org.gradle.internal.exceptions.LocationAwareException: Execution failed for task ':app:packageWikimedarCoverage'.] from daemon DaemonInfo{pid=19363, address=[4724251d-ce2d-44f7-93c9-194202ca32af port:43127, addresses:[/0:0:0:0:0:0:0:1, /127.0.0.1]], state=Idle, lastBusy=1505033699974, context=DefaultDaemonContext[uid=06a0b1a6-72c4-4479-ad15-e7bb44a594fa,javaHome=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.144-5.b01.fc25.x86_64,daemonRegistryDir=/home/anon/.gradle/daemon,pid=19363,idleTimeout=10800000,daemonOpts=-XX:MaxPermSize=256m,-XX:+HeapDumpOnOutOfMemoryError,-Xmx1024m,-Dfile.encoding=UTF-8,-Duser.country=US,-Duser.language=en,-Duser.variant]} (build should be done).
run command 'gradle assemblewikimedar -i -P customDir=/home/anon/mybuild/kiwix-build/BUILD_android_arm/kiwix-android-custom_wikimedar/custom -P zim_file_size=1043986445 -P version_code=172530 -P version_name=2017-08 -P content_version_code=172530'
current directory is '/home/anon/mybuild/kiwix-build/BUILD_android_arm/kiwix-android-custom_wikimedar'
env is :
  GJS_DEBUG_TOPICS : 'JS ERROR;JS LOG'
  PATH : '/home/anon/mybuild/kiwix-build/BUILD_android_arm/android-ndk-r13b-arm-14/bin:/home/anon/mybuild/kiwix-build/BUILD_android_arm/INSTALL/bin:/usr/lib64/ccache:/home/anon/bin:/usr/local/bin:/home/anon/racket/bin:/usr/lib64/ccache:/home/anon/mybuild/bin:/home/anon/bin:/usr/local/bin:/home/anon/racket/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/anon/bin:/home/anon/bin'
  CXXFLAGS : '-fPIC -D_LARGEFILE64_SOURCE=1 -D_FILE_OFFSET_BITS=64 --sysroot=/home/anon/mybuild/kiwix-build/BUILD_android_arm/android-ndk-r13b-arm-14/sysroot '
  CFLAGS : '-fPIC -D_LARGEFILE64_SOURCE=1 -D_FILE_OFFSET_BITS=64 --sysroot=/home/anon/mybuild/kiwix-build/BUILD_android_arm/android-ndk-r13b-arm-14/sysroot '
  TERM : 'screen-256color'
  XDG_MENU_PREFIX : 'gnome-'
  SHLVL : '2'
  LESSOPEN : '||/usr/bin/lesspipe.sh %s'
  XDG_CURRENT_DESKTOP : 'GNOME'
  LS_COLORS : 'rs=0:di=38;5;33:ln=38;5;51:mh=00:pi=40;38;5;11:so=38;5;13:do=38;5;5:bd=48;5;232;38;5;11:cd=48;5;232;38;5;3:or=48;5;232;38;5;9:mi=01;05;37;41:su=48;5;196;38;5;15:sg=48;5;11;38;5;16:ca=48;5;196;38;5;226:tw=48;5;10;38;5;16:ow=48;5;10;38;5;21:st=48;5;21;38;5;15:ex=38;5;40:*.tar=38;5;9:*.tgz=38;5;9:*.arc=38;5;9:*.arj=38;5;9:*.taz=38;5;9:*.lha=38;5;9:*.lz4=38;5;9:*.lzh=38;5;9:*.lzma=38;5;9:*.tlz=38;5;9:*.txz=38;5;9:*.tzo=38;5;9:*.t7z=38;5;9:*.zip=38;5;9:*.z=38;5;9:*.Z=38;5;9:*.dz=38;5;9:*.gz=38;5;9:*.lrz=38;5;9:*.lz=38;5;9:*.lzo=38;5;9:*.xz=38;5;9:*.bz2=38;5;9:*.bz=38;5;9:*.tbz=38;5;9:*.tbz2=38;5;9:*.tz=38;5;9:*.deb=38;5;9:*.rpm=38;5;9:*.jar=38;5;9:*.war=38;5;9:*.ear=38;5;9:*.sar=38;5;9:*.rar=38;5;9:*.alz=38;5;9:*.ace=38;5;9:*.zoo=38;5;9:*.cpio=38;5;9:*.7z=38;5;9:*.rz=38;5;9:*.cab=38;5;9:*.jpg=38;5;13:*.jpeg=38;5;13:*.gif=38;5;13:*.bmp=38;5;13:*.pbm=38;5;13:*.pgm=38;5;13:*.ppm=38;5;13:*.tga=38;5;13:*.xbm=38;5;13:*.xpm=38;5;13:*.tif=38;5;13:*.tiff=38;5;13:*.png=38;5;13:*.svg=38;5;13:*.svgz=38;5;13:*.mng=38;5;13:*.pcx=38;5;13:*.mov=38;5;13:*.mpg=38;5;13:*.mpeg=38;5;13:*.m2v=38;5;13:*.mkv=38;5;13:*.webm=38;5;13:*.ogm=38;5;13:*.mp4=38;5;13:*.m4v=38;5;13:*.mp4v=38;5;13:*.vob=38;5;13:*.qt=38;5;13:*.nuv=38;5;13:*.wmv=38;5;13:*.asf=38;5;13:*.rm=38;5;13:*.rmvb=38;5;13:*.flc=38;5;13:*.avi=38;5;13:*.fli=38;5;13:*.flv=38;5;13:*.gl=38;5;13:*.dl=38;5;13:*.xcf=38;5;13:*.xwd=38;5;13:*.yuv=38;5;13:*.cgm=38;5;13:*.emf=38;5;13:*.ogv=38;5;13:*.ogx=38;5;13:*.aac=38;5;45:*.au=38;5;45:*.flac=38;5;45:*.m4a=38;5;45:*.mid=38;5;45:*.midi=38;5;45:*.mka=38;5;45:*.mp3=38;5;45:*.mpc=38;5;45:*.ogg=38;5;45:*.ra=38;5;45:*.wav=38;5;45:*.oga=38;5;45:*.opus=38;5;45:*.spx=38;5;45:*.xspf=38;5;45:'
  SESSION_MANAGER : 'local/unix:@/tmp/.ICE-unix/2508,unix/unix:/tmp/.ICE-unix/2508'
  XAUTHORITY : '/run/user/1000/gdm/Xauthority'
  LOADEDMODULES : ''
  HOME : '/home/anon'
  GDMSESSION : 'gnome'
  VERSION_NAME : '2017-08'
  XDG_DATA_DIRS : '/home/anon/.local/share/flatpak/exports/share/:/var/lib/flatpak/exports/share/:/usr/local/share/:/usr/share/'
  S_COLORS : 'auto'
  GOPATH : '/home/anon/'
  CPPFLAGS : '-I/home/anon/mybuild/kiwix-build/BUILD_android_arm/INSTALL/include '
  DESKTOP_SESSION : 'gnome'
  MAIL : '/var/spool/mail/anon'
  LSCOLORS : 'Gxfxcxdxbxegedabagacad'
  CC : '/home/anon/mybuild/kiwix-build/BUILD_android_arm/android-ndk-r13b-arm-14/bin/arm-linux-androideabi-gcc'
  DBUS_SESSION_BUS_ADDRESS : 'unix:path=/run/user/1000/bus'
  XDG_VTNR : '2'
  XDG_SESSION_DESKTOP : 'gnome'
  WINDOWPATH : '2'
  GDM_LANG : 'en_US.UTF-8'
  TMUX : '/tmp/tmux-1000/default,4488,1'
  QT_QPA_PLATFORMTHEME : 'qgnomeplatform'
  LOGNAME : 'anon'
  LDFLAGS : '-L/home/anon/mybuild/kiwix-build/BUILD_android_arm/INSTALL/lib -L/home/anon/mybuild/kiwix-build/BUILD_android_arm/INSTALL/lib64 --sysroot=/home/anon/mybuild/kiwix-build/BUILD_android_arm/android-ndk-r13b-arm-14/sysroot '
  HISTCONTROL : 'ignoredups'
  OLDPWD : '/home/anon/mybuild'
  ANDROID_HOME : '/home/anon/mybuild/kiwix-build/TOOLCHAINS/android-sdk-r25.2.3'
  PWD : '/home/anon/mybuild/kiwix-build'
  XDG_SESSION_ID : '6'
  SSH_AUTH_SOCK : '/run/user/1000/keyring/ssh'
  JAVA_OPTS : '-Xms1024m -Xmx10240m'
  TMUX_PANE : '%2'
  SHELL : '/bin/zsh'
  JOURNAL_STREAM : '8:37462'
  XDG_SESSION_TYPE : 'x11'
  CONTENT_VERSION_CODE : '172530'
  USERNAME : 'anon'
  _ : '/home/anon/mybuild/bin/python'
  WINDOWID : '31457286'
  GJS_DEBUG_OUTPUT : 'stderr'
  LANG : 'en_US.UTF-8'
  DISPLAY : ':1'
  PKG_CONFIG_PATH : ':/home/anon/mybuild/kiwix-build/BUILD_android_arm/INSTALL/lib64/pkgconfig'
  VTE_VERSION : '4601'
  MODULESHOME : '/usr/share/Modules'
  VIRTUALENVWRAPPER_SCRIPT : '/usr/bin/virtualenvwrapper.sh'
  XDG_SEAT : 'seat0'
  HOSTNAME : 'anon'
  USER : 'anon'
  VERSION_CODE : '172530'
  HISTSIZE : '10000'
  XDG_RUNTIME_DIR : '/run/user/1000'
  COLORTERM : 'truecolor'
  XMODIFIERS : '@im=ibus'
  PKG_CONFIG_LIBDIR : '/home/anon/mybuild/kiwix-build/BUILD_android_arm/android-ndk-r13b-arm-14/sysroot/lib/pkgconfig'
  LD_LIBRARY_PATH : '/home/anon/oldicu/:/home/anon/oldicu/::/home/anon/mybuild/kiwix-build/BUILD_android_arm/INSTALL/lib:/home/anon/mybuild/kiwix-build/BUILD_android_arm/INSTALL/lib64'
  GNOME_DESKTOP_SESSION_ID : 'this-is-deprecated'
  QT_IM_MODULE : 'ibus'
  NDK_DEBUG : '0'
  ZSH : '/home/anon/.oh-my-zsh'
  PAGER : 'less'
  LC_CTYPE : 'en_US.UTF-8'
  JAVA_TOOL_OPTIONS : '-Xms1025m -Xmx10240m'
  echo : ''
  MODULEPATH : '/etc/scl/modulefiles:/etc/scl/modulefiles:/etc/scl/modulefiles:/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/share/modulefiles'
  CXX : '/home/anon/mybuild/kiwix-build/BUILD_android_arm/android-ndk-r13b-arm-14/bin/arm-linux-androideabi-g++'
  _VIRTUALENVWRAPPER_API : ' mkvirtualenv rmvirtualenv lsvirtualenv showvirtualenv workon add2virtualenv cdsitepackages cdvirtualenv lssitepackages toggleglobalsitepackages cpvirtualenv setvirtualenvproject mkproject cdproject mktmpenv mkvirtualenv rmvirtualenv lsvirtualenv showvirtualenv workon add2virtualenv cdsitepackages cdvirtualenv lssitepackages toggleglobalsitepackages cpvirtualenv setvirtualenvproject mkproject cdproject mktmpenv mkvirtualenv rmvirtualenv lsvirtualenv showvirtualenv workon add2virtualenv cdsitepackages cdvirtualenv lssitepackages toggleglobalsitepackages cpvirtualenv setvirtualenvproject mkproject cdproject mktmpenv'
  LESS : '-R'
ERROR

[nataren]

You can see the full output log in the attached file. output.txt

[mgautierfr]

@nataren, That's strange. It's gradle failing because of some kind of missing heap space.
Have you try this : https://stackoverflow.com/questions/41645747/android-studio-gradle-build-failing-java-heap-space

Maybe @mhutti1 can help. He knows the gradle build system better than me.

[nataren]

Creating the file`~/.gradle/gradle.properties with the heap size that I can spare worked for me:

org.gradle.jvmargs=-XX\:MaxHeapSize\=10g -Xmx10g

The build succeeded and I got several apk's at ./BUILD_android_arm/kiwix-android-custom_wikimedar/app/build/outputs/apk, the listing is:

app-wikimedar-coverage.apk
app-wikimedar-local_download_server.apk
app-wikimedar-release-unsigned.apk 
app-wikimedar-debug.apk
app-wikimedar-mock_network.apk

I tried installing app-wikimedar-release-unsigned.apk on an android phone but the Package Installer is telling me: The package appears to be corrupt.

[mhutti1]

@nataren Is your architecture right? If so can you share the apk?

[kelson42]

For the "release-unsigned" APKs, I just want to remind that ticket kiwix/kiwix-android#151

[nataren]

The phone's arch is ARM Cortex-A53. I used --target-platform android_arm as command argument for kiwix-build.py.

[mhutti1]

@nataren can you try the debug version then?

[nataren]

The debug version can be installed but it fails to launch.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

[kelson42]

@MohitMaliDeveloper I believe moving to bundle has changed many things. Would we be able to provide a bundled app in a form of one file which could be sideloaded?