WIP: integrating the relavant parts of this project into https://gitlab.com/Mis012/android_translation_layer_PoC/

[Mis012]

Hi,
is this project dead?

it would sure be a shame...
there is some stuff going on with https://github.com/minecraft-linux/,
but I wasn't really able to get that to work with another game...

[Fuseteam]

@Mis012 what are you basing that on? besides from what I can tell there's only one core dev, the best way to keep a project alive would be to get more devs onboard :p

[Cloudef]

I haven't actively moved this forward. But good place to get this project more alive would be to integrate dalvik in it or write your own dalvik vm. If you are looking to just run android software then https://anbox.io/ exists as alternatively to do just that. If I could monetize this project somehow, I would put more work into it for sure.

[Mis012]

so I have made a translation layer... xD

https://gitlab.com/Mis012/android_translation_layer_PoC

would you mind helping to hook your stuff into dalvik?

[Mis012]

right now dalvik is willing to load the .so that the Java code wants, I am able to LD_PRELOAD a .so that implements the bionic functions that dlopen complains about, but when glibc tries to execute stuff from DT_INIT_ARRAY (after I removed the first element which was -1, glibc really didn't like that), for some reason the addresses are not relocated, so I get a nice SIGSEGV

I'm assuming you solved this and more on your end?

[Cloudef]

Yes, you can check the loader.c and libjvm-java.c to see how libraries are loaded, you can probably take the code from there and hook into your implementation.

https://github.com/Cloudef/android2gnulinux/blob/master/src/loader.c#L192-L202
https://github.com/Cloudef/android2gnulinux/blob/master/src/libjvm-java.c#L35-L74

basically you want to use the bionic_dlopen and bionic_dlsymfrom the "libdl" this project provides. Since you use dalvik for the jvm and jni, you don't need any of the libjvm-*.so libraries from this project, but you probably have to modify dalvik to use bionic_dlopen and bionic_dlsym instead of the glibc versions, you can't override dlopen and dlsym, because bionic_dlopen and bionic_dlsym use those internally to load non bionic system libraries as well.

[Mis012]

one issue is that I need to be able to load non-bionic libs as well, not sure if there is a foolproof way to determine whether a lib was linked against bionic?

[Cloudef]

the bionic_dlopen will fallback to normal dlopen if the bionic way fails. The bionic_dlsym should work for both kind of shared objects. For detecting, using heuristic that if the library comes with the apk/app, then it's most likely a bionic library would work I think.

[Fuseteam]

this issue scares everytime i see "project dead?" pop up in my feed :/

[Mis012]

hmm, I get the same issue calling bionic_dlopen with the "modified" library, guess I broke it?
because it seems that it works fine with the original :)

now onto patching dalvik I guess

[Cloudef]

hmm, I get the same issue calling bionic_dlopen with the "modified" library, guess I broke it?

Yeah, most likely the library won't map properly due to the edit, as you did not add padding to compensate to the removal somewhere before the edit. This project's purpose is to work with unedited bionic libraries however, so editing them is not needed, and even if edited they wouldn't work with glibc anyhow. I'm not sure what's the reason for potential 0 and -1 values in those lists, but it's just what google did:

android2gnulinux/src/linker/README.TXT

Line 63 in dfcd8eb

the entries in the array can be 0 or -1, and should

[Mis012]

I added the padding, obviously... -_-

[Cloudef]

Maybe you forgot to edit the DT_INIT_ARRAYSZ as well then. Can't say without actually seeing what you did :)

[Mis012]

I planned to do that after figuring out why it got stuck on the first entry (because it didn't relocate for some reason)

[Mis012]

reee, it seems that bionic linking always fails when loading from PATH, and always succeeds when loading an exact file by name?
even if it's actually a glibc library

[Cloudef]

You need to setup the path for the linker https://github.com/Cloudef/android2gnulinux/blob/master/src/loader.c#L197 (this doesnt affect the dlopen fallback, only for bionic library search, for dlopen make sure your LD_LIBRARY_PATH is set correctly. Andre sets rpath instead, but LD_LIBRARY_PATH should work as well, https://github.com/Cloudef/android2gnulinux/blob/master/GNUmakefile#L97)

Oh also, btw it's probably good idea to grab some libs from ndk (mainly libstdc++.so) and put them into separate directory (runtime-ndk is used in andre). The linker will complain if any dependencies are missing when you bionic_dlopen, so that shouldn't be hard to figure out what you need from ndk, of course usually non bionic libraries work as well, so you don't most of time need anything from ndk. IIRC libstdc++.so is mostly the only problematic one where you can't use system version or it crashed on some apps, my memory is getting hazy here :)

[Mis012]

looks like I've managed to get separate handling to somewhat work... now it complains about missing android/opengl/GLSurfaceView$Renderer, so I think I will be doing some API reimplementation now :P

[Cloudef]

Sounds like that app is using android java apis to render stuff, you may want to try some unity games and see if they boot. Let me know if you had success.

[Mis012]

well, I've got https://github.com/android/ndk-samples/tree/main/gles3jni to draw the background... but not sure why it's not drawing the triangles

[Cloudef]

I don't see anything unusual in that sample. It should work with the current state of this project. What logs do you get? Also can you create a apitrace or renderdoc dump? https://github.com/Cloudef/android2gnulinux#debugging-with-apitrace <- there's this note related to graphics api debugging, but you can ignore it in this case since the ndk sample doesn't use native code to create a window, thus the libEGL.so wrapper in this project is not needed.

[Mis012]

well, I use Gtk's GLArea, so the apitrace has a lot of Gtk stuff, but I can share it :P

dalvikvm.trace.gz

[Mis012]

of course I tried apitrace -a egl as well, but it doesn't work on Wayland, apparently...

and I needed to build gtk from git because they just fixed forcing GLES a month ago, and I couldn't manage to satisfy it's X11 dependencies for a 32bit build

[Mis012]

another issue is that I believe Gtk doesn't guarantee using EGL, so I might have to make my own GLArea that does if android guarantees that

I also tried Worms, looks like that will be quite some fun before I get to test the native part...

[Mis012]

YES

it works :)

turns out it's common practice for the app to wait for me to call onSurfaceChanged so it knows what size the window is
and before it gets that information, it doesn't do anything...

[Cloudef]

Great! Next step get some real games running :)

[Mis012]

I might have to make my own GLArea though... since it seems that even the example hackily relies on disabling the alpha channel in EGL instead of supplying proper alpha value to RGBA code...

[Mis012]

btw, do you have some simple Unity app that works with this project that I could test?

would work a bit differently then GLSurfaceView I imagine, but running such apps in their own Window using your fake libEGL would probably be fine... just need to look how android deals with native activities, I would be surprised if the entry point isn't some Java code but it can be in our case anyway I guess, since that's the simpler way to do JNI :P

[Cloudef]

You could also do your own libandroid.so and libegl.so that integrates with your current GTK implementations. Some shovelware unity apps that I have confirmed to launch are: "The Wolf", some of the idolmaster games. Basically any unity game should work though. Shadowverse might be a good target as they also have windows build that runs under wine which you can compare against. I don't remember if I got shadowverse to actually launch though, but since you run through dalvik I think most things should work as long as the native parts aren't missing.

[Fuseteam]

Yay thanks for renaming the issue

[Mis012]

heh, so I need GLES 1.0 for Worms Armageddon... and GTK won't gib me that

I tried to get something to work with shoveling rendered frames through EGLImage, but in the end I gave up for now and decided to hack in a GLFW Window...

[Cloudef]

well... if I decide that I want to do that, then it would still be cleaner to just do -lpthread_wrapper... or even dlopen

I guess that would work only because of the custom linker that searches bionic__ prefixed symbols from the global namespace. Otherwise bionic programs/libs could load wrong library (glibc one aand not the wrapper one).

E/        (21559): Registry load failed because the file seems to be corrupted: Failed to open /home/Mis012/Github_and_other_sources/android_translation_layer_PoC/data/files/fusion.registry with errno 2 (No such file or directory)
E/AppDataOutputStream(21559): Failed to synchronize, not saving /home/Mis012/Github_and_other_sources/android_translation_layer_PoC/data/files/fusion.registry: Bad file descriptor
dalvikvm: src/libpthread.c:439: bionic_pthread_cond_init: Assertion `cond && (!attr || IS_MAPPED(attr))' failed.

any idea? (I assume the two errors are not necessarily related, or even fatal)

Only the last error seems relevant. Can you check which part of that assert fails?

[Cloudef]

Hmm the pthread wrappers seems to be missing pthread_condattr_* wrappings https://linux.die.net/man/3/pthread_condattr_init

[Mis012]

I'm not at all sure what the wrappers should do though... can't seem to figure it out from looking at the existing ones

[Mis012]

int
bionic_pthread_condattr_destroy(bionic_condattr_t *attr)
{
   assert(attr);
   int ret = 0;
   if (IS_MAPPED(attr)) {
      ret = pthread_condattr_destroy(attr->glibc);
      munmap(attr->glibc, sizeof(*attr->glibc));
   }
   return ret;
}

int
bionic_pthread_condattr_init(bionic_condattr_t *attr)
{
   *attr = (bionic_condattr_t){0};
   attr->glibc = mmap(NULL, sizeof(*attr->glibc), PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
   return pthread_condattr_init(attr->glibc);
}

seems to work

[Mis012]

well, I gave up on the latest version of angry birds, it wanted too much random stuff just to track you... but I've got some 3.x version to work \o/

now I want to try implementing the stuff that it uses for audio

[Cloudef]

Btw, there's linker here that works with both GNU/Linux and android https://github.com/ptitSeb/box86/tree/master/src/elfs, I may use it in this project too instead of writing my own as it's battle tested. This project may work on musl also through gcompat, I should try it as my main system is musl based. It seems though that the only thing where glibc loader loader trips (and maybe musl?) is the PreInit array, and what box86 does is just ignore that. So if that's the case, then your original idea of patching the ELF will probably work. I'll prototype this idea, if there's no incompatibilities, it will be simpler and cleaner approach. I can preload the custom libc and such into library or executable before they are run, with namespacing perhaps, as we don't want glibc libraries to use the bionic versions of the functions, not sure if this is possible with dlopen alone so we may need a linker anyhow.

[Cloudef]

I'm not at all sure what the wrappers should do though... can't seem to figure it out from looking at the existing ones

The bionic and glibc pthread structs aren't compatible and thus we need to do translation on them. That's why you need the wrappers. https://github.com/aosp-mirror/platform_bionic/blob/c44b1d0676ded732df4b3b21c5f798eacae93228/libc/include/bits/pthread_types.h

[Mis012]

I figured it out eventually.

I've implemented audio enough for it to work with angry birds, but it seems that the alsa callback API is basically impossible to use for anything non trivial without random crashes...
might have to try to figure out how to emulate the behavior with polling :/

next I was looking into getting Cut The Rope to run, but right now I'm not getting any obvious errors yet it only clears the GL viewport to white and then seems to do nothing else... :/

[Cloudef]

What sound api angry birds used, OpenSLES?

[Cloudef]

Btw would you be interested integrating your dalvik work into this repo?

[Mis012]

they used some android java API

integrating into this repo doesn't really make sense, it makes more sense the other way around...

what might make sense would be to keep libandroid in the translation layer repo, but move linker and pthread wrapper from the dalvik repo to it's own repo

[Mis012]

fwiw implementing the java parts in C does have it's appeal, for where it's possible... but that will probably be a minority of applications, even ones that are mostly native :/

so the best general approach is to have a translation layer which works for the java applications, including ones which have native components

I would absolutely appreciate if you were to help me with the translation layer, even if it's only for mostly native stuff

---

I'm actually not sure what the deal would be with Native activities, since currently I only support running one Java activity, but looking at https://developer.android.com/reference/android/app/NativeActivity it looks like android has a java wrapper around that?

[carTloyal123]

Any updates on this? I actually have a mildly unrelated question, just want to load and execute functions from an android apk .so binary file to use in a normal C/C++ program. Is it possible to extract that functionality from what is in this repo? @Cloudef Great work here!

[Cloudef]

@carTloyal123 yes, you should be able to use the bionic_dlopen and bionic_dlsym functions, if the native .so needs JNI then it might be bit more involved, but you can see how I load unity .so's in the loader.c:
https://github.com/Cloudef/android2gnulinux/blob/master/src/loader.c#L237-L243
https://github.com/Cloudef/android2gnulinux/blob/master/src/loader.c#L17-L164

[carTloyal123]

Amazing thank you. I will start there. Is what your doing to load unity libs similar to what the apkenv 'modules' folder is doing for specific games?

Also, to complicate the situation a little bit more, my specific situation only has an so file for the arm/arm64 architecture. Best I can understand this will not play nice with the current state of this project, is that right? I assume I can't just compile this project for arm and it will work. I'm a bit of a noob with low level architecture differences past the compiler having the option to compile for a different architecture.

[Cloudef]

Arm hasn't been tested. There are some features such as the function wrappers that definitely won't work (they are only for debugging). However if it compiles then you should be able to run the program with qemu in theory. Also you have to change the define here to ANDROID_ARM_LINKER https://github.com/Cloudef/android2gnulinux/blob/master/GNUmakefile#L14

[carTloyal123]

I will give it a go and see how far I get. Thank you for the responses! I much appreciate it as it seems very few people are deep into the android/native Linux conversion world and even less actually respond on projects. Will report back soon!

[carTloyal123]

So, looks like the first hurdle is here in libc.c: Not Implemented

Any thoughts? @Cloudef

[carTloyal123]

I am also curious why this was marked for the x86 linker only. What is the technical reason? This just as a point of curiosity since I am trying to learn about architecture differences on this journey from a code standpoint when possible.

[Cloudef]

The sigaction struct on arm/arm64 is most likely of a different size. You need to look bionic libc for reference.

EDIT:
https://android.googlesource.com/platform/bionic/+/master/libc/include/bits/signal_types.h#88
It seems the one in libc.c is 32 bit sigaction and 64 bit sigaction is the one that's "not implemented", the #ifdef should be for all 32 bit platforms and not just x86.

[carTloyal123]

Awesome thank you! I will give that a look. So really the current sigaction is for 32-bit and the one needed for arm64/x84_64 will of course be a 64-bit version. Thank you!

[Mis012]

fwiw I've cleaned up the linker (and the libc etc wrappers) quite a bit, I could split it into a separate repo if anyone is interested in reintegrating it into this project or using it standalone: https://gitlab.com/Mis012/dalvik_standalone/-/tree/art-master/bionic_translation

I assumed this project was not in active development anymore, but of course if it is, it would make sense to have this in a separate repo so it can be used by both this project and my project

it would be really great to have more users for this, since to my knowledge it's the most sane implementation of this idea that currently exists... and very likely the only one in C that supports 64bit systems, since the AOSP commit that I backported that from was already C++

[Mis012]

@carTloyal123: it works fine on x86, x86_64 as well as aarch64
don't recall when it was last tested on 32bit arm, but it should work

the libc likewise works fine

[Mis012]

note that the bionic_dlopen function can't assume that it's only ever called for a bionic-linked .so, so you need to use either the env or the function call to set the directory that contains bionic-linked .so files

[carTloyal123]

I would appreciate if you put the libc components in a separate repo. It looks like in my research that many people would like the simplicity of just having dlopen and dlsym from the bionic world to use in normal C projects. Most repos take it all the way to fully running android apps on GNU systems. In my ideal world, I would simply clone the libc repo, include a header where I need and the be able to use something like bionic_dlopen and bionic_dlsym. I am happy to do the split myself from your gitlab repo as well @Mis012

[Mis012]

@carTloyal123: I've split it into https://gitlab.com/android_translation_layer/bionic_translation
fwiw, https://gitlab.com/android_translation_layer/dalvik_standalone is not all the way to running android apps, that part is in https://gitlab.com/android_translation_layer/android_translation_layer

@Cloudef: btw, what is the license for the code in this repo (excluding the linker, which already has a license file)? I guess adding a LICENSE file for the repo would be ideal.

[Cloudef]

@Mis012 MIT, I'll add license later

[Mis012]

@Cloudef: what is the future of this project with android_translation_layer now existing? From the goal of adding java support to run the original java code, it would seem that the path forward would be to just collaborate on android_translation_layer (which uses art to basically run android apps as java apps with JNI, like android does, rather than as native code that happens to call into java for some stuff like your TODO suggests, which I don't think is really possible)

[carTloyal123]

I think it is fine to just leave this project where it is, no need to make it more complicated than that.

[Mis012]

uhm? you're not interested in running android apps anyway, so that part doesn't concern you
the linker works by itself, you don't need anything other than that repo

what are you working on, btw?