Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Wasm32 as target for cross compiling #2404

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

kjartanm
Copy link

This PR adds Wasm32 as target for cross compiling, Emscripten setup, and fences off some Emscripten-incompatible elements.

See Feature request #1670

kjartanm added 3 commits February 20, 2021 13:18
To enable Emscripten builds, we need to fence off a couple of elements that creates unrecoverable errors during build process.
This adds Wasm32 as target together with Emscripten setup to enable cross compiling of or-tools into wasm.
@kjartanm kjartanm requested a review from lperron February 24, 2021 14:48
@Mizux Mizux self-requested a review March 23, 2021 08:06
@Mizux Mizux added the Feature Request Missing Feature/Wrapper label Mar 23, 2021
@Mizux Mizux added this to the v9.1 milestone Mar 23, 2021
@Mizux Mizux modified the milestones: v9.1, v9.2 Sep 8, 2021
@Mizux Mizux modified the milestones: v9.2, v9.3 Oct 5, 2021
@Mizux Mizux self-assigned this Oct 27, 2021
tools/cross_compile.sh Outdated Show resolved Hide resolved
@Mizux Mizux modified the milestones: v9.3, v10.0 Feb 11, 2022
@akutruff
Copy link

What's blocking WASM support right now? Is this dead?

@Mizux
Copy link
Collaborator

Mizux commented Sep 21, 2022

mostly lack of human ressource (i.e. lack of time) to work on it + fear of the technical debt increase to maintain it.

BTW I'm not against it, on the contrary I'd love to have an ortools-lite running on the edge on client side web browser...

@Mizux Mizux added the Lang: WASM WASM wrapper issues label Sep 21, 2022
@akutruff
Copy link

Wish I could help on the build errors - Seems like there's some simple permissions error somewhere?

@Mizux Mizux removed this from the v10.0 milestone Apr 26, 2024
@Mizux Mizux added this to the Backlog milestone Apr 26, 2024
@chadbrewbaker
Copy link

I have an immediate need for this. What is the blocker?

This is how Z3 does it, they provide not just a WASM build but also a JS interface. I agree this should be effectively an or-tools-lite in the default build, you should have to pass some extra flag to build a really fat binary.

Minimal viable commit is defining a JSON for an or-tools graph format and having it pass some unit tests?

@Mizux
Copy link
Collaborator

Mizux commented Jun 7, 2024

My 2 cents,

We have a job to test emscripten build
https://github.com/google/or-tools/actions/workflows/amd64_web.yml
src: https://github.com/google/or-tools/blob/stable/.github/workflows/amd64_web.yml

using:
make --directory=cmake web_build
src:

or-tools/cmake/Makefile

Lines 660 to 698 in ef7bb07

#########
## WEB ##
#########
WEB_STAGES := env devel build test
define make-web-stage-target =
#$$(info STAGE: $1)
#$$(info Create target: web_$1.)
.PHONY: web_$1
web_$1: docker/web/Dockerfile
#@docker image rm -f ${IMAGE}:web_$1 2>/dev/null
${DOCKER_BUILD_CMD} --target=$1 --tag ${IMAGE}:web_$1 -f $$< ..
#$$(info Create target: save_web_$1 (debug).)
.PHONY: save_web_$1
save_web_$1: cache/web/docker_$1.tar
cache/web/docker_$1.tar: web_$1
@rm -f $$@
mkdir -p cache/web
docker save ${IMAGE}:web_$1 -o $$@
#$$(info Create target: sh_web_$1 (debug).)
.PHONY: sh_web_$1
sh_web_$1: web_$1
${DOCKER_RUN_CMD} -it --name ${IMAGE}_web_$1 ${IMAGE}:web_$1
#$$(info Create target: clean_web_$1.)
.PHONY: clean_web_$1
clean_web_$1:
docker image rm -f ${IMAGE}:web_$1 2>/dev/null
rm -f cache/web/docker_$1.tar
endef
$(foreach stage,$(WEB_STAGES),$(eval $(call make-web-stage-target,$(stage))))
## MERGE ##
.PHONY: clean_web
clean_web: $(addprefix clean_web_, $(WEB_STAGES))
rm -f cache/web

Dockerfile: https://github.com/google/or-tools/blob/stable/cmake/docker/web/Dockerfile
cmake config:

RUN emcmake cmake -S. -Bbuild \
-DCMAKE_BUILD_TYPE=Release -DBUILD_DEPS=ON \
-DBUILD_EXAMPLES=OFF \
-DUSE_COINOR=OFF \
-DUSE_GLPK=OFF \
-DUSE_HIGHS=OFF \
-DUSE_PDLP=OFF \
-DUSE_SCIP=OFF

Need to compare/test with this PR...

@@ -265,6 +265,25 @@ QEMU_ARGS+=( -L "${SYSROOT_DIR}" )
QEMU_ARGS+=( -E LD_PRELOAD="${SYSROOT_DIR}/usr/lib/libstdc++.so.6:${SYSROOT_DIR}/lib/libgcc_s.so.1" )
}

function expand_wasm_config() {
local -r EMSDK_VERSION=2.0.14
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Mizux,
Is it planned for next release or will it take time? Also, is this command work with stable branch:
emcmake cmake -S . -B build -DBUILD_DEPS=ON -DBUILD_PYTHON=ON

I tried after making changes in cross_compile.sh and I am getting error as:
Could NOT find Python3 (missing: Python3_INCLUDE_DIRS Python3_LIBRARIES Development Development.Module Development.Embed) (found version "3.11.9")

However, I don't see such error when using just cmake?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I don't get it, what did you try to do/expect ?

BUILD_PYTHON is used to generate automatically python wrapper on top of the C++ library to have access to ortools in python (which is just a wrapper on top of the native library, since we won't reimplement ortools in pure Python/Java nor .Net).

On the other side, AFAIK, emcmake is used to transpile C++ code into wasm byte code to have a "browser compatible" source code so there is no point to try to build a python wrapper on top of this wasm library.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi there, I recently pointed @airen1986 to this issue from pyodide/pyodide#5013. If configured correctly, Pyodide would allow runtime for a Python interface for this WASM library through an Emscripten-compiled Python extension module.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, I guess I understand what you want to do:
0. This is not something we support

  1. currently our wrapper is generated by swig which is currently use to build 64bit code -> don't know how it work with WASM32 ptr size etc...
  2. IIRC emcmake is working by injecting a CMake cross toolchain thus my bet is the Toolchainfile do not provide a python thus the issue.
  3. I would suggest to play with my pet project: https://github.com/Mizux/python-native which should have the same cmake stuff than or-tools so once you have something work there it should be easier to patch/fixup or-tools and dev iteration should be faster IMHO...

sorry this comment is little bit dry, will try to reformulate if you want...

@@ -403,6 +423,9 @@ function main() {
ppc)
expand_bootlin_config
declare -r QEMU_ARCH=ppc ;;
wasm32)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note for myself, emscripten generate wasm32 by default
ref: https://emscripten.org/docs/tools_reference/settings_reference.html#memory64

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature Request Missing Feature/Wrapper Lang: WASM WASM wrapper issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants