Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lack of context initialization in case of host memory #18

Open
e-ago opened this issue May 24, 2017 · 0 comments
Open

lack of context initialization in case of host memory #18

e-ago opened this issue May 24, 2017 · 0 comments
Labels

Comments

@e-ago
Copy link
Collaborator

e-ago commented May 24, 2017

In case of GDS_ALLOC_CQ_ON_GPU=1 or GDS_ALLOC_DBREC_ON_GPU=1 the gds_peer_malloc() function is called and the cuInit(0) function initialize the context.

On the contrary, if both QP and DBREC reside on host memory, libgdsync doesn't initialize properly the context before calling the gds_register_va() function and the following error can occur:

[19078] GDS ERR gds_register_mem_internal() Error 709 (context is destroyed) while register address=0x7f4570008000 size=65536 flags=00000003
[19078] GDS ERR register_range() got 22 while registering range [0x7f4570008000,0x7f4570017fff]
[19078] GDS ERR gds_register_va() error while registering range, returning 0 as error value
[19078] GDS ERR gds_create_cq() error 2 in ibv_exp_create_cq, old errno 2
[19078] GDS ERR gds_create_qp() error 2 while creating TX CQ, old_errno=2
[19078] [1] MP ERR mp_init() qp creation failed

To overcome this issue the application using gds needs to call, for example, a cudaFree(0) before creating QPs and CQs.

@e-ago e-ago added the bug label May 24, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant