C API: errors while running tests

[e-polevikov]

Hi guys,

First of all, thank you for the great tool! I have been using the pcodec python bindings recently and it works perfectly.

Currently, I am trying to get familiar with the C API and I encountered some errors while running tests.

1. When I execute run_test.sh I get undefined reference to pco_auto_compress/pco_auto_decompress error.
2. If I replace pco_auto_compress with pco_sumpler_compress and pco_auto_decompress with pco_simple_decompress, everything works fine until the moment when memory deallocation happens. Then, I get segmentation fault.

Seems like there is some issue with the pco_free_pcovec function. Would you please comment on this? Is there any chance to fix this issue?

[mwlon]

I can't vouch for the C API's quality - in fact I'm pretty confident it should be destroyed and started over. I'm hoping someone with a bit more C expertise than me could work on this. Would you be interested in working on this?

[mwlon]

To elaborate a bit: Rust shouldn't allocate and own the memory in these "FFI vecs". The C user should allocate the memory (probably based on a guarantee function), and Rust should just fill them in mutably.

[e-polevikov]

I can't vouch for the C API's quality - in fact I'm pretty confident it should be destroyed and started over. I'm hoping someone with a bit more C expertise than me could work on this. Would you be interested in working on this?

I would like to try but currently I don't know If I will be able to allocate enough resources on that in the nearest future.

For now, I found the following workaround: call free(dvec.raw_box) and free(cvec.raw_box) to deallocate memory. Do you think it's okay to work with the current API version this way? Valgrind shows no memory leaks in this case but prints some Invalid read of size X errors (not sure if it might cause any troubles).

[mwlon]

I'm not sure. Sounds risky!

[e-polevikov]

I'm not sure. Sounds risky!

I see. Anyway, I will try to play a bit with the current version this way on my datasets, if everything goes smoothly, I'll probably try to fix the issue or rewrite the API as you suggested.

Thank you for your comments.