intltool race condition during parallel builds (sometimes)

[sunweaver]

Expected behaviour

Builds always succeed on any supported architecture.

Actual behaviour

Builds sometimes fail, obviously a race condition.

22:08 < sunweaver> https://buildd.debian.org/status/package.php?p=mate-utils
22:08 < sunweaver> according to bunk (on OFTC #debian-buildd), this is a parallel build issue.
22:09 < sunweaver> probably needs fixing upstream.

See https://buildd.debian.org/status/package.php?p=mate-utils (the build sucesses might change with my next upload, which I will do once I have a fix for this issue).

Steps to reproduce the behaviour

Building the Debian package from Debian unstable on a Debian unstable system.

while dpkg-buildpackage -B -J100 --no-sign; do true; done

MATE general version

1.20

Package version

1.20.2

Linux Distribution

Debian testing/unstable

Some feedback from Jeremy Bicha (Debian GNOME Team and Ubuntu Developer):

13:41 < jbicha> sunweaver: I didn't try it, but maybe you need dh_auto_build --no-parallel for mate-utils
13:42 < jbicha> and as a long term fix, maybe it's suffering from https://launchpad.net/bugs/1687644 Because intltool is unmaintained, I recommend switching 
                to gettext alone
13:43 < jbicha> https://wiki.gnome.org/MigratingFromIntltoolToGettext and https://wiki.gnome.org/Initiatives/GnomeGoals/GettextMigration
13:45 < jbicha> mate-utils had trouble building on Ubuntu too, but it worked after I retried it a few times

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[lukefromdc]

J100 is a LOT of jobs at once. How many cores do you have? I've had no trouble with just "make -j" which gives unrestricted jobs with mate-utils, but an attempt on my same system (AMD Bulldozer 1st gen "8 core") to build kdenlive or GTK will fail unless the number of jobs is restricted, generally I use 8 for that, one per supported thread.

[sunweaver]

Hi,

On Saturday, 12 January 2019, lukefromdc wrote: J100 is a LOT of jobs at once. How many cores do you have? I've had no trouble with just "make -j" which gives unrestricted jobs with mate-utils, but an attempt on my same system (AMD Bulldozer 1st gen "8 core") to build kdenlive or GTK will fail unless the number of jobs is restricted, generally I use 8 for that, one per supported thread.

The -J100 is for stress testing parallel builds and intact internal build orders. Of course, you wouldn't do that on production build servers. However, on the diverse Debian build archs, we see those failures here and there. Mike -- Sent from my Sailfish device

[shirishag75]

It's still an issue, see https://bugs.debian.org/919058

[lskovlun]

It doesn't have to use multicore. Going to the gsearchtool/help/pt directory and repeatedly saying
itstool -m pt.mo ../C/index.docbook ../C/legal.xml
will eventually cause an exception in itstool. So then the question becomes, what is this? an error in the input data (that shouldn't cause a crash of course), in itstool, in libxml2 or what?

[lskovlun]

I just noticed that someone else is having problems with nondeterministic exceptions in itstool:
itstool/itstool#21
and in my own investigations I am seeing clear signs of dangling pointers and/or memory corruption.

[monsta]

Well, #256 is now merged, so the migration from intltool to gettext is done. And itstool issue is tracked in #210. Should we close this one?

[lukefromdc]

If we don't use that anymore, we should close this as it should never be seen again

[lukefromdc]

I just got this (or something similar) on a new faster system (ryzen): parallel build with gettext enabled failed, but restarting the build finished it. Got this error:

Traceback (most recent call last):
  File "/usr/bin/itstool", line 1614, in <module>
    doc.merge_translations(translations, opts.lang, strict=opts.strict)
  File "/usr/bin/itstool", line 1002, in merge_translations
    lcnode.setProp(attr, origlang)
  File "/usr/lib/python3/dist-packages/libxml2.py", line 3588, in setProp
    if ret is None:raise treeError('xmlSetProp() failed')
libxml2.treeError: xmlSetProp() failed
  GEN    sl/
  GEN    sq/
  GEN    sr@latin/
  GEN    sr/
  GEN    sv/
  GEN    ta/
make[3]: *** [Makefile:615: pt/pt.stamp] Error 1
make[3]: *** Waiting for unfinished jobs....
make[3]: Leaving directory '/home/luke/Desktop/Development/MATE_Development_Work/mate-utils_1.25.1+git20210710/gsearchtool/help'
make[2]: *** [Makefile:454: all-recursive] Error 1
make[2]: Leaving directory '/home/luke/Desktop/Development/MATE_Development_Work/mate-utils_1.25.1+git20210710/gsearchtool'
make[1]: *** [Makefile:525: all-recursive] Error 1
make[1]: Leaving directory '/home/luke/Desktop/Development/MATE_Development_Work/mate-utils_1.25.1+git20210710'
make: *** [Makefile:457: all] Error 2

Reopening but not sure if this is the same as #210 or this issue. Close it again if it is an instance of the former

[raveit65]

@lukefromdc
It is simply a problem with Portuguese translation.
https://github.com/mate-desktop/mate-utils/blob/master/.build.yml#L119
sed -i s/"IGNORE_HELP_LINGUAS ="/"IGNORE_HELP_LINGUAS = pt"/g gsearchtool/help/Makefile.am

[raveit65]

Any way i filed out an report.
https://bugzilla.redhat.com/show_bug.cgi?id=1981081

[lukefromdc]

I guess this can be closed again then, since that is a different issue?