BIRT 4.22 Export PDF fails in multi threaded environment when running many reports first time

[bineetsingh30]

Description

Given no BIRT report has run before after the process started, and now if many (17 in my case) run parallelly, one of the report fails sometimes with NullPointerException. (2 stack traces attached, stackTrace2 is being caused due to stackTrace1)
stackTrace1.txt
stackTrace2.txt

Environment

• OS: Linux
• BIRT Version: 4.22.0

[hvbtup]

To me, this looks like a race condition (similar to #2385, but in a different place).

It seems as if either OpenPDF is no longer multithreading-safe,
or we (in the BIRT code) use OpenPDF in a wrong way.

You should try if you can fix it by changing how #2144 is implemented, but maybe this even needs changes in OpenPDF itself.

Or if you can add some synchronized(...) blocks around the critical paths.

[merks]

The stack traces suggest that the soft cache in ICU4J is not thread safe:

SEVERE: Index 34 out of bounds for length 33
java.lang.ArrayIndexOutOfBoundsException: Index 34 out of bounds for length 33
	at java.base/java.util.ArrayList.add(ArrayList.java:455)
	at java.base/java.util.ArrayList.add(ArrayList.java:467)
	at com.ibm.icu.impl.ResourceBundleWrapper.initKeysVector(ResourceBundleWrapper.java:87)
	at com.ibm.icu.impl.ResourceBundleWrapper.access$900(ResourceBundleWrapper.java:29)
	at com.ibm.icu.impl.ResourceBundleWrapper$2.load(ResourceBundleWrapper.java:238)
	at com.ibm.icu.impl.ResourceBundleWrapper$1.createInstance(ResourceBundleWrapper.java:44)
	at com.ibm.icu.impl.ResourceBundleWrapper$1.createInstance(ResourceBundleWrapper.java:41)
	at com.ibm.icu.impl.SoftCache.getInstance(SoftCache.java:69)
	at com.ibm.icu.impl.ResourceBundleWrapper.instantiateBundle(ResourceBundleWrapper.java:149)
	at com.ibm.icu.impl.ResourceBundleWrapper.access$100(ResourceBundleWrapper.java:29)
	at com.ibm.icu.impl.ResourceBundleWrapper$2.load(ResourceBundleWrapper.java:159)
	at com.ibm.icu.impl.ResourceBundleWrapper$1.createInstance(ResourceBundleWrapper.java:44)
	at com.ibm.icu.impl.ResourceBundleWrapper$1.createInstance(ResourceBundleWrapper.java:41)
	at com.ibm.icu.impl.SoftCache.getInstance(SoftCache.java:69)
	at com.ibm.icu.impl.ResourceBundleWrapper.instantiateBundle(ResourceBundleWrapper.java:149)
	at com.ibm.icu.impl.ResourceBundleWrapper.getBundleInstance(ResourceBundleWrapper.java:126)
	at com.ibm.icu.util.UResourceBundle.instantiateBundle(UResourceBundle.java:378)
	at com.ibm.icu.util.UResourceBundle.getBundleInstance(UResourceBundle.java:149)
	at com.ibm.icu.util.UResourceBundle.getBundleInstance(UResourceBundle.java:128)
	at org.eclipse.birt.core.i18n.ResourceHandle.<init>(ResourceHandle.java:83)
	at org.eclipse.birt.report.engine.i18n.EngineResourceHandle.<init>(EngineResourceHandle.java:42)
	at org.eclipse.birt.report.engine.emitter.pdf.PDFPageDevice.<init>(PDFPageDevice.java:264)
	at org.eclipse.birt.report.engine.emitter.pdf.PDFRender.createPageDevice(PDFRender.java:89)
	at org.eclipse.birt.report.engine.layout.emitter.PageDeviceRender.start(PageDeviceRender.java:144)
	at org.eclipse.birt.report.engine.layout.emitter.PageEmitter.start(PageEmitter.java:70)
	at org.eclipse.birt.report.engine.nLayout.LayoutEngine.start(LayoutEngine.java:263)
	at org.eclipse.birt.report.engine.emitter.CompositeContentEmitter.start(CompositeContentEmitter.java:208)
	at org.eclipse.birt.report.engine.api.impl.RunAndRenderTask.doRun(RunAndRenderTask.java:144)
	at org.eclipse.birt.report.engine.api.impl.RunAndRenderTask.run(RunAndaRenderTask.java:69)

Maybe that was never thread safe? I don't see stuff on the stack in BIRT that's change since 2021, and even those look not related in any way to threading...

[bineetsingh30]

Hi @hvbtup, this is not related to the OpenPDF issue. As @merks suggested, it looks like thread safety issue in ICU4J. The class com.ibm.icu.impl.ResourceBundleWrapper doesn't seem to be thread safe.

[merks]

There are quite some calls to that method as well:

I'm not sure if they'd all need to be synchronized externally...

[bineetsingh30]

After looking at the code for com.ibm.icu.impl.ResourceBundleWrapper and I can see specific case in which such issue can occur. A bug on ICU should also be raised, but I don't have permission to raise a bug in ICU.

[merks]

I think it comes from here, right?

https://github.com/unicode-org/icu

I guess they track issues here:

https://unicode-org.atlassian.net/jira/software/c/projects/ICU/issues?jql=project%20%3D%20ICU%20ORDER%20BY%20cf%5B10011%5D%20ASC

But you can create an issue only if you log in.

But even when I logged in I still can't create an issue:

One might create a PR to get their attention:

https://github.com/unicode-org/icu/pulls

They certainly don't make it easy to contact anyone about anything!! 😢

[hvbtup]

Hi @hvbtup, this is not related to the OpenPDF issue.

You're right.

[hvbtup]

Looking at our code in PDFPageDevice.java, this call to new EngineResourceHandle(...) has not been changed for a really long time, so it seems that actually this is a multi-threading issue in ICU4J.

[merks]

Looking at our code in PDFPageDevice.java, this call to new EngineResourceHandle(...) has not been changed for a really long time, so it seems that actually this is a multi-threading issue in ICU4J.

Yes, looking up the stack at all the source, I even see comment as if it should be thread safe:

But in the end, the failure suggests two threads are adding to the same list at the same time...

---

If I wanted to work around such a problem, I might try to generate a single small "warmup report" before continuing with the multi-thread approach, and hope that primes the cache sufficiently...

[hvbtup]

| But in the end, the failure suggests two threads are adding to the same list at the same time...

That's my impression, too. But I don't dare to to say "my conclusion", I'm too tired...

[bineetsingh30]

I have reached out to them in their google support group (as described on https://unicode-org.github.io/icu/contacts.html). Will update here in case I am able to raise a bug on ICU.

[merks]

Thank you.

Please keep us informed. BIRT gets ICU from Orbit and Orbit automatically finds updated versions in Maven Central:

https://github.com/eclipse-orbit/orbit-simrel/blob/1d4764ee7fc8e96d1060042222a059de11370c17/maven-osgi/tp/Maven.target#L323-L328

So if the problem is fixed and the fix is published to Maven Central, we'll update to it quickly...

[jabagawee]

might be fixed by unicode-org/icu#3902

[bineetsingh30]

Hi @jabagawee, looking at your change, it should fix this issue. Thanks for your help.
Once your changes are merged, and ICU4J is released, BIRT would need to update to the new version of ICU4J.