Hi Alex,
If I understand this, you are saying that:
A) If you run the same report in XI 3.1 it returns
B) If you run the BEx query outside of Webi then it returns
C) If you run the UNV based Webi report in BI 4.1 it doesn't return (but sometimes will)
Have you tested with a new report using the BICs connectivity in BI 4.1?
Also, what is the version of your BI 4.1 environment exactly? BI 4.1 SP01 and above does have a 64-bit BAPI driver available which allows for the Webi Processing Server to natively process UNV/BAPI universes, before that, it had to use the Connection Server 32 bit process which can add a lot of overhead.
Here's an article on that: here
Other than that, the Array Fetch Size could very well be the issue. I've seen where some migrated reports and universes are defaulting to an Array Fetch Size of 10. This means a LOT of fetches are done and it can really slow things down. Setting this to 1000 can help a lot if resources allow that high of a number.
The Adaptive Processing Server is used for BICs based Webi reports but not so much for UNV based reports. It would be the Webi Processing Server and potentially the Connection Server 32-bit that would be used.
Running the back-end traces on the BW side may help determine root cause too but if the same report runs quicker in XI 3.1, then its likely something configuration wise on the BI 4.1 environment and my guess would be the Array Fetch size as mentioned.
Thanks!
jb