Calc running hot and slow

[Snick]

I have just given up transcribing out of sheer frustration with Calc because today it is having one of its go-slow episodes.

Most times I can load Calc and rattle through a page of (typed) weather data pretty quickly. I work by column and change cells using the arrow keys. On a good day I add data to a cell then switch to the next cell without any perceptible delay. Today, on a go-slow day, Calc quickly starts to lag in switching cells, sometimes taking an estimated second or more to change cells. The result is that instead of (for eg)

Code: Select all

35, 35, 36, 37

I get

Code: Select all

35, 3, 5363, [empty cell]

The obvious warning that this is likely to happen is that Calc runs 'hot', that is, the CPU fan runs harder than usual even though the RSS (Resident Set Size [of Memory]) shows the same value as when Calc is not running slow and hot.

If I stop entering data into Calc but leave it running it will cool down, but almost as soon as I start to add new data the fan speed goes up and the lag in changing cells manifests itself again. This generally only happens in the evening (UK time).

System is LibreOffice 6.4.4.2 in Xubuntu 20.04.

Any ideas, anyone?

Nick

[Randi]

We have occasionally seen some odd behavior regarding the speed, but there has been no mention of it running hot.
In general speed problems been unrepeatable.
However, many times when Adjust Angle was used (Angle Set is non-zero on the Setup sheet) we have noticed that the cursor will jump to the right and then jump back when entering data.

[ggordon]

This is a known problem. I'm working on a solution that while it won't speed things up, will hopefully buffer the input so as to prevent the loss of data.

I've observed the problem on Windows but not on Linux. However, I have suspected that it would show up on Linux too. It appears the problem occurs when you rotate the image from its original orientation and your system has a memory limitation.

I've been quite frustrated encountering the same problem on Windows. My Windows PC does not have much memory, but my Linux system has lots of memory and I have never had the problem there.

I don't have a lot of time available to work on this, so all I can say is that I'm aware of the problem and working on a solution.

If you could add enough additional memory to your system, the problem would likely go away. Otherwise you'll have to wait for me to resolve the issue. I would love to stop everything and work on this, but I have ongoing projects that need to be addressed outside of OW.

---

If you are interested, here is more detail.

The problem is that when you rotate the image, Calc rotates the image again every time you change the contents of a cell. To rotate the image, it creates a temporary copy of the image. Since our log page images are rather large, this requires quite a bit of memory. As long as there is enough memory available, this happens fast enough so as to not be noticeable. However, this becomes a problem if your system does not have enough memory available. Calc has to wait for the operating system to free up memory for it to use. If memory is in short supply, there becomes an ongoing need for the operating system to conduct memory management, thus causing a heavier load on the CPU.

This would still be okay if your input was buffered. However, our spreadsheets include event handlers. When event handlers are executed, Calc stops buffering input and relies on the spreadsheet programmer (us) to handle the buffering. Unfortunately none of this is documented. I only recently figured out this was happening and am working on a way to buffer the input so that nothing gets lost.

You may wonder why you have this problem even though the system says that it is not using 100% of memory. Operating systems never allow 100% of memory to be in use if at all possible. They reserve a certain amount so as to minimize the impact when individual applications need more memory than is allocated to them. Just because the system shows free or available memory, that does not necessarily mean it has enough to meet the demands of everything running on the system.

[Snick]

Thanks, this makes sense, and I can find and induce or avoid the problem by either having a page rotated or not; the effect is near instant. The simplest solution then is to cope with all but the wildest of image rotations.
For information, I am running 8Gb of RAM which I would have liked to think was adequate in Xubuntu; certainly I have to push QGIS hard to achieve the same stress I can induce in Calc. But then Linux (and LibreOffice) appear to be slowly suffering the same excesses that have effected Windows for years.
Thanks again,
Nick

[ggordon]

It's unfortunate that LibreOffice rotates the image each time an entry is made in a cell. It doesn't seem like that should be necessary. Hopefully they have some good reason for doing it.

My Windows PC with 6 GB, but almost nothing else running, has problems with rotated images. My Linux system is running Cinnamon/Ubuntu 18.04 with 16 GB and I never have problems with rotated images. I wasn't sure whether it was just a memory issue or whether there was a difference between Windows and Linux. Your feedback seems to confirm that it is a memory issue.

My Linux PC had 8 GB and I had lots of things running at all times. Prior to my arrival at OW I was having sluggish response at times. So I upgraded to 16 GB and everything has run great since then.

Yes, the Linux memory requirements have grown significantly since I first started using Linux. A few years ago I installed Xubuntu on a 20 year old laptop and it ran great. But gradually Xubuntu grew to where it was no longer usable. I tried a couple of other minimal Linux distros, but they were all too sluggish.

[ggordon]

We have learned more about the problem since these earlier posts. It turns out to be a CPU limitation rather than a memory limitation and that many computers can be configured to use their graphical processing unit (GPU) instead of the CPU to do the rotations and eliminate the problem.

For more information, see this post:
http://www.oldweatherforum.org/viewtopi ... 321#p13321