Tuesday, March 8, 2022

Enabling Calc support for 16384 columns by default

Last couple of weeks I have been working on the 16k columns support in Calc. There's been a lot of work on this already by Noel and others, but so far this has been hidden behind the experimental option, and normally documents open only with the "normal" 1024 columns support. The goal of this work is to finish the 16k support stable enough for it to be the default, so that people who need this many columns can finally get them without any complications.

As of now all Calc tests pass with the default switched to 16k, and I've also dealt with all the known problems from tdf#133764 (minus few rare corner cases that I can deal with later). But I'm pretty sure there are more hidden problems lurking, either crashes because of incorrect bounds checking, or performance problems when some code suddenly deals with 16x more columns. So the next step is to enable this by default in master and collect compl... feedback from  guin... testers :).

I have the change already in Gerrit and expect to push it later today. If you'll be lucky unlucky noticing any problems, please report them in bugzilla and set your bugreport to block tdf#133764, and I'll sort it out. One of the things that remains to be decided is how to handle this from the users point of view. So far it seems to it'll be fine to just load everything in 16k without saying anything, but a part of collecting feedback is checking whether and how much backwards compatibility handling would be needed. If all goes well, and so far I don't see why it shouldn't, 7.4 will ship with 16k columns being the default.

Just to be clear, this is only about columns, not rows. The experimental option also changes rows from 1048576 to 16777216, but that's not focus of the work, so the default will stay at 1048576. First of all, Excel also only supports 1m rows, so there's not(?) such a demand for it, and it'll also require more work (16m rows is a lot of pixels, and it doesn't fit into 32bits).

This work is funded/sponsored by DEVxDAO as part of its mission to support open source and transparent research and development of emerging technologies and frameworks.

Fun meaningless fact: Column 16384 is called XFD.


5 comments:

  1. ...XFD? an aha moment a week later: Z*Z*X(26*26*24=16224)+Z*F(26*6=156)+D(4) = 16384

    ReplyDelete
  2. This comment has been removed by the author.

    ReplyDelete
  3. I guess that once you have reached the column XFD, your spreadsheet is eXtremely Filled with Data ;-)

    ReplyDelete
  4. I'm not a spreadsheet power user, so I've never come close to pushing row or column limits. Out of curiosity, this post prompted to check current limits. I'm using 7.3.1.3 in Windows, and 7.3.4.2 in Linux; I only upgrade when the first or second digit in the version number changes.

    Since the vast majority of Libre Calc users are casual and never approach the row or column limits, I'd suggest adding 3 options to the Calc -Default tab for default spreadsheet size: small (16384 rows, 256 cols), medium (1048576 rows, 1024 cols) and large (16777216 rows, 16384 cols). Out of the hundreds of personal spreadsheets I've created, I don't think any of them have ever exceeded the small limits. I am demanding about my system's responsiveness, and I'm guessing restricting default sheet size helps in that regard.

    Thanks.

    ReplyDelete