Wiktionary:Lua memory errors

Wiktionary often experiences Lua (Scribunto) memory errors in certain entries with many language sections (chiefly certain one- and two-Latin-letter entries), on some pages with many translations, and on some single Han character entries.

Background
Lua is the scripting language found on Module-namespace pages (see WT:LUA). It is used in many templates. It is used when there is  (a module invocation) on a page, or in text that is transcluded on the page, from a template for instance.

It has a garbage collector, which frees up memory after it is no longer accessible from the Lua code. Memory is not freed immediately, so memory usage is somewhat unpredictable. Memory can rise or fall each time you preview a page, and removing a module invocation from a page can actually increase memory.

The memory limit is 50 megabytes. When Lua memory goes over this limit, an error with the explanation "not enough memory" is displayed in place of the module invocation and the page is placed in Category:.

Difficulties
Certain pages have had memory errors since at least 2019. These include some one- and two-Latin-letter entries and some single Han character entries. Examples include a.

Some pages with many translations have had memory errors.

Lua-free templates
For the Latin-script entries like a, the primary solution has been to swap out most uses of key highly-used templates, such as head and m, for their Lua-free equivalents, such as head-lite and m-lite. The  templates are written in pure wiki-syntax, hence taking up no Lua memory at all.

The Lua-free templates have more limited functionality than the corresponding Lua versions. For basic usage with most Latin-script languages, they are a like-for-like drop-in replacement. However, they should normally be avoided for links to non-Latin-script languages (this is mainly an issue in etymology sections). This is because the Lua-free templates do not perform automatic transliteration and normalisation. Also, certain rarely-used parameters are not supported so that the  templates stay easy to maintain. In practice, our very long entries use a mix of the Lua-free and regular templates, saving the Lua memory for where it is actually needed.

A full list of Lua-free templates can be found at Category:Lua-free templates.

Use Template:multitrans
On pages with many translations, the preferred solution to memory errors is to use multitrans. This involves wrapping the entire translation table with a call to multitrans and changing all calls to t inside the wrapped code with tt, and all calls to t+ with tt+. See orange, wolf and several other pages for examples of how to do this. This often reduces the memory by a factor of two (e.g. from 50+ MB to 25 MB). The way it works is to load the translation and linking modules only once, instead of once per translation (which may amount to several hundred copies in memory).

Translation subpages
On pages with many translations, if the multitrans solution doesn't work, an alternative has been to move translations to a fake  subpage created by adding   to the title and translation subpage on the top of the page; see Category:Translation subpages for the full list.

These translation subpages are not true subpages because the subpage feature is turned off in the main namespace. This allows there to be entries for terms that contain slashes. A true subpage has a link to the parent page under its title. This does not happen to titles with slashes in the main namespace. For instance, water/translations does not have a link to water under its title.

Derivation subpages
These work similarly to translation subpages. These store all derived terms from an individual lemma on a separate, fake subpage, and are mostly used with Han/CJKV characters. It is created by adding  to the title and derivative subpage on top of the page; they will then be added to Category:Derivative subpages.

Increasing the memory limit
A Wiktionary admin posted a Phabricator task that suggested increasing the memory limit. This would probably not be a long-term solution because memory usage would probably continue increasing and eventually exceed the new limit.

Replacing letters templates
Some of the most vulnerable targets for Lua memory errors are single-letter Latin alphabet pages. In fact, all five original Latin vowel letters (a, e, i, o, and u) have faced module errors multiple times. Hence, the Lua-dependent or some other derivative appears often on these pages. One workaround is to replace some of the templates dependent on with "simple" versions composed of plain wikitext, in order to reshuffle the garbage collection. Unfortunately, garbage reshuffling doesn't always happen and sometimes replacing letters list templates may worsen module errors.

Optimizing Lua memory usage
It might be possible to redesign some modules (including core modules) to reduce memory usage. Larger changes might be required in order to have a noticeable effect. It remains to be seen whether this can be achieved in practice.

Workaround for users
Content of a section that cannot be seen because it is replaced by “ Lua error: not enough memory ” can often be viewed by first clicking “&#x5b; edit ]” next to the section header, and then the button Show preview below the edit window.

Per-language pages
This is an old proposal, which predates the Lua memory issue, but might solve it. A very detailed overview and links to old discussions can be found at Per-language pages proposal.