Module talk:grc-utilities

New tokenize function
I created a new version of the lua function in Module:User:Erutuon/grc and added it in.

The old version used a lot of Ustring functions, especially lua and lua. This made Module:grc-decl slow, because it frequently uses functions in Module:grc-accent that require tokenization.

The new version doesn't use any Ustring functions at all, except initially to decompose the characters. It relies only on the current character, the previous character, and the current token to decide which token to put the current character in. It's more verbose and difficult to understand, but significantly faster. That means that adding accents to every declined form in Module:grc-decl/sandbox with lua in Module:grc-accent is a bit faster than it was before. — Eru·tuon 21:10, 10 November 2017 (UTC)