[tex-live] Primitive parity, \expanded and \Ucharcat

[Joseph Wright]

Hello all,

In adding features to expl3, the LaTeX team have been making use of a 
variety of 'new' (post-e-TeX) 'utility' primitives in various engines. 
Almost always these originate in pdfTeX and have migrated to other 
engines, but are not in any way tied to PDF output, etc. Depending on 
the exact engine in use, some or all of these primitives may be 
unavailable, and that then limits macro-level features.

It seems sensible long-term to have cross-engine feature stay 'in sync' 
with each other. In particular, (u)pTeX has picked up a number of pdfTeX 
features, meaning that XeTeX often is the most 'limited' engine. The 
team would like, if possible, to have a common feature set in all 
engines in this regard. At the same time, there are a few 'bits and 
pieces' that make sense to raise at the same time. I'll lay out the 
various areas below.

Doing the work here is non-trivial, but luckily there is an automated 
build system available via GitHub which is allowing us (me/David 
Carlisle) to do some testing. I'm building up patches in various 
branches at https://github.com/josephwright/texlive-source: assuming 
these look good, I'll merge them as required and send diff files to 
where/whoever is best. The branches on GitHub should hopefully have 
clear names for what they address.

The areas we are keen to look at are as follows:

- 'pdfutils': (u)pTeX has picked up a number of pdfTeX primitives, and
   a subset have made their way into XeTeX too. However, XeTeX is still
   missing several, most notably an expandable RNG. We are part-way
   though working out patches to add the rest to XeTeX (RNG is done,
   file data and timer to do)

- banners: pdfTeX and LuaTeX have \<engine>banner, other engines lack
   that. The banner includes TeX version and details of the TeX system,
   so is potentially useful. Adding this to (u)pTeX/XeTeX looks
   straight forward: still to-do.

- \expanded: This was slated for pdfTeX 1.50 but that has never
   appeared, but the primitive is useful as it allows 'function-like'
   expandable macros. We can see this begin very useful for simplifying
   some macro code, and in many ways it feels like an e-TeX primitive.
   The GitHub expanded branch adds it to pdfTeX/XeTeX/(u)pTeX

- Allowing \Ucharcat (XeTeX) to make \active tokens: this was raised
   recently on the XeTeX list, but fits here as we've put a branch
   together to show it works

It's likely I'll finish the outstanding patches by the weekend. Note 
that at present each feature addition is in a separate Git branch, so to 
add all of them I'll have to do a little tidying up: that will happen 
once I know which of these suggestions are useful.

Feedback most welcome.

Joseph