Frequently Asked Question List for TeX
Misc
TeX Engine development
While Knuth has declared that TeX will never change in any substantial way, there remain things that one might wish had been done differently, or indeed implemented at all. These have been explored in a number of engine developments, some of which have somewhat complex histories.
The first set of major additions to Knuth’s TeX were provided by the NTS project in ε-TeX: these additions are available in all current engines. pdfTeX added a range of new primitives, and many these have also been taken up by other engine developers.
Active projects
pdfTeX
pdfTeX was developed by Hàn Thế Thành, incorporating the ability to create PDF output directly into the TeX engine. Although originally developed in parallel, pdfTeX today includes the ε-TeX extensions as-standard. Development of pdfTeX has mostly stopped: changes are largely focussed on bug fixes. However, the central importance of pdfTeX as the “standard” engine in modern TeX distributions means further changes may be seen, particularly in the area of accessibility.
pdfTeX started as a topic for Hàn Thế Thành’s Master’s thesis, and seems first to have been published in TUGboat 18(4), in 1997 (though it was certainly discussed at the TUG’96 conference in Russia). While the world was making good use of “pre-releases” of pdfTeX, Thành used it as a test-bed for the micro-typography which was the prime subject of his Ph.D. research. Since Thành was finally awarded his Ph.D., day-to-day maintenance and development of pdfTeX 1.0 (and later) has been in the hands of a group of pdfTeX maintainers (which includes Thành); the group has managed to maintain a stable platform for general use.
XeTeX
XeTeX, originally developed by Jonathan Kew, a Unicode TeX
engine which is able to load system fonts using the HarfBuzz library, which
is accomplished by an extended syntax to the \font primitive. It builds on
ε-TeX and also adds a range of new primitives useful
for working with non-Latin languages. Otherwise, the process of typesetting is
essentially the same as TeX’s. Like pdfTeX, major developments are
not currently ongoing with the XeTeX codebase,
though bug fixes and cross-compatibility with other engines do continue.
LuaTeX
As detailed above, development of pdfTeX is “in essence” complete.
As pdfTeX development ran down, development of a new system, LuaTeX was started.
Lua is a interpreter designed to be incorporated into
other applications. LuaTeX consists of a TeX-like engine with a Lua interpreter
“embedded” in it; the lua interpreter has access to many of the data
structures used for typesetting, so that the programmer may also interpolate
chunks of Lua code into their (La)TeX macros, or as “call-backs” for use when
the TeX-like engine does certain operations.
This arrangement offers the prospect of a “semi-soft” typesetting engine: it will have its basic behaviour, but the user gets to redefine functionality if an idea occurs — there will be no need to persuade the world first, and then find a willing developer to work on the sources of of the distribution.
The LuaTeX project has pursued avenues that many of other projects have had in their sights, notably Unicode character representations and support for OpenType fonts. This included incorporating the extensions pioneered by Aleph. Current LuaTeX (v1.0.8) is considered close to feature-complete by the development team: at the time of writing, the possibility of a v2 development path has been suggested.
pTeX
pTeX is an old Japanese-specific extension of TeX82, which aims to support proper typesetting of Japanese text but only supports a limited character set, JIS X 0208 (6879 characters).
upTeX is developed as an extension of pTeX to support full Unicode characters. It also includes extensions to overcome the difficulties of pTeX in processing 8-bit Latin characters due to conflicts with legacy multibyte Japanese encodings.
𝜀-pTeX and 𝜀-upTeX are 𝜀-TeX extensions of pTeX and upTeX respectively. In the current release, some extensions derived from pdfTeX and Ω are also available.
Text taken from the Guide to pTEX for developers unfamiliar with Japanese.
Historical projects
ε-TeX
As part of the work of the NTS project,
a number of extensions to Knuth’s TeX were developed. Many of these
support programming in a way which makes tasks easier than in Knuth’s TeX,
or indeed possible at all. Others extend the range of available registers
in TeX or make more internal information available. Current TeX distributions
are delivered with the ε-TeX extensions available in almost
all formats other than tex (Knuth’s TeX) itself.
ExTeX
The ExTeX project was building on the experience of the many existing TeX development and extension projects, to develop a new TeX-like system. The system was developed in Java.
ExTeX was designed to accept Unicode input, and to implement all of TeX’s primitives, but with some marked as obsolete, and “modern” alternatives provided. Desirable extensions from ε-TeX, pdfTeX and Ω were identified for incorporation.
Omega and Aleph
Omega was developed as an extension of TeX, to use with multilingual texts, expressed in a variety of input encodings. Omega used 16-bit, Unicode-encoded, characters. It provided many innovative concepts, notably including the “translation process” that takes a character stream and transforms it according to various processes that may be internally specified, or be a separate program.
While Omega showed a lot of promise at its mid-1990s announcement, progress was slow, and development was essentially dead by the time that one of the original developers withdrew (taking with him a bunch of research students).
Before that distressing event, a separate thread of development had started, to produce a program called Aleph, which merged the facilities of ε-TeX into a stable Omega codebase and added other extensions. Aleph also proved an attractive platform for many people; but its development, too, has dried up.
Some of the work from these projects has been incorporated into LuaTeX.
New Typesetting System (NTS)
The NTS project set out to produce an advanced replacement for TeX, to provide a basis for developing such modifications: this “New Typesetting System” would share Knuth’s aims, but would implement the work in a modern way taking account of the lessons learned with TeX. While a first demonstrator NTS did appear, it wasn’t practically useful, and the project seems no longer active.
In parallel with this work, the NTS developed a set of extensions that can be used with a (“true”) TeX system. Such a modified system is known as an ε-TeX system, and the concept has proved widely successful.
FAQ ID: Q-enginedev
Tags: pdftex–luatex–xetex
Last updated: 2018-05-24