I6 Template Layer

Extent Of Long Block.

When there's a long block, we need enough of the entries to store the number of characters, plus one for the null terminator.

Character Set.

On the Z-machine, we use the 8-bit ZSCII character set, stored in bytes; on Glulx, we use the opening 16-bit subset of Unicode (which though only a subset covers almost all letter forms used on Earth), stored in half-words.

The Z-machine does have very partial Unicode support, but not in a way that can help us here. It is capable of printing a wide range of Unicode characters, and on a good interpreter with a good font (such as Zoom for Mac OS X, using the Lucida Grande font) can produce many thousands of glyphs. But it is not capable of printing those characters into memory rather than the screen, an essential technique for texts: it can only write each character to a single byte, and it does so in ZSCII. That forces our hand when it comes to choosing the indexed-text character set.

KOV Support.

See the BlockValues.i6t segment for the specification of the following routines. Because no block values are ever stored in a text, they can freely be bitwise copied or forgotten, which is why we need do nothing special to copy or destroy a text.

Debugging.

This shows the various forms a text's short block can take:

Creation.

A newly created text is a two-word short block with no long block, like this:

Copy Short Block.

When a short block for a constant is copied, the new copy isn't a constant any more.

Transmutation.

What happens if a text is stored in packed form, but we need to access or change its individual characters? The answer is that we have to "transmute" it into long block form. Sometimes this is a permanent change, but often it's only temporary, and will soon be followed by an un-transmutation.

Mutability.

That neatly handles the question of how to make a text mutable. (Note that constants are never created in unpacked form.)

Casting.

In general computing, "casting" is the process of translating data in one type into semantically equivalent data in another: the only interesting cast here is that a snippet can be turned into a text.

Data Conversion.

We use a single routine to handle two kinds of format translation: a packed I6 string into an unpacked text, or a snippet into an unpacked text.

In each case, what we do is simply to print out the value we have, but with the output stream set to memory rather than the screen. That gives us the character by character version, neatly laid out in an array, and all we have to do is to copy it into the text and add a null termination byte.

What complicates things is that the two virtual machines handle printing to memory quite differently, and that the original text has unpredictable length. We are going to try printing it into the array TEXT_TY_Buffers, but what if the text is too big? Disastrously, the Z-machine simply writes on in memory, corrupting all subsequent arrays and almost certainly causing the story file to crash soon after. There is nothing we can do to predict or avoid this, or to repair the damage: this is why the Inform documentation warns users to be wary of using text with large strings in the Z-machine, and advises the use of Glulx instead. Glulx does handle overruns safely, and indeed allows us to dynamically allocate memory as necessary so that we can always avoid overruns entirely.

In either case, though, it's useful to have TEXT_TY_BufferSize, the size of the temporary buffer, large enough that it will never be overrun in ordinary use. This is controllable with the use option "maximum indexed text length".

Z Version.

The two versions of this routine, one for each virtual machine, are in all important respects the same, but there are enough fiddly differences that it's clearer to give two definitions, so:

Glulx Version.

Comparison.

This is more or less strcmp, the traditional C library routine for comparing strings, but it does pose a few interesting questions. The answers are:

(a) Two different unexpanded texts with substitutions are never equal, so "[X]" and "[Y]" aren't equal as texts even if X and Y are equal. (b) Otherwise we test the current value of the text as expanded, so "[X]" and "17" can be equal as texts if X is 17.

Hashing.

This calculates a hash value for the string, using Bernstein's algorithm.

Printing.

Unicode is not the native character set on Glulx: it came along as a late addition to Glulx's specification. The deal is that we have to explicitly tell the Glk interface layer to perform certain operations in a Unicode way; if we simply perform print (char) ch; then the character ch will be printed in ZSCII rather than Unicode.

Capitalised printing.

It turns out to be useful to have a variation on this:

Serialisation.

Here we print a serialised form of a text which can later be used to reconstruct the original text. The printing is apparently to the screen, but in fact always takes place when the output stream is a file.

The format chosen is a letter "S" for string, then a comma-separated list of decimal character codes, ending with the null terminator, and followed by a semicolon: thus S65,66,67,0; is the serialised form of the text "ABC".

Unserialisation.

If that's the word: the reverse process, in which we read a stream of characters from a file and reconstruct the text which gave rise to them.

Substitution.

Perishability.

As noted above, a perishable constant is one which must be expanded before the values it refers to vanish from existence.

Recognition-only-GPR.

An I6 general parsing routine to look at words from the position marker wn in the player's command to see if they match the contents of the text txt, returning either GPR_PREPOSITION or GPR_FAIL according to whether a match could be made. This is used when the an object's name is set to include one of its properties, and the property in question is a text: "A flowerpot is a kind of thing. A flowerpot has a text called pattern. Understand the pattern property as describing a flowerpot." When the player types EXAMINE STRIPED FLOWERPOT, and there is a flowerpot in scope, the following routine is called to test whether its pattern property – a text – matches any words at the position STRIPED FLOWERPOT. Assuming a pot does indeed have the pattern "striped", the routine advances wn by 1 and returns GPR_PREPOSITION to indicate a match.

This kind of GPR is called a "recognition-only-GPR", because it only recognises an existing value: it doesn't parse a new one.

Blobs.

That completes the compulsory services required for this KOV to function: from here on, the remaining routines provide definitions of text-related phrases in the Standard Rules.

What are the basic operations of text-handling? Clearly we want to be able to search, and replace, but that is left for the segment RegExp.i6t to handle. More basically we would like to be able to read and write characters from the text. But texts in I7 tend to be of natural language, rather than containing arbitrary material – that's indeed why we call them texts rather than strings. This means they are likely to be punctuated sequences of words, divided up perhaps into sentences and even paragraphs.

So we provide facilities which regard a text as being an array of "blobs", where a "blob" is a unit of text. The user can choose whether to see it as an array of characters, or words (of three different sorts: see the Inform documentation for details), or paragraphs, or lines.

Blob Access.

The following routine runs a small finite-state-machine to count the number of blobs in a text, using any of the above blob types (except REGEXP_BLOB, which is used for other purposes). If the optional arguments ctxt and wanted are supplied, it also copies the text of blob number wanted (counting upwards from 1 at the start of the text) into the text ctxt. If the further optional argument rtxt is supplied, then ctxt is instead written with the original text txt as it would read if the blob in question were replaced with the text in rtxt.

Get Blob.

The front end which uses the above routine to read a blob. (Note that, for efficiency's sake, we read characters more directly.)

Replace Blob.

The front end which uses the above routine to replace a blob. (Once again, characters are handled directly to avoid incurring all that overhead.)

Replace Text.

This is the general routine which searches for any instance of ftxt, as a blob, in txt, and replaces it with the text rtxt. It works on any of the above blob-types, but two cases are special: first, if the blob-type is CHR_BLOB, then it can do more than search and replace for any instance of a single character: it can search and replace any instance of a substring, so that ftxt is not required to be only a single character. Second, if the blob-type is the special value REGEXP_BLOB then ftxt is interpreted as a regular expression rather than something literal to find: see RegExp.i6t for what happens next.

Character Length.

When accessing at the character-by-character level, things are much easier and we needn't go through any finite state machine palaver.

Get Character.

Characters in a text are numbered upwards from 1 by the users of this routine: which is why we subtract 1 when reading the array in the block-value, which counts from 0.

Casing.

In many programming languages, characters are a distinct data type from strings, but not in I7. To I7, a character is simply a text which happens to have length 1 – this has its inefficiencies, but is conceptually easy for the user.

TEXT_TY_CharactersOfCase(txt, case) determines whether all the characters in txt are letters of the given casing: 0 for lower case, 1 for upper case. In the case of ZSCII, this is done correctly handling all of the European accented letters; in the case of Unicode, it follows the Unicode standard.

Note that there is no requirement for txt to be only a single character long.

Change Case.

We set ctxt to the text in txt, except that all the letters are converted to the case given (0 for lower, 1 for upper). The definition of what is a "letter", what case it has and what the other-case form is are as specified in the ZSCII and Unicode standards.

Concatenation.

To concatenate two texts is to place one after the other: thus "green" concatenated with "horn" makes "greenhorn". In this routine, from_txt would be "horn", and is added at the end of to_txt, which is returned in its expanded state.

When the blob type is REGEXP_BLOB, the routine is used not for simple concatenation but to handle the concatenations occurring when a regular expression search-and-replace is going on: see RegExp.i6t.

Setting the Player's Command.

In effect, the text typed most recently by the player is a sort of text already, though it isn't in text format, and doesn't live on the heap.