I6 Template Layer

Multiple Blocks.

Some of the data we want to store will be fixed in size, but some of it will need to expand or contract. The latter can change unpredictably in size and might at any point overflow their current storage, so they're stored in a doubly linked list of blocks. The pointer to such a flexible-length array is by definition the pointer to the block heading this linked list. For instance, the data in a text

might be stored in a list of blocks like so:

Note that the unique pointer to BN2 is the one in the header of the BN block. When we need to grow such a text, we add additional blocks; if the text should shrink, blocks at the end can at our discretion be deallocated. If the entire text should be deallocated, then all of the blocks used for it are deallocated, starting at the back and working towards the front.

A multiple-block is one whose flags byte contains the BLK_FLAG_MULTIPLE. This information is redundant since it could in principle be deduced from the kind of value stored in the block, which is recorded in the -->BLK_HEADER_KOV word, but that would be too slow. BLK_FLAG_MULTIPLE can never change for a currently allocated block, just as it can never change its KOV.

A multiple-block header is longer than that of an ordinary block, because it contains two extra words: -->BLK_NEXT is the next block in the doubly-linked list of blocks representing the current value, or NULL if this is the end; -->BLK_PREV is the previous block, or NULL if this is the beginning. The need to fit these two extra words in means that the data section is deferred, and so for a multiple-block data begins at the byte offset BLK_DATA_MULTI_OFFSET rather than BLK_DATA_OFFSET.

The Heap.

Properly speaking, a "heap" is a specific kind of structure often used for managing uneven-sized or unpredictably changing data. We use "heap" here in the looser sense of being an amorphous-sized collection of blocks of memory, some free, others allocated; our actual representation of free space on the heap is not a heap structure in computer science terms. (Though this segment could easily be rewritten to make it so, or to adopt any other scheme which might be faster.) The heap begins as a contiguous region of memory, but it need not remain so: on Glulx we use dynamic memory allocation to extend it.

For I7 purposes we don't need a way to represent allocated memory, only the free memory. A block is free if and only if it has -->BLK_HEADER_KOV equal to 0, which is never a valid kind of value, and also has the multiple flag set. We do that because we construct the whole collection of free blocks, at any given time, as a single, multiple-block "value": a doubly linked list joined by the -->BLK_NEXT and < – BLK_PREV.

A single block, at the bottom of memory and never moving, never allocated to anyone, is preserved in order to be the head of this linked list of free blocks. This is a 16-byte (i.e., n=4) block, which we format when the heap is initialised in HeapInitialise(). Thus the heap is full if and only if the -->BLK_NEXT of the head-free-block is NULL.

So far we have described a somewhat lax regime. After many allocations and deallocations one could imagine the list of free blocks becoming a very long list of individually small blocks, which would both make it difficult to allocate large blocks, and also slow to look through the list. To ameliorate matters, we maintain the following invariants:

(a) In the free blocks list, B-->BLK_NEXT is always an address after B; (b) For any contiguous run of free space blocks in memory (excluding the head-free-block), taking up a total of T bytes, the last block in the run has size 2ⁿ where n is the largest integer such that 2ⁿ ≤ T.

For instance, there can never be two consecutive free blocks of size 128: they would form a "run" in the sense of rule (b) of size T = 256, and when T is a power of two the run must contain a single block. In general, it's easy to prove that the number of blocks in the run is exactly the number of 1s when T is written out as a binary number, and that the blocks are ordered in memory from small to large (the reverse of the direction of reading, i.e., rightmost 1 digit first). Maintaining (b) is a matter of being careful to fragment blocks only from the front when smaller blocks are needed, and to rejoin from the back when blocks are freed and added to the free space object.

Initialisation.

To recap: the constant MEMORY_HEAP_SIZE has been predefined by the NI compiler, and is always itself a power of 2, say 2ⁿ. We therefore have 2ⁿ + 2⁴ bytes available to us, and we format these as a free space list of two blocks: the 2⁴-sized "head-free-block" described above followed by a 2ⁿ-sized block exactly containing the whole of the rest of the heap.

Net Free Space.

"Net" in the sense of "after deductions for the headers": this is the actual number of free bytes left on the heap which could be used for data. Note that it is used to predict whether it is possible to fit something further in: so there are two answers, depending on whether the something is multiple-block data (with a larger header and therefore less room for data) or single-block data (smaller header, more room).

Make Space.

The following routine determines if there is enough free space to accommodate another size bytes of data, given that it has to be multiple-block data if the multiple flag is set. If the answer turns out to be "no", we see if the host virtual machine is able to allocate more for us: if it is, then we ask for 2^m further bytes, where 2^m is at least size plus the worst-case header storage requirement (16 bytes), and in addition is large enough to make it worth while allocating. We don't want to bother the VM by asking for trivial amounts of memory.

This looks to be more memory than is needed, since after all we've asked for enough that the new data can fit entirely into the new block allocated, and we might have been able to squeeze some of it into the existing free space. But it ensures that heap invariant (b) above is preserved, and besides, running out of memory tends to be something you don't do only once.

(The code below is a refinement on the original, suggested by Jesse McGrew, which handles non-multiple blocks better.)

Block Allocation.

Now for the Flex routines. Those with names ending in Internal are private and should only be called by other Flex routines. Even the public ones must be used with care, or memory leaks or crashes will occur.

The routine FlexAllocate(N, K, F) allocates a block with room for size net bytes of data, which will have kind of value K and with flags F. If the flags include BLK_FLAG_MULTIPLE, this may be either a list of blocks or a single block. It returns either the address of the block or else throws run-time problem message and returns 0.

If it does succeed and return a nonzero address, then the caller must be able to guarantee that FlexFree will later be called, exactly once, on this address. In other words, FlexAllocate and FlexFree behave somewhat like C's malloc and free routines, with all the advantages and hazards that implies.

In allocation, we try to find a block which is as close as possible to the right size, and we may have to subdivide blocks: see case II below. For instance, if a block of size 2ⁿ is available and we only need a block of size 2^k where k<n then we break it up in memory as a sequence of blocks of size 2^k, 2^k, 2^k+1, 2^k+2, ..., 2^n-1: note that the sum of these sizes is the 2ⁿ we started with. We then use the first block of size 2^k. To continue the comparison with binary arithmetic, this is like a subtraction with repeated carries:

10000000₂ - 00001000₂ = 01111000₂

Errors.

In the event that FlexAllocate returns 0, the caller may not be able to survive, so the following is provided as a standardised way to halt the virtual machine.

Merging.

Given a free block block, find the maximal contiguous run of free blocks which contains it, and then call FlexRecutInternal to recut it to conform to invariant (b) above.

Recutting.

Given a segment of the free block list, containing blocks known to be contiguous in memory, we recut into a sequence of blocks satisfying invariant (b): we repeatedly cut the largest 2^m-sized chunk off the back end until it is all used up.

Deallocation.

As noted above, FlexFree must be called exactly once on each nonzero pointer returned by FlexAllocate.

There are two complications: first, when we free a multiple block we need to free all of the blocks in the list, starting from the back end and working forwards to the front – this is the job of FlexFree. Second, when any given block is freed it has to be put into the free block list at the correct position to preserve invariant (a): it might either come after all of the currently free blocks in memory, and have to be added to the end of the list, or in between two, and have to be inserted mid-list, but it can't be before all of them because the head-free-block is kept lowest in memory of all possible blocks. (Note that Glulx can't allocate memory dynamically which undercuts the ordinary array space created by I6: I6 arrays fill up memory from the bottom.)

Certain blocks outside the heap are marked as "resident" in memory, that is, are indestructible. This enables Inform to compile constant values.

Resizing.

A block which has been allocated, but not yet freed, can sometimes have its data capacity changed by FlexResize.

When the data being stored stretches or shrinks, we will sometimes need to change the size of the block(s) containing the data – though not always: we might sometimes need to resize a 1052-byte text to a 1204-byte text and find that we are sitting in a 2048-byte block in any case. We either shed blocks from the end of the chain, or add new blocks at the end, that being the simplest thing to do. Sometimes it might mean preserving a not very efficient block division, but it minimises the churn of blocks being allocated and freed, which is probably good.

Block Size.

These two routines are provided for the use of the BlockValue routines only.

Debugging Routines.

These two routines are purely for testing the above code.