To show you how to make use of Statice in your programs, we present this tutorial of example programs. The tutorial is designed to be read sequentially. Within the tutorial you will see cross-references to more detailed documentation, but we recommend that you go through the tutorial in sequence, and postpone the crossreferences until later. However, feel free to read it in whatever style suits you best.
We start with a simple example that demonstrates the basic concepts and facilities of Statice. The explanations are sketchy, designed to give you an overall idea of what Statice is.
In this example we define a bank database. A bank database is made up of accounts, and each account has a name and a balance. The following forms define the schema:
(define-schema bank (account))
(define-entity-type account ()
((name string :inverse account-named :unique t)
(balance integer)))
The statice:define-schema form defines a schema named bank, and says that it has one entity type, named account. The statice:define-entity-type form defines the entity type named account, and says that it has two attributes, named name and balance. The name attribute is declared to be unique, which means that no two accounts can have the same name.
statice:define-entity-type is analogous to defstruct or defflavor, in that it defines a new type. The attributes of an entity are analogous to slots or instance vari- ables. The main difference between entity types and types defined by defstruct and defflavor is that entity types reside not in a Lisp world, but in a Statice database. Another difference is that each attribute has a type. In this case, account names are strings, and account balances are integers.
The statice:define-entity-type form automatically defines accessor functions (for accessing the value of an entity's attributes) and an entity constructor function (for creating new entities of this entity type). We show examples of using these functions later on in this section.
We need to state where the bank database will be stored. Every Statice database is stored in a file in a Statice File System. The Statice File System might be on your own host or on another host on the network. The following form defines a variable to hold a pathname, which we will use to indicate where the database is stored. This pathname is different from others because beet is not a host name, but the name of a Statice File System.
(defvar *bank-pathname* #p"beet:>finance>bank")
Below, we define make-bank-database. This function creates a new database in the place specified by the pathname, and initializes it to be a bank database with no accounts.
(defun make-bank-database ()
(make-database *bank-pathname* 'bank))
When you define an entity type, statice:define-entity-type automatically defines an entity constructor function. For example, make-account is the entity constructor function for making new account entities. Below we define make-new-account, which makes a new account in the bank database by calling make-account, the entity constructor.
(defun make-new-account (new-name new-balance)
(with-database (db *bank-pathname*)
(with-transaction ()
(make-account :name new-name :balance new-balance))))
Notice that the definitions of make-new-account (above) and the definition of deposit-to-account (below) use statice:with-database and statice:with-transaction when accessing the database.
The statice:with-database form is analogous to with-open-file. It opens the database (if necessary) and makes this database the current database during the execution of its body.
Every operation that examines or modifies a database must be done within the dy- namic extent of a statice:with-transaction form. statice:with-transaction delimits a single transaction on the database. A transaction is a group of operations on a database, with the following properties:
To read information about an entity in the database, we use the reader functions that were defined automatically by statice:define-entity-type form. There is a reader for each attribute of an entity. In the bank example, the readers are called account-name and account-balance.
To write information about an entity in the database, we use the writer functions that are defined automatically. To call a writer function, use setf with the reader function. In the bank example, you can use setf with account-name and account-balance.
Readers and writers are called accessor functions.
The statice:define-entity-type form for bank also defines an inverse reader function called account-named, which is the inverse of account-name: given a name, it returns the account entity. (Inverse reader functions are not defined by default; you can request them by using the :inverse attribute option, as we did when we defined the bank entity type.)
deposit-to-account adds to someone's balance. First, account-named finds the account entity from the name provided. Then account-balance accesses and updates the balance of the account.
(defun deposit-to-account (name amount)
(incf (account-balance (account-named name))
amount))))
transfer-between-accounts moves a specified amount from one account into another account. If there are insufficient funds in the from account, it signals an error.
(defun transfer-between-accounts (from-name to-name amount)
(with-database (db *bank-pathname*)
(with-transaction ()
(decf (account-balance (account-named from-name)) amount)
(incf (account-balance (account-named to-name)) amount)
(when (minusp (account-balance (account-named from-name)))
(error "Insufficient funds in ~A's account" from-name)))))
bank-total computes the total of all accounts in the database. It uses a special form called statice:for-each, which successively binds the variable a to each account entity.
(defun bank-total ()
(with-database (db *bank-pathname*)
(with-transaction ()
(let ((result 0))
(for-each ((a account))
(incf result (account-balance a)))
result))))
We could have written this program using structures or instances to represent bank accounts, but by using Statice we gain two key advantages.
The use of transactions provides further benefits. Actions performed within a transaction are:
If you call bank-total while some other process is calling transfer-between-accounts, you'll get a correct total, because bank-total won't " see " the state in between the decf and the incf.
A Statice database holds a set of entities. Each entity represents some thing or concept in the real world. Every entity has a type, called its entity type. In the bank example, there is one entity type, named account.
The first thing in the bank example is a statice:define-schema form.
(define-schema bank (account))
A schema is a description of everything that can appear in a database; this de- scription consists of a list of the entity types. In the bank example, we define a schema named bank, and say that all the entities in the database are of entity type account. The symbol bank is called the schema name. In other words, a bank database contains account entities.
The second thing in the bank example is a statice:define-entity-type form, which defines the account entity type.
(define-entity-type account ()
((name string :inverse account-named :unique t)
(balance integer)))
An entity type can inherit from other entity types. The list following the entity type name includes those entity types that this one inherits from. In the bank example, account doesn't inherit from anything.
Next is a list of descriptions of the attributes of the account entity type. Attributes are used to represent properties of entities and relationships between entities. Each attribute has a name and a type. In the bank example, there are two attributes. The attribute named name has type string, and the attribute named balance has type integer. This means that each account has a name, which is a string, and a balance, which is an integer. Attribute types are always presentation types, but only some presentation types are allowed.
statice:define-entity-type automatically defines accessor functions for each attribute. In the bank example, there are two reader functions, account-name and account-balance. There are two corresponding writer functions, which you call by using setf with account-name and account-balance. statice:define-entity-type also automatically defines an entity constructor function, used to create new entities of this entity type. In the bank example, the entity constructor function is make-account.
After the name and type of an attribute comes a set of attribute options, expressed as alternating keywords and values. In the bank example, the attribute name has two options. The :inverse option defines an inverse reader function named account-named. The :unique option says that only one account in the database can have a particular name.
After defining the schema, the bank example makes a database.
Every Statice database is stored in its own file in a special kind of file system called a Statice file system. This defvar form defines a dynamic variable that holds the pathname of the particular database we are working with:
(defvar *bank-pathname* #p"beet:>finance>bank")
The value of the variable *bank-pathname* is a pathname to a file on a Statice file system; we call this kind of pathname a database pathname, because it indicates the location of a Statice database. The host component of the pathname is beet, but beet is the name of a Statice file system rather the name of a host. The functions in the program use *bank-pathname* as an implicit argument to specify the database. Notice that each statice:with-database form uses this pathname to refer to the database.
If we were working with more than one bank, and each bank had its own database, we would change the value of the *bank-pathname* variable from time to time as we changed our attention from one bank to another. The program could have also been written by having each function take a pathname as an explicit argument.
To find out where beet resides, Statice consults the namespace system, looking for a namespace object of type file-system named beet. This namespace object says what actual host to use, along with other information. It would be possible to move the beet file system from one host to another, using tapes or disk packs, without modifying our example program.
Pathnames are used to name databases within a Statice File System. The path- names are hierarchical, with component names separated by ">" characters, as they are in LMFS. Unlike in LMFS, there are no file types or file versions, just file names. In the bank example, the directory is >finance> and the name is bank. Many familiar Genera commands can be used with database pathnames, such as Show Directory, Create Directory, Rename File, and Delete File. Genera's Dired and File System Editor tools can also be used with database pathnames. Relative pathnames and wildcard pathnames work the same way as for LMFS pathnames. However, it's not possible to open database pathnames, because they refer to Stat- ice databases rather than files. So Copy File, Edit File, and Show File don't work on database pathnames.
Next, the bank example defines make-bank-database, which calls the Statice function statice:make-database to actually create a new database.
(defun make-bank-database ()
(make-database *bank-pathname* 'bank))
statice:make-database takes as arguments the pathname and the name of the schema for the new database. It makes a new database and copies the schema into the database. The newly created database contains the schema, but no entities. That is, the database is set up so that it can hold entities of type account, but there aren't any accounts yet.
In the bank example, the form statice:with-database surrounds every reference to the database. statice:with-database does the following:
The first time a database is used by a Lisp environment, Statice opens it. Once it has been opened, it stays open, and need not be opened again until Lisp is cold- booted. Opening happens automatically; the only thing you'll notice is a pause the first time statice:with-database is used. statice:make-database also opens the database it makes, so if you were to run the bank example, even the first usage of statice:with-database would not have to open the database. (There is no need to close a database.)
In the bank example, the Lisp variable db is bound to a Lisp object called a database instance that represents the database. Database instances are used as arguments to various Statice functions. They are needed only by programs that refer to two different databases at the same time. The bank example, like many real Statice applications, only uses one database, and so it never uses the db variable at all.
Whenever any Statice functions are used, there must be a current database. The current database is used as the default database by many Statice functions. This is why we don't need to use the db variable. statice:with-database binds the current database throughout the dynamic scope of its body. Dynamic scope means that if the body calls another function, the same database is still current while that function runs, so the called function doesn't need to use statice:with-database.
Operations on Statice databases are grouped together into transactions. Before any operation can be performed on a database, a transaction must be begun. A transaction terminates either successfully or unsuccessfully. When a transaction terminates successfully, we say it commits. When a transaction terminates unsuccessfully, we say it aborts. (Opening a database is not considered an operation on the database, so it need not be done within a transaction.)
In the bank example, the form statice:with-transaction surrounds every reference to the database. A transaction begins when the statice:with-transaction special form is entered. If the statice:with-transaction form returns to its caller, the transaction commits. If the statice:with-transaction form exits abnormally, due to a throw or a return through the statice:with-transaction form, the transaction aborts. Anything else that unwinds the stack, such as the killing of the process, also causes the transaction to abort.
statice:with-transaction has dynamic scope. The empty list in the statice:with-transaction form is for keyword options and values, which are rarely used.
The group of operations performed within a transaction are performed atomically. If the transaction commits, all of its effects take place; if the transaction aborts, none of them take place. For example, if a transaction begins, and executes some operations that modify values in the database, and then the transaction aborts, the modifications are undone and the database is left in its original state. In the bank example, the benefits of atomic transactions can be seen in the transfer-between-accounts function. Because the two accounts are modified within a single transaction, we can be sure that either the amount will be moved from one account to the other, or else nothing will happen. The total amount in all accounts is guaranteed to be stay the same; the database as a whole remains consistent.
In general, the atomic property of transactions prevents databases from being left in inconsistent, intermediate states. A transaction that modifies a database takes the database from one consistent state to another consistent state. Of course, the meaning of " consistent " depends on the application. In the bank example, consistency means that no amount enters or leaves the database due to a transfer between accounts.
Statice allows concurrent access to databases: more than one process can access a database at the same time. The processes might be on the same host or on different hosts. Transactions are used to keep these processes out of each other's way. The operations done in a transaction are isolated from all other transactions. This means that no transaction in progress is ever aware of the effects of another transaction in progress. In fact, transactions let Statice applications disregard concurrency altogether, so you can write programs as if the database were reserved for yourself.
In the bank example, suppose there were two processes. The first process is running the transfer-between-accounts function, in a transaction we'll call T1. The second process is running the bank-total function, in a transaction we'll call T2. Now, suppose the operations of these two transactions happened to occur in the following order:
If this were allowed to happen, bank-total would return the wrong answer: it would be short by the amount being transferred. Transactions make sure that this cannot happen. The two transactions are isolated from each other, so T2 never observes the results of a transaction in progress. In this case, as soon as transaction T1 tried to modify the from-account, it would wait until transaction T2 terminates, and then proceed.
A system failure is any event that causes the entire system to stop, requiring a warm boot or cold boot. When a system failure occurs, Statice databases are not damaged. Any transactions that were in progress at the time of the failure are aborted, which means their effects are discarded. The database is left in a consistent state. In other words, the transactions are atomic even if there is a system crash in the middle of a transaction.
When a database is being used over a network, there is a server host that actually stores the database, and one or many client hosts that run Statice programs affecting the database. A system failure on a user host aborts all transactions being done by that host. A system failure on a server host aborts all transactions being done by any user host that involve databases on this server. In any case, all unfinished transactions are aborted, and the database is left consistent.
When a transaction commits, the results of the transaction are written into the database, and will be visible to any future transactions. As soon as a statice:with-transaction form returns, Statice guarantees that the changes made by that transaction are persistently stored, and will be remembered even if there is a system failure.
If you nest a statice:with-transaction dynamically within another statice:with-transaction form, and the outer one is aborted, then both the outer transaction and the inner one are aborted. Nothing is committed until the end of the outermost statice:with-transaction is reached.
If an error is signalled during a transaction, the normal Genera signalling mechanism begins: the lists of handlers are searched to find a handler for this error. The mere signalling of an error does not cause the transaction to abort. The transaction is aborted only if the error handler causes a throw to outside the scope of the statice:with-transaction.
If the error is not handled by any bound handler, default handler, or global handler, it causes the debugger to be invoked. If you then press ABORT, the debugger throws to the nearest restart handler, which is normally outside the scope of the statice:with-transaction. So, if an error reaches the debugger and you press ABORT, that normally aborts the transaction. In the bank example, if the " Insufficient funds " error is signalled, the debugger is entered, and the user will presumably abort the transaction.
But if the error is handled, the handler need not abort the transaction. For example, if there were a condition-case within the scope of a statice:with-transaction, and it handled a condition, the transaction would not be aborted.
The next function in the bank example is make-new-account.
(defun make-new-account (new-name new-balance)
(with-database (db *bank-pathname*)
(with-transaction ()
(make-account :name new-name :balance new-balance))))
make-new-account takes arguments called new-name and new-balance, which should be a string and an integer, respectively. After opening the database and starting a transaction, make-new-account calls the function make-account.
make-account is an entity constructor function. It makes a new entity of type account in the database. It also initializes the values of the name and balance attributes of the new entity to the values of the variables new-name and new-balance.
Entity constructor functions are defined automatically for each entity type. Their names are formed by prefixing the entity type name with make-. You can use the :constructor option to statice:define-entity-type to specify a different name for the constructor.
Constructors take keywords arguments, one for every attribute of the entity type. If a keyword is given with a value to the constructor, that value is stored as the value of the corresponding attribute. The names of the keywords are the same as the names of the attributes, but in the keyword package.
The value returned by an entity constructor function is a Lisp object called an entity handle. An entity handle is an object in the Lisp world that represents an entity in a Statice database. Lisp programs pass around and manipulate entity handles in order to refer to entities.
An entity handle is an instance of a flavor. For every entity type, Statice defines a flavor named the same as the entity type. In our example, there is a flavor named account, and the entity handle returned by make-account is an instance of the account flavor. Remember that flavors are considered type names by the Lisp type system, and so the entity handle is of the Lisp type account, just as the corresponding entity is of the Statice entity type account.
Entity handles preserve the identity of entities. That is, there is never more than one entity handle in a Lisp world for a given entity. If you have two entity handles and want to know whether they refer to the same entity, you can use eq to check. The entity type of an entity is fixed when the entity is created, and can never be changed. An entity stays the same type for its entire lifetime. An entity will not disappear from the database unless you explicitly delete it with statice:delete-entity.
The next function in the bank example is deposit-to-account. It adds a specified amount to the balance of the account with a specified name.
(defun deposit-to-account (name amount)
(with-database (db *bank-pathname*)
(with-transaction ()
(incf (account-balance (account-named name)) amount))))
To get the value of an attribute of an entity, you use a reader function, or reader. To set the value of an attribute of an entity, you use setf with the reader. The function that is called when you use setf with a reader is called a writer function, or writer. Both readers and writers are called accessor functions, because they enable you to access the value of an attribute for either reading or writing. Reader functions are defined automatically for every attribute of every entity type. The name of a reader is formed by concatenating the entity type name, a hyphen, and the attribute name. In the bank example, two readers are defined, account-name and account-balance.
Writers are also defined automatically. To call a writer, use the setf syntax with a reader, such as:
(setf (account-balance account ) new-value )
statice:define-entity-type offers four options that enable you to specify the name of readers and writers: :reader, :writer, :accessor, and :conc-name. Accessors are implemented as generic functions, which means you can write methods for them to specialize their behavior. For example, the reader account-balance is a generic function. Statice defines one method for it, on the account flavor. The argument to account-balance is an entity handle of type account. This reader returns the value of the balance attribute of the entity referred to by its entity handle argument. That is, account-balance takes an account and returns its balance, where the account is indicated by the entity handle for the account entity. The writer associated with account-balance is also a generic function with one method. Writers are implemented as setf generic functions. For information on defining methods for setf generic functions (also called setter functions).
The function account-named is an inverse reader function. Inverse readers are defined when you use the :inverse attribute option in the schema. You specify the name of the inverse reader; this is the value of the :inverse option. account-named takes a string argument, and returns the entity handle for the account entity whose name value is the same as the argument. The operation performed by account-named is the inverse of the operations performed by account-name: the former goes from a string to the corresponding entity handle, while the latter does the opposite.
Kind of Function Argument Value
Inverse Reader attribute's value entity handle
Reader entity handle attribute's value
We discuss inverse writer functions later:
How deposit-to-account Works
Now we look at how the function deposit-to-account works. It first opens the database and starts a transaction. It calls account-named to find the account; account-named returns an entity handle that refers to the desired account entity. It calls account-balance to read out the current balance, adds in the specified amount, and then writes the sum back into account-balance. Finally, the body of statice:with-transaction returns, and the transaction commits. The results are now stored in the database.
deposit-to-account shows why it's important that transactions are isolated from each other. Suppose there is an account named " George " with a balance of 100. Suppose that there were two concurrent transactions, each trying to deposit 10 into George's account. If everything works properly, there should be 120 in George's account when the two transactions complete. But suppose they took place this way:
If this could happen, George's balance would only be 110 even though both transactions seemed to do their job. But in Statice this cannot happen because transactions are isolated from each other. Statice guarantees that if these two transactions are run concurrently, the overall effect will the the same as if they had run separately, one after the other. That's why deposit-to-account must do both its reading and its writing within a single transaction.
The transfer-between-accounts function does the same kinds of things as deposit-to-account. Again, it's important that all the operations be performed within a single transaction, to assure that concurrent transactions don't interfere with each other.
Another important reason to use a single transaction is that transfer-between-accounts does two semantically related operations that write into the database. We must be sure that either both operations take place, or neither. The transaction assures us that the two operations will be done atomically, even if the system crashes, the process is killed, the user types c-m-ABORT, o r the error is signalled and leads to a throw.
The final function in the bank example is bank-total, a function that returns the sum of the balances of all the accounts in the database.
(defun bank-total ()
(with-database (db *bank-pathname*)
(with-transaction ()
(let ((result 0))
(for-each ((a account))
(incf result (account-balance a)))
result))))
In order to add up the balances of all accounts in the database, we need a way to find all accounts in the database. The statice:for-each special form lets us do this. In the bank-total function, statice:for-each establishes a variable called a, and binds a successively to entity handles for each account entity in the database. It runs the body once for each entity, and the body accumulates the sum of the balances.
The extra level of list structure in the syntax of statice:for-each, is needed because statice:for-each has many other capabilities. It can iterate over a selected subset of entities; it can iterate in sorted orders; and it can iterate over tuples of entities from different entity types. Here we see statice:for-each in its most basic form, in which it iterates over all entities of a given entity type.
There is an important difference between Lisp objects and Statice entities. Lisp objects are kept track of only if you save references to them. It is possible for a Lisp object to be unreferenced and unreachable. Such an object is called "garbage" and can be deallocated automatically.
In contrast, Statice keeps track of all entities in a database. It can always access any entity, by using statice:for-each on the entity type of the entity. As a result, no entities ever become garbage. Notice that make-new-account makes a new entity, but never "puts" it anywhere. In Lisp, if you make a new Lisp object and then just ignore it, it immediately becomes garbage. But in Statice, it's installed into a database and can always be found again.