     Initial desgn spec for Oregon VME58 and PC68 Motion Controller Driver

Issue 2


1. Requirements:

The driver should be simple, keeping complexity in the server.

Communications from the server to the driver will be via ioctl() calls.

There will be blocking and non-blocking ioctl calls.

Communications from the driver to the server will be via signals, or
returned values from blocking ioctl calls.

We must be able to read position while axes are in motion.

The driver source will work with VME58 and PC68 versions of the board.

The system should be able to have motion commands active on more
than one axis at a time.

The driver should exist as a loadable module, initially for 2.0 kernels.

The driver should handle multiple cards.

The driver must protect the card interface from user processes crashing or
exiting while a command is being issued.

The driver will work correctly with programs based on the Linux POSIX
thread library.


2. Oregon board theory of operation:

Commands are issued to the board as an ASCII string.  Some commands
are executed as they are issued, some are queued and executed in
sequence, and some are only actioned when a 'GO' command is sent.

There is a 'DONE' flag associated with each axis.  An interrupt can be
generated when a DONE flag is set.  Typically you queue a sequence of
motion commands, then issue a 'clear DONE and GO' command.  If you
queue a INTERRUPT DONE command after the GO, then you will get an
interrupt when the motion is complete.  There are commands which will
block all further processing on an axis until its command queue is
empty, which could be used to force a Read Position command to wait
until a motion command was complete, rather than executing
immediately.

As there is a single DONE flag per axis, it makes no sense to queue
multiple INTERRUPT DONE commands to the board without waiting for and
clearing the interrupts between commands.  It is perfectly valid to
queue a sequence of motion commands followed by a single INTERRUPT
DONE command.

There is no way to relate responses coming back from the board (eg. to
a Read Position command), to the commands which requested the
response, or to a particular axis.  This means there can only ever be
one response-generating command outstanding on the board at a given time.

Generally speaking, one requests that DONE be set on completion of a
motion command sequence.  It is not possible to generate an interrupt when
shared memory has been updated.  It would be possible to generate an
interrupt when a response-generating command is actioned, by queuing
commands to wait for the queue to be processed, and then to interrupt;
however, this would only work when no other interrupting command or motion
command was outstanding.  This means a driver must poll the shared memory
status flag, or the card response buffer, until the response is available.

The board can operate in Single-Axis, Multi-Axis, or All-Axis modes.
Oregon Micro Systems recommend using Multi-Axis mode, even when moving
only a single axis, if the board is used under a multitasking operating
system.


3. Design overview:

3.1 Command queuing:

Most commands from the server will be passed as ASCII strings, as
described in the Oregon manual.  This satisfies the requirement to
keep the complexity in the server.

The driver must pass commands to the board without waiting for earlier
commands to complete, so as to be able to read position while axes are
in motion.  Reading position could be handled via the shared memory on
the VME58, but the overall design must be suitable for PC68 boards also.

The board has a single 256 character input queue, from which the
microcontroller moves commands in to axis specific queues.  The axis
specific queues are 200 entries each, and a simple move takes only
two entries.  Commands which execute immediately (eg. Read Position)
are taken from the input queue and actioned.  They do not use space in
the axis specific queue.  The common command queue is character based,
so if you send "AX MR12345;GD ID" you have used 16 characters of
that buffer (14 without the spaces).  If, for example, you wanted
to queue two moves at different velocities, and then interrupt, you
would use rather more of the queue.  Given that the microcontroller
will move commands from the common queue as soon as it can, I think
it unlikely that we will be able to fill a 256 character queue in
normal use.

Ioctl calls fail if there was insufficient queue space at the time the
command was issued.  That should be OK except for very long sequences
(such as contour definitions) which exceeded 256 characters.  A future
enhancement could be a flag on the ioctl which said 'error if insufficient
queue space', and if you don't set the flag the driver blocks until there
is queue space.

3.2 Handling of response generating commands:

The basic Linux kernel tick is 10ms, and that limits the highest
rate that we can expect to service Read Position commands at (because the
driver cannot be woken up to poll more often than every tick).  The board
provides two methods of reading position information; via shared memory or
via a Read Position command.  In either case we can read the position of
all axes in one go.  the first iotcl requesting position information will
trigger a request to the board and a 10ms timer.  Subsequent requests will
just be queued.  Once the timer expires all requests will be satisfied
together from that one response.  Response generating commands of other
types will be queued within the driver, probably using a semaphore, and
executed sequentially (typically at 10ms intervals).

>From a performance viewpoint, this means it should be possible to read the
position of all axes, either from one process or several processes, 100
times per second.  This does depend on the system being sufficiently
lightly loaded that the user processes can be scheduled and issue a new
request within the 10ms window.  The VME58 board itself is reportedly able
to update its shared memory 1024 times per second, but it is assumed that
that would only be possible if the driver was in a tight loop polling the
status register and triggering a new update as soon as the previous was
complete.

As response generating commands should complete quickly, all such commands
have been implemented as blocking commands with an internal driver timeout
of 500ms.  A higher level ioctl request has been defined for Read
Position, which will free a simple position monitoring thread from having
to deal with OMS command/response strings.  It also makes it possible for
the driver to use shared memory access to read position, should it be
considered more appropriate.

It is the servers responsibility to ensure that it does not issue
response-generating commands as normal command strings.  The driver
does not analyse the strings sent from the server in order to police
that.  Should the server generate such a command, the responses to
subsequent response-generating commands would be corrupted.

Various schemes could be implemented to make position information
available more frequently than every 10ms; one such scheme under
investigation is to reduce the basic kernel tick from 10ms to 1ms.


3.3 ioctl types:

The server can issue normal commands with one of four qualifiers:

a) Send commands and signal when done (non-blocking)
b) Send commands and block until done
c) Send commands and return
d) Send commands and wait response string

Again the driver does not analyse the content of the commands; for cases
(a) and (b) the server should include an INTERRUPT DONE command in the
string it sends, and the driver will take the appropriate action once the
interrupt occurs.  If the server includes INTERRUPT DONE commands
in the strings it sends with case (c), this will result in interrupts
from the board that the driver is not expecting.  In that case the driver
will queue the DONE interrupt and handle it in the same way it handles
error interrupts (see below).  Case (d) is used for commands which generate
a response; such commands always block.


3.4 Device files:

The driver supports a single device file for each board (not one per
axis), and allows that device to be openned multiple times.  Command
sequences generated by the server must not make assumptions as to which
axis is currently being addressed, otherwise commands may be queued to the
wrong axis, depending on what other processes have been doing most
recently.  The simplest way round this is to prefix each command with a
single axis selection command, such as 'AX'.  However, it is recommended
by Oregon Micro Systems that in a multitasking environment Multi-Axis mode
is usually most appropriate.  In Multi-Axis mode one would replace 'AU
MR1000;' with 'AM MR,,,,1000;', and you could possibly omit the 'AM' if a
policy of always leaving the board in AM mode was adopted.



3.5 Interrupt sources:

There are several sources of interrupt within the board, as follows:

a) Encoder slip detected
b) Limit (overtravel) switch tripped
c) DONE flag set
d) I/O bits 0 or 1 active
e) Command error

Cases (a), (b), and (c) occur on a per-axis basis, while (d) and (e) are
on a per-board basis.  Until such time as a need is identified, interrupts
from (d) will not be enabled.  Command errors, (e), must be treated as
affecting all active axes, as command queuing means it is not possible to
tell which particular command was considered to be in error.  (a), (b) and
(e) are considered to be asynchronous error interrupts, and are picked up
either by the server polling for outstanding interrupts, or by the server
registering for a signal to be sent on interrupt.  (c), the DONE flag, is
handled by the driver for commands issued with blocking/signalling ioctls;
any DONE interrupts which did not result in a process being
woken/signalled can be picked up by the server polling.  Any attempt to
issue an interrupt generating command while any type of interrupt is
pending for the specified axis will result in an error.  The server should
then read and acknowledge outstanding interrupts before proceeding.


3.6 Driver/Server interaction:

The driver and server processes can interact in several different ways,
for both normal command completion, and error handling.  Commands
to the driver generally include a bit mask to indicate which axes they
should operate on.  For example, a server process might issue a blocking
ioctl with a qualifier which says 'send this and block until DONE on axis
Z'.  Alternatively a server might issue a poll request which says 'read
and acknowledge all interrupts on axes U and V'.  This scheme allows for
one process to drive all axes, or for a number of processes to drive one
or more axes each.  When issuing a 'signal when done' type of ioctl, the
server specifies whether the DONE flag should be cleared as a result of
the signal being sent, or not.  This allows a process to have multiple
events all generating the same signal, or to allocate a different signal
per event type.  If a different signal is allocated to each event then the
server can specify that the DONE flag be cleared when the signal is
generated, as it knows which event has occurred from the signal number.
If multiple events can result in the same signal (perhaps the server
controls four axes, and has them all signal completion with SIGPWR), then
the command should specify that DONE is not cleared; in the servers signal
handler it would then issue a 'read and acknowledge all interrupts' for
all relevant axes.

The driver always has interrupts enabled from the boards, and its
interrupt handler will queue any interrupts received.  This queue takes
the form of a byte of flags for each of Slip, Limit, Done, and Command
Error.  Each byte is a bit mask with a bit per axis (Command Error
interrupts result in the bits for all axes getting set).  DONE interrupts
will not be queued in the driver if it has been instructed to clear DONE
when signalling command completion.  For other occurances of DONE, and all
error interrupts, the driver sets the relevant flags.  If a server process
has registered for a signal to be sent on error for the relevant axes,
then a signal will be sent.  This signal does not result in the flags
being cleared; it is up to the server to issue a 'read and acknowledge'
with an appropriate axis mask to clear the flags.

It is important to understand that there are registers in the OMS board
which provide the current state of, for example, the limit switches, and
there are seperate flags within the driver which record interrupt
conditions from the card (eg. limit switch tripped) which have not yet
been passed to a server process.

The design of the driver is such that for a given axis there should
only ever be one process which has registered a signal handler to
receive errors.  This is because such information is registered
against the axis, and not against the open file descriptor.

The driver has a concept of an axis being busy, and will generally fail
interrupting command requests if so.  An axis becomes busy when a command
which should result in a DONE interrupt is issued (as determined by the
ioctl request type, and not by analysis of the command string).  An axis
remains busy until a DONE interrupt is generated by the board, even if the
command times out or the user process is killed.  A special blocking
request can be sent by error recovery code, which allows you to issue a
command to an axis even though the axis is busy.  See the section on error
recovery below.


4.  ioctl definitions:

4.1 Request/response structure:

Almost all ioctls require some parameter exchange beyond the ioctl request
type;  the following common structure is used for all such ioctls,
although not all fields are used in all cases:

typedef struct {
	unsigned char axes;	/* Bit 0 for axis X, 1 for Y, etc	*/
	unsigned char signal;	/* Signal number to wake caller		*/
	int timeout;		/* Timeout in seconds (0 to disable)	*/
	char *cmdstr;		/* ASCII command to issue to OMS board	*/
	int cmdlen;		/* Length of cmdstr			*/

	char *rspstr;		/* Buffer for any response string	*/
	int rsplen;		/* Length of response buffer		*/
	unsigned char f_slip;	/* Current slip flags			*/
	unsigned char f_limit;	/* Current limit flags			*/
	unsigned char f_done;	/* Current done flags			*/
	unsigned char f_home;	/* Current home flags			*/
	unsigned char f_cmderr;	/* Current command error flags		*/
	unsigned char io_low;	/* Current state of i/o bits 0-7	*/
	unsigned char io_high;	/* Current state of i/o bits 8-13	*/
	long positions[8];	/* Current axis positions		*/
	long *mem_params;	/* Pointer to 8*12 entry array for shared
				 * memory axis parameters		*/
} OMS_req_t, *OMS_req_ptr;

Locations in the positions[] array are always allocated the same way,
regardless of which axes you are requesting information on; for example,
the position of axis U is in positions[OMS_U_POS].

Response strings are always null terminated within the length of the
supplied buffer.  The f_* fields are used to report either the current OMS
board register values, or the driver queue of outstanding interrupts,
depending on the ioctl request issued.

The OMS_req_t structure is allocated by the user code, and a pointer to it
is passed as the third argument to the ioctl call.  The driver reads some
fields from it, and writes return data to it.  The structure references
other areas, such as the command string.  Those areas are also created
within the user space, by the user code.  For commands returning response
strings, the user code pre-allocates a response buffer and puts a pointer
to it in the OMS_req_t structure.


4.2 IOCTL types:

4.2.1  OMS_RESET

Reset the driver.  This will terminate outstanding commands, issue a KILL
to the card, and reinitialise data structures.  Later versions of the
driver may issue a RESET command to the board, but there are currently
problems with the card crashing on reset.

{
	OMS_req_t req;

	res = ioctl(fd, OMS_RESET, NULL);
}


4.2.2  OMS_REG_HANDLER

Register error signal handler.


4.2.3  OMS_READ_INTS

Read outstanding interrupt flags.  This leaves the interrupt flags set,
until an OMS_ACK_INTS is issued.  

{
	OMS_req_t req;

	memset(&req, 0, sizeof(OMS_req_t));
	req.axes = OMS_T_AXIS | OMS_U_AXIS;
	res = ioctl(fd, OMS_READ_INTS, &req);

	if (req.f_slip & OMS_T_AXIS)
		/* Slip interrupt queued */
		...
	/* similar for f_limit, f_done, and f_cmderr */
	/* Interrupts still queued */
}


4.2.4  OMS_ACK_INTS

Read and clear outstanding interrupt flags.  This is exactly the same as
OMS_READ_INTS, except that any interrupt flags returned to the user are
also cleared from the drivers queue.


4.2.5  OMS_READ_FLAGS

Read current slip/limit/done status (ie. the current value of the OMS
registers, rather than any outstanding unacknowledged interrupts queued by
the driver.  Also returns the current state of the user I/O bits.

{
	OMS_req_t req;

	memset(&req, 0, sizeof(OMS_req_t));
	req.axes = OMS_T_AXIS;
	res = ioctl(fd, OMS_READ_FLAGS, &req);

	if (req.f_limit)
		/* Limit switch for axis T currently tripped */
		...
	/* similar for f_done, f_home, f_slip, and f_cmderr */
	/* Current I/O bit status in f_io_lo and f_io_hi */
}

4.2.6  OMS_READ_POSITIONS

Read current position of all axes.

{
	OMS_req_t req;

	memset(&req, 0, sizeof(OMS_req_t));
	req.axes = 0xff;
	res = ioctl(fd, OMS_READ_POSITIONS, &req);

	current_t_position = req.positions[OMS_T_POS];	/* etc */
}

4.2.7  OMS_READ_AXIS_INFO

Read the shared memory parameters associated with all axes, in to buffer
referenced by mem_params.


4.2.8  OMS_CMD_BLK

Issue command and block until interrupt.  The supplied command string
must include some type of interrupting command (eg. Interrupt Done, ID).
The ioctl will not return until

{
	char cmd[80];
	int res;
	int steps_to_move = 33446;

	memset(&req, 0, sizeof(OMS_req_t));
	req.axes = OMS_T_AXIS;
	req.timeout = 2;
	sprintf(cmd, "at mr%d gd id", steps_to_move);
	req.cmdstr = cmd;
	req.cmdlen = strlen(cmd);
	res = ioctl(fd, OMS_CMD_BLK, &req);
}


4.2.9  OMS_CMD_SIG

Issue command and signal when done, do not clear DONE flag.  Non-blocking.


4.2.10 OMS_CMD_SIG_ACK

Issue command and signal when done, implicit acknowledge by clearing DONE
flag.


4.2.11 OMS_CMD_RET

Issue command and return.  Non-blocking.

{
	char cmd[80];
	int res;
	int max = 50000;	/* Max velocity for U, V, R and S */

	memset(&req, 0, sizeof(OMS_req_t));
	sprintf(cmd, "am vl,,,,%d,%d,%d,%d;", max, max, max, max);
	req.cmdstr = cmd;
	req.cmdlen = strlen(cmd);
	res = ioctl(fd, OMS_CMD_RET, &req);
}


4.2.12 OMS_CMD_RESP

Issue command and await response (always blocking).

{
	char cmd[80];
	char rsp[80];
	int res;

	memset(&req, 0, sizeof(OMS_req_t));
	sprintf(cmd, "wy");
	req.cmdstr = cmd;
	req.cmdlen = strlen(cmd);
	req.rspstr = rsp;
	req.rsplen = 80;
	res = ioctl(fd, OMS_CMD_RESP, &req);

	if (!res)
		printf ("Board type is '%s'\n", rsp);
}

4.2.13 OMS_DUMP

This is a debugging aid; it causes the driver to dump areas of its
workspace to the console.


4.2.14 OMS_DEBUG

This sets the debug reporting level of the driver; the higher the value
the more debug output is generated.

{
	ioctl(fd, OMS_DEBUG, (void *)2);
}


4.2.15 OMS_CMD_FORCE

This is the same as OMS_CMD_BLK, except that this command will be accepted
even when the axis is busy, or interrupts are queued.  It is intended for
use in error recovery only.


4.3 Error Codes

The OMS_CMD_* requests will fail if there are any outstanding DONE or
slip/limit/cmderr flags set for the relevant axes.  These flags must be
cleared with an OMS_ACK_INTS before a new OMS_CMD_* can be issued.  As
stated earlier, the drivers DONE flag will only be set as a result of an
OMS_CMD_SIG, or an OMS_CMD_RET with appropriate OMS command strings.
OMS_CMD_BLK and OMS_CMD_SIG* will fail if some other blocking or
signalling command is already outstanding.  Use OMS_CMD_FORCE to overcome
that for eror handling.

OMS_ERR_INV_FUN

This code is returned if an ioctl is issued with an invalid request type.

OMS_ERR_MULTI_AXES

A command has been issued with more than one bit set in req.axes, where
that is not allowed.  Commands which result in an interrupt (eg.
OMS_CMD_BLK) can only be active on a single axis, as the driver needs to
know which DONE flag to wait for.  Commands for reading flags, for
example, can specify multiple axes in one request.

OMS_ERR_AXIS_BUSY

This is returned if an interrupt generating command is issued while
another such command is active for the specified axis.

OMS_ERR_NIF

Non-implemented function, returned if you try to use some ioctl which has
not yet been implemented.

OMS_ERR_TIMEOUT

Returned if a command times out.  Response generating commands have a hard
coded timeout of 500ms.  OMS_CMD_BLK, _SIG, _SIG_ACK, and _FORCE can be
issued with a timeout, specified in seconds, via the req.timeout field.
Error recovery should be invoked following such a timeout, see below.

OMS_ERR_NO_AXES

Returned if req.axes is zero for a command where at least one axis should
be specified.

OMS_ERR_BAD_AXES

Invalid axis specification for the request type being issued.

OMS_ERR_DONE_PENDING

This error indicates that an earlier interrupt generating command has
terminated early (perhaps due to a timeout), but the DONE interrupt has not
yet been received.  This condition would normally be cleared by error
recovery code following the earler command failure.

OMS_ERR_RESP_OVERLOAD

There is a limit to the number of response generating commands the driver
can queue (8 in the initial release).  If this limit is reached then any
further OMS_CMD_RESP or OMS_READ_POSITIONS requests will fail with this
error.

OMS_ERR_TOO_LONG

The command string sent is too long for either the driver buffer or the
board input buffer.

OMS_ERR_POS_DATA

Returned if the driver is configured to read positions via ASCII commands,
and the response string from the board is invalid.

OMS_ERR_NO_SPACE

Returned if there is no space in the boards input buffer for the command
you are trying to issue.

OMS_ERR_INT_QUEUED

This is returned if you try and issue an interrupt generating command
while there is some unacknowledged interrupt queued in the driver for the
particular axis.  An OMS_ACK_INTS should be issued to clear any such
interrupts before retrying the command.


4.4 Error recovery

XXX This is messy at the moment and requires more thought and rework XXX

This section deals with error recovery, primarily OMS_ERR_TIMEOUT being
returned from interrupt generating commands.  In this case the DONE
interrupt may or may not occur round about the time you are attempting the
error recovery.  In addition, any attempt to issue further commands with
OMS_CMD_BLK, for example, with fail with either OMS_ERR_DONE_PENDING or
OMS_ERR_INT_QUEUED.  To overcome this the OMS_CMD_FORCE ioctl request
should be used with an appropriate command string.  One way to handle this
is to issue a stop and interrupt sequence for the appropriate axis,
followed by a pause and then a read and acknowledge of outstanding
interrupts.  That should be sufficient to halt the stepper motor and pick
up any outstanding DONE interrupts.  Example code is as follows:

{
	OMS_req_t req;
	int res;
	char cmd[80];
	char axis_name = 'S';
	unsigned char axis_mask = OMS_S_AXIS;

	/* Fill in req with a command, and short timeout */
	res = ioctl(fd, OMS_CMD_BLK, &req);

	switch (res)
	{
	case OMS_ERR_TIMEOUT:
		sprintf (cmd, "a%cstid", axis_name);
		req.cmdstr = cmd;
		req.cmdlen = strlen(cmd);
		req.axes = axis_mask;
		req.timeout = 2;
		res = ioctl(fd, OMS_CMD_FORCE, &req);
		if (res)
			PANIC();
		sleep(1);
		res = ioctl(fd, OMS_ACK_INTS, &req);
		if (res)
			PANIC();
		/* You may want to check f_slip, f_limit, etc */
		break;
	...
	}
}

The main problem is that the missing DONE interrupt may arrive just after
you issue your Stop, but before the Stop is picked up and actioned by the
microcontroller on the board.  The OMS_CMD_FORCE will then return as a
result of the wrong interrupt.  We rely on the sleep(1) to give time for
the real interrupt to arrive, and the OMS_ACK_INTS to pick it up.  A one
second pause may not actually be long enough, depending on how long it
takes to stop the stepper motor.

A better approach may be to avoid using interrupts at this stage, and to
issue a stop as a normal OMS_CMD_RET, and then issue, using OMS_CMD_RESP,
AXRQ once a second until the axis command queue is empty.  Then
OMS_ACK_INTS to tidy up.  You may still have DONE_PENDING at this stage,
which can be cleared by OMS_CMD_FORCE with string AXID.  Messy - needs
more thought.

Also, you need to be aware that a limit switch being set seems to inhibit
all interrupts for that axis, so AXSTID with a limit switch set will
just time out.


5.  Support for PC68 board

Initial driver development will be on a VME58 board.  Register accesses
are confined to two small functions (one to read and one to write), so
it will be easy to support the different register access method of the
PC68.  There are differences between the boards in terms of the number of
axes, number of user defined I/O bits, etc. but the server can issue a
'Who Are You' command to find the board type and act appropriately.
Commands and responses on the PC68 are transferred a byte at a time via an
I/O port; this will require changes to the code which currently accesses
shared memory queues.

Latest information from OMS suggests that changes may need to be more
extensive, as the PC68 does not have limit or slip registers and such data
has to be retrieved via the command/response queues.


6. User library

A very simple user library which takes multi-parameter functions and
creates an OMS_req structure before calling the driver could be produced.
This may be of use for issuing simple commands, but once signal handlers
and analysis of the returned structure is taken in to account, the user
processes begin to need to know more of the interface to the driver.
Given the simple driver interface, it may not be appropriate to try and
hide aspects of it behind a library.

The library currently consists of a header file defining the driver
interface, and a function to map driver error numbers to text strings.


7. Possible enhancements

It may be appropriate to define more high level ioctl commands, similar to
OMS_READ_POSITION, to cover response-generating commands such as RA
(Request Axis Status).


8. Outstanding worries:

What do the missing manual pages say?

What happens if the command buffer for an axis fills?  Does it block
the common command buffer?  Ans: Yes, it does.

Does the PC68 provide the same set of registers and flags, via its own i/o
port interface?  Ans: No, it doesn't.

I have had some trouble with insmod crashing when inserting modules.  I
fixed a similar problem in 2.1 kernels a while ago, and need to see if
the same applies to 2.0.  Problem seems to have gone away - may have been
a bug in the driver.

Does reading limit reg (for example) clear limit bit in status reg?  If
so I may loose interrupts when doing an OMS_READ_FLAGS.  Ans: No, it
doesn't appear to, so this is not a problem.

Do we want to read stepper position or encoder position with
OMS_READ_POSITION?  Ans: Needs to be configurable in some way.

May be problems with SIGALRM (for example) interrupting blocked calls on
the driver; will have to ignore such signals in the driver if so.

