Sunday, August 19, 2018

Using the VMS PATCH utility to fix a VAX LISP issue

In my previous post about VAX LISP, I complained about a misfeature of the editor that prevented binding editor functions to the Ctrl-S and Ctrl-Q keys.  In the manual, it is stated that these keys cannot be bound due to "system restrictions". It appears, though, that it is a measure to prevent users from footing themselves in the foot because Ctrl-S and Ctrl-Q could be sent by terminals to implement software flow control.  Software flow control is optional, though; it can be switched on and off for each terminal line and it is not needed for virtual terminal connections (Telnet, DECNET) as flow control is implemented by the network layer and networked terminal sessions usually have enough buffer space.  Thus, reserving Ctrl-S and Ctrl-Q really only makes sense for asynchronous terminal lines that require software flow control, and there is no reason to reserve these characters for all terminal sessions, regardless of technology.
I am used to having Ctrl-S and Ctrl-Q be bound to certain functions from GNU Emacs, and I want to use the same bindings with VAX LISP as they've become part of my muscle memory.  Thus, I wanted to remove the explicit checks for these keys in the EDITOR::BIND-COMMAND and BIND-KEYBOARD-FUNCTION functions.  Lacking the source code to VAX LISP, I could not just remove the checks and recompile, so I needed a different way.
This is where the VMS PATCH utility comes into play.  It is a standard tool that is used to modify program executables, usually to remove bugs without requiring a full reinstallation of the software package that needs to be fixed.

Finding what to patch

In order to remove the checks for Ctrl-S/Q in EDITOR::BIND-COMMAND and BIND-KEYBOARD-FUNCTION, I disassembled the two function using the standard VAX LISP function DISASSEMBLE.  VAX assembler code is easy to read, so it was rather straightforward to find out how the checks work.  Here is the relevant extract of the disassembly output from the VAX LISP BIND-KEYBOARD-FUNCTION function:
000FF  MOVL   -10(FRAME), -(STACK)
00103  MOVZBL #89, -(STACK)
00107  MOVZBL #99, -(STACK)
0010F  MOVL   'CHAR/=, LEX
00113  MOVL   4(LEX), FUNC
00117  JMP    @4(FUNC)
0011A  CMPL   RES, SV
0011D  BEQL   121
0011F  BRB    145
00121  MOVL   FRAME, -(STACK)
00124  MOVL   #A28, -(STACK)
0012B  MOVQ   BIND, -(STACK)
0012E  MOVL   '"The character argument must be a control character (other than~%~
            CTRL/S or CTRL/Q): ~S", -(STACK)
Apparently, #89 and #99 refer to Ctrl-S and Ctrl-Q, and the CHAR/= function is called to check whether the argument is one of those characters.
The easiest way to get rid of the check seemed to me to replace the BEQL 121 instruction, which jumps to printing the error message and returning from the function, to BRB 145, which is the jump to the rest of the function, which performs the actual binding.

Locating the code

To make the actual modification using the PATCH utility, I now needed to know where in the executable file these instructions can be found - the addresses in the DISASSEMBLE output are only relative to the start of the function, and the LISP.EXE executable file does not include any symbols.  I thus picked out a few instructions that seemed characteristic and assembled them to machine code using the VMS macroassembler.  Then, I searched for this instruction sequence in LISP.EXE using a trivial perl program, which gave me the offset of the function that I needed to update.
Knowing the offsets of the locations in the binary file that I needed to change, I could now go about and use the PATCH utility in interactive mode to devise the actual patch.  The PATCH manual suggests that the VMS standard debugger would be used for patch development, but as I did not have the symbol table for LISP.EXE, using PATCH itself for development was more appropriate as it can work in /ABSOLUTE mode which uses file offsets for loctions.
First, I needed to find and disassemble the code using the locations I had devised.  As I only had the offset of the MOVZBL #89, -(STACK) instruction and VAX instructions have variable lengths, I needed to probe a bunch of addresses before I could come up with this:
PATCH>e/i 00403E68:00403E9A
00403E68:  MOVQ    R10,-(R7)
00403E6B:  MOVL    B^0F0(R8),-(R7)
00403E6F:  MOVZBL  #089,-(R7)
00403E73:  MOVZBL  #099,-(R7)
00403E77:  MOVAB   B^18(R7),R8
00403E7B:  MOVL    B^30(R11),R9
00403E7F:  MOVL    B^04(R9),R11
00403E83:  JMP     @B^04(R11)
00403E86:  CMPL    R6,AP
00403E89:  BEQL    00403E8D
00403E8B:  BRB     00403EB1
00403E8D:  MOVL    R8,-(R7)
00403E90:  MOVL    #00000A28,-(R7)
00403E97:  MOVQ    R10,-(R7)
00403E9A:  MOVL    B^38(R11),-(R7)
This is exactly the same instruction sequence that I disassembled using VAX LISP above. As you can see, it is significantly harder to read the disassembly produced by PATCH as it does not show symbolic register names, and it also does not translate data pointers like in the last instruction - VAX LISP showed the error message where PATCH prints B^38(R11).  The addresses in the disassembly, however, are absolute offsets into the LISP.EXE file, and that was what I needed.

Creating the actual patch

Equipped with this information, I could create the actual patch to change VAX LISP so that it no longer disallows binding of Ctrl-S and Ctrl-Q, which I reproduce below:
'BLEQ ^X00403E5E'
'BLEQ ^X00403EB1'
'BEQL ^X002F324A'
'BRB ^X002F3265'
I used the REPLACE/INSTRUCTION command which requires the specification of an address, the current instruction(s) at the location and the new instruction(s) to put at the location.  PATCH would only perform the replacement if the instructions currently in the file match what is specified, otherwise it would abort with an error.  This is useful to prevent the patch from patching the wrong file or the wrong version of a file.

Better patches

As I did not have the source code or the symbol table for LISP.EXE when I developed this patch, I needed to resort to patching in absolute mode which is kind of brittle.  PATCH can also work in symbolic mode if a symbol table is available for the executable file, which allows patches to work even if the executable has been linked differently than the executable used to develop the patch (i.e. you could write a patch for a library function that'd work for executables that were linked statically against the library).  Also, PATCH allows patches to insert more instructions than were originally present in the executable.  It does this by adding additional disk space to the executable to hold the new, longer code, and then patch the required jump instructions to the patch area into the original location.

Closing thoughts

It is quite nice to be able to fix a bug in or remove a feature from a commercial program without having the source code, just by working with a bit of disassembly and on-board utilities. Back in the day, it was also useful to be able to distribute fixes to customers in the form of tiny patches. These could even be sent as a printed letter and typed in by the customer on a terminal, with reasonable effort and good chances of success.

0 Kommentare:

Post a Comment