    PICHEX: A package for manipulating Microchip PIC16C84 hex files
    ===============================================================

                               David Tait

                      Electrical Engineering Labs
                         School of Engineering
                             The University
                          Manchester  M13 9PL
                                   UK

                          david.tait@man.ac.uk
              http://www.man.ac.uk/~mbhstdj/piclinks.html


                     Copyright (C) 1996 David Tait
                             18th July 1996




INTRODUCTION

This package provides a suite of programs for manipulating Microchip hex 
files.  The software is designed for the type of hex dump used to 
program the PIC16C84; these are based on Intel hex format and integrate 
into a single file all the information needed to specify the contents of 
the PIC16C84 program memory, data memory, ID words and configuration 
word.  The PICHEX package can perform the following operations: 

        o  Convert INHX8M files to INHX16 and vice-versa.
        o  Convert between INHX8M and John Morrison binary.  (*)
        o  Combine separate program and data hex files.
        o  Split an integrated hex file into separate files.
        o  Add/replace a configuration word.
        o  Add/replace a 4-word ID record.
        o  Identify which areas of a hex file are present.
        o  Compute a file checksum using Microchip's own algorithm.
        o  Build a data hex file from a free-format hex list.
        o  Dump a data hex file as a hex list.
        o  Intelligently compare two hex files.
        
        (*) Not verified yet.

The toolkit consists of a number of small DOS executables (sorry, but 
there's no colourful GUI here).  The package includes C source and it 
should be relatively painless to port the toolkit to other environments 
like Unix/Linux for example.  Although the utility of this package is 
questionable - some freely available PIC programmer software does 
everything PICHEX does and more - I hope it will have some tutorial 
value.  In fact, one aim of writing this package was to provide an 
"everything you wanted to know about Microchip hex files, but were 
afraid to ask" guide in source form.  Even if it fails in that role, you 
may find the C source for handling integrated Microchip hex files will 
save you some time if you ever decide write your own PIC programmer 
software, disassembler, simulator or whatever.  Anyway, if you find the 
toolkit useful please let me know (if you don't then please delete it 
and accept my apologies for having wasted your time).  


AUDIENCE

Hex file utilities are either something you desperately need or they are 
just a waste of disk space depending on what tools you already have.  

This package is unlikely to be useful to the assembly language 
programmer who has access to a modern PIC programmer.  That's because a 
good assembler (like Microchip's MPASM) can produce all the required 
elements of the PIC16C84 hex file and a modern PIC programmer can make 
use of them.  Still, there are some PICHEX functions that may prove 
helpful: e.g., adding data records from a hex list, particularly if this 
is the only part of the file that will change between development 
cycles; and the PICHEX add ID function can be used in a batch file to 
automatically update ID bytes each time a PIC has been programmed.  

The package should be more useful if you use a PIC programmer that 
doesn't recognise integrated hex files.  In this case the PICHEX 
functions can be used to manipulate the files into separate program and 
data parts or into a program file and a hex list.  If you are building a 
PIC16C84 based project based on an electronics magazine article you may 
find that a hex dump is all that you have.  In that situation PICHEX may 
help you if the format provided is incompatible with your programmer.  
The file VPP.ZIP contains information (in the form of a working example) 
on how the PICHEX library can be used to make a programmer front-end 
that is able to understand integrated hex files (read VPP.TXT in VPP.ZIP 
for the details).  

PIC16C84 hex files are a commodity item with one group of enthusiasts.  
These people will often have homebrewed their own PIC programmer but are 
are unlikely to have ever written even a trivial PIC program.  Instead 
they rely on a select group of gurus to disseminate PIC files from time 
to time and when this happens there is a flurry of activity while hex 
code is grabbed from various BBSs, FTP or WWW sites.  They are the 
pay-TV hackers (I guess there really should be a distinction between the 
gurus who hack and those who use hacks but I don't know a good name for 
the latter).  It turns out that the PIC16C84 is a very popular processor 
for constructing pirate smartcards, and hex files for PIC-based cards 
are found on many BBS and Internet sites.  I found this out a couple of 
years ago when I made the mistake of making a PIC16C84 programmer design 
available on the Internet. Back then I knew nothing about the smartcard 
applications of the PIC16C84 but I soon learned about them when I 
started getting e-mail asking me why my programmer didn't work with the 
various smartcard dumps.  Having found myself on the periphery of the 
pay-TV hacking culture I started reading and answering PIC-related posts 
in the newsgroup alt.satellite.tv.crypt.  The posts in a.s.tv.c.  soon 
convinced me that confusion about incompatible hex file formats was not 
confined to the people struggling with my programmer.  I readily admit 
that I know nothing about the technical details of pay-TV encryption, or 
what a PIC-based smartcard actually needs to do, but I do know a thing 
or two about hex dumps (not that there is really that much to know 
anyway).  



AIMS AND ALTERNATIVES

The aim of the PICHEX package is to provide a toolkit to manipulate hex 
files while at the same time passing on some information about hex file 
formats.  To research what kind of operations would be worthwhile 
including in the toolkit I grabbed some smartcard dumps from various 
pay-TV sites on the Internet (my WWW page mentioned in the header of 
this document has links to some of these sites).  The dumps I found were 
mostly intended for use with equipment designed for Eurocrypt encoded 
D2-MAC satellite TV transmissions (as far as I can tell - I don't own a 
satellite TV system and I don't have first hand knowledge of what's 
what).  There was a variety of incompatible formats around: integrated 
INHX16; integrated INHX8M; files split into separate program and data 
parts in both INHX16 and INHX8M; and various forms of binary file.  I 
wrote the PICHEX package based on the manipulations needed to convert 
between the various dump styles I encountered on these sites, although I 
sincerely hope that it will prove useful in a wider context than just 
pay-TV hacking.  As most binary files I found were in a common format 
(apparently defined for the "John Morrison" PIC programmer - I'll call 
them JM binary for short) I included operations to support these too.  
(Unfortunately, I couldn't find a good description of JM binary files.  
I worked out some of the details with the aid of packages that contained 
both hex and JM binary equivalents, however, there are 32 bytes in these 
binary files that I'm still unsure about.  The readbin() and writebin() 
functions in jm2hex.c and hex2jm.c encapsulate my best guess at the 
format.  If anyone can provide definitive information then please get in 
touch.  In the meantime, the JM binary converters are not to be trusted, 
not that any of my other programs are guaranteed to work.) 

To be honest as I write this I find it hard to justify the modest amount 
of time I spent producing this stuff.  While I was trawling the net 
looking for hex dumps I found plenty of tools that are far more 
comprehensive or more user friendly than PICHEX. (see my WWW page cited 
above for a reference to "Defiant's" site as most things are there.) For 
pay-TV hackers in particular I came across a neat package by Michael 
Stegen containing a small program that allowed the user to request the 
single-PIC smartcard code in several different formats and clearly this 
obviates the need for hex conversion utilities.  I also discovered some 
PIC programmer software by Bengt (no other name given) based on freeware 
Pascal sources written by Cathy Moss; this software is useful even if 
you don't have a compatible programmer because it includes many 
ancillary features, for example, it has a built-in disassembler, it can 
load and dump files in a variety of formats and has a hex editor that 
lets you change any part of a hex file with ease.  If you know a little 
about hex file structure, a couple of programs by Jaap Schaap give you 
most of the functionality of PICHEX; one program fixes up checksums and 
the other converts file formats.  To use this package all you do is edit 
the hex file data fields then generate the correct checksums with one 
program and convert it to the desired format with the other.  A 
Windows-based PIC16C84 disassembler by Uwe Keller can also convert 
between file formats (the documentation that came with this program 
claimed to describe JM binary format, but either it's wrong - it 
describes a 2114 byte format whereas all the examples I found had 2144 
bytes - or there is more than one JM format).  I suppose you might find 
these tools more useful than PICHEX.  



USING PICHEX

There's not much to say about the programs themselves but some brief 
documentation is provided at the end this document.  The core hex 
loading/dumping routines are general purpose but the programs themselves 
are only intended for the use with PIC16C84 files and will ignore hex 
records for addresses not applicable to the PIC16C84 (including the 
"device" records in Parallax files).  All the PICHEX programs produce a 
short usage message if they are invoked without arguments.  The batch 
file DEMO.BAT can be run to see PICHEX in action (make sure you run DEMO 
in a directory that contains FIRST.HEX and the toolkit programs are in a 
directory listed in PATH). DEMO.BAT also offers a short tutorial on hex 
file formats.  

The C source for PICHEX is in SRC.ZIP and includes some notes on 
compiling the programs as either standalone executables or in library 
form.  If you discover any bugs in the toolkit programs I would 
certainly like to know about them but as you have the source code you 
might find it quicker to fix the bugs yourself rather waiting for me to 
reply.  Please remember that these programs are provided as is and 
without warranty of any kind and I will not be held responsible for any 
misuse.  




                        PICHEX TOOLKIT CONTENTS


       ADDCFG    Add/replace config record
       ADDID     Add/replace ID word record
       CMPHEX    Compare contents of two hex files
       COMBHEX   Combine program and data files
       HEX2JM    Convert hex to John Morrison binary
       HEX2LIST  Dump a hex file as a list of hex bytes
       JM2HEX    Convert John Morrison binary to hex
       LIST2HEX  Convert a hex byte list to hex file
       SPLITHEX  Split a hex file into program and data
       SWAPHEX   Convert INHX8M to INHX16 and vice-versa
       WHATHEX   Describe a hex file



-----------------------------------------------------------------------
ADDCFG

       
       Adds/replaces a config record in a Microchip hex file.  
       
Usage: addcfg cftext infile [outfile] 
       
       If outfile is not specified stdout is used.  
       
       cfgtext is a string containing one or more of: 
       
            'c' (code protect)
       'p' (power-up timer enable)
       'w' (watchdog enable)
       'l' (LP oscillator)
       'x' (XT "")
       'h' (HS "")
       'r' (RC "")


-----------------------------------------------------------------------
ADDID

       
       Adds/replaces an ID record in a Microchip hex file.  
       
Usage: addid id1 id2 id3 id4 infile [outfile] 
       
       If outfile is not specified stdout is used.  The id words 
       id1-id4 are specified as hex strings.  
       

----------------------------------------------------------------------- 
CMPHEX 

       
       Compares two Microchip hex files.  The files are not 
       compared directly (they can even be different formats), 
       instead their programming effect is compared.  
       
Usage: cmphex hexfile1 hexfile2 
       

----------------------------------------------------------------------- 
COMBHEX 

       
       Produces a single Microchip hex file from two others: one 
       file provides the program, ID and config words, and the 
       other provides the EEDATA bytes (data file origin can be 
       either 0 or 0x2100). The input files can be in different 
       formats (either INHX8M or INHX16) but INHX8M is used for 
       the output file.  
       
Usage: combhex progfile datafile [outfile] 
       
       If outfile is not specified stdout is used.  
       

----------------------------------------------------------------------- 
HEX2JM 

       
       Converts Microchip INHX8M or INHX16 files to John Morrison 
       binary format.  
       
Usage: hex2jm hexfile binfile 
       

----------------------------------------------------------------------- 
HEX2LIST 

       
       Dumps the EEDATA contents of Microchip hex file as a list 
       of hex bytes.  The first line of the list is the EEDATA 
       origin.  If the file contains no data then the first 64 
       bytes of program memory are dumped.  
       
Usage: hex2list hexfile [listfile] 
       
       If listfile is not specified stdout is used.  
       
       (Note: not verified.) 
       

----------------------------------------------------------------------- 
JM2HEX 

       
       Converts John Morrison binary files to Microchip INHX8M 
       format.  
       
Usage: jm2hex binfile [hexfile] 
       
       If hexfile is not specified stdout is used.  
       
       (Note: not verified.) 
       

----------------------------------------------------------------------- 
LIST2HEX 

       
       Converts a list of hex bytes to an EEDATA file in INHX8M 
       format.  The first entry in the list is the origin used 
       for the hexfile and can be either between 0 and 0x3F or 
       between 0x2100 and 0x213F. The bytes must be separated by 
       spaces or newlines and can be in upper or lower case.  
       Unused bytes in the list are silently ignored.  
       
Usage: list2hex listfile [hexfile] 
       
       If hexfile is not specified stdout is used.  
       
----------------------------------------------------------------------- 
SPLITHEX 

       
       Splits a Microchip hex file into two others: one file 
       containing only the program words and the other containing 
       only EEDATA bytes.  The output files are written in the 
       same format (either INHX8M or INHX16) as the input.  The 
       ID words and config word are discarded.  
       
Usage: splithex hexfile progfile datafile [0] 
       
       If last parameter is 0 then an origin of 0 rather than 
       0x2100 is used for the EEDATA hex file.  
       

----------------------------------------------------------------------- 
SWAPHEX 

       
       Converts Microchip hex files in INHX8M format to INHX16 
       format and vice-versa.  
       
Usage: swaphex infile [outfile] 
       
       If outfile is not specified stdout is used.  
       

----------------------------------------------------------------------- 
WHATHEX 

       
       Describes the contents of a Microchip hex file.  The 
       length of program and data areas is given, the ID words 
       are printed and the configuration is analysed.  An overall 
       checksum is calculated two ways: the Microchip way, and 
       one that includes the program area, data area and only the 
       active bits of the configuration word.  
       
Usage: whathex hexfile 


























