Wednesday 11 January 2012

Batch file driven compilation

Building a Delphi project with batch files (and a handful of command line tools)

The humble batch file. Ubiquitous since the days of MS-DOS, batch files offer a means of chaining together a list of commands to do work that requires the involvement of several executable files in a sequence.

Little changed in the era of Windows 7 & 8, batch files can be used to automate tedious and repetitive tasks in a way that can be replicated across every member of the Windows family.

This is precisely the objective I had in mind when it came to co-ordinating all of the tools involved in building a large Delphi project. For demonstration and “toy” projects, it's generally sufficient to compile an executable from the IDE, and then copy or run the resulting binary as-is.

But when you're putting together something bigger there are real benefits to a consistent script-based approach to directing all of the separate applications and actions that you might need to produce your final distributable.

There are a slew of commercial and free/open source visually orientated tools that can help you with this, but I didn't want to learn my way around another user interface and work flow when I already had a clear idea of how everything would fit together.

Automated Build, and Continuous Integration tools (http://bit.ly/zkGkdm)

If you prefer pointing-and-clicking to the utilitarianism of the command line interface, there are plenty of graphical tools that can help you take control of your application build process.

Some of the most frequently referenced include the open source Java-based CruiseControl and derivatives (http://bit.ly/qqPY), Team Foundation Server from Microsoft (http://bit.ly/QiewU), geared towards Visual Studio, and FinalBuilder, a commercial product written in Delphi (http://bit.ly/wIraXy).

It's very easy to log events as they happen using commands within the batch file, and conditional expressions allow you to branch the order of processing depending on system or “variable” states. In other words, you lose little in the way of functionality by choosing a command line build system.

And there's a certain satisfaction to the UNIX-like practise of teaming a collection of small, purpose specific tools to create more or less complex behaviours, and working directly against the command prompt. 


Batch file fundamentals

Wikipedia is a useful starting point for details of some of the commands that can be invoked from a batch file, but I've also found the Google Groups archived Usenet posts an invaluable reference source:


Rather than go through each command individually (both the Wikipedia List of DOS commands article and in-built help command issued from the command line provide all of this information. In addition help can be called with a parameter – the name of another command – for a more detailed explanation, e.g., help if), I want to highlight a couple of lesser known commands, and some of the “programming like” capabilities that they afford us.

The first of these is CALL. The Windows help for the CALL command says:

Calls one batch program from another. 

This concise summary belies its enormous utility, particularly when used in conjunction with the second important command, or instruction, LABEL. The Windows help description for CALL continues:

CALL command now accepts labels as the target of the CALL. The syntax is:
CALL :label arguments

A new batch file context is created with the specified arguments and control is passed to the statement after the label specified.

It's non-obvious on first inspection but what this means is that you can create a library of batch file “sub routines” that can then be called from any other (or the same) batch file, in much the same way that you might use – say – a “Strings” library unit in Delphi.
The only complication here is that a batch file sub-routine can't return a result: it can only set a global “result” variable (or any number of global variables – none of these is actually a sub-routine “result”, but can be treated as such) that are visible to, and can be accessed by, the caller.

Lets have a look at a short example to see this in action. Here we're using two batch files – a.bat and b.bat. a.bat is our “main” file, b.bat is our library file. a.bat is going to read a list of names from a file and, by calling a routine in b.bat, determine the longest name in the file then print this on screen.

Coding conventions

"Constant" environment variables. Although these are indistinct from other variables, I've adopted a naming scheme that sets them apart from environment variables that may be updated later in time. For these constant variables, the first letter of the name will always be a lower-case 'c' (for “constant”); for variables that may change, the first letter of the name will always be a lower-case 'v' (for “variable”).

The only exceptions to this rule are for variables that are used “globally” (from any point within the batch file). The first letter of the name for these variables is always a lower-case 'g' (for “global”), and then followed by a second letter 'c' or 'v' depending upon whether or not it might be modified after its declaration.


a.bat script listing:

1:   @ECHO OFF  
2:    
3:   SET viLongestName=0  
4:   SET vsLongestName=  
5:   ECHO JOHN   >  c:\temp\names.txt  
6:   ECHO JIM    >>c:\temp\names.txt  
7:   ECHO THOMAS >>c:\temp\names.txt  
8:   ECHO MARK   >>c:\temp\names.txt  
9:    
10:  SETLOCAL ENABLEDELAYEDEXPANSION  
11:  FOR /F %%S IN (c:\temp\names.txt) DO (  
12:   CALL "b.bat" TRUE :strlen %%S  
13:   IF !gvStrLen! GTR !viLongestName! (  
14:    SET viLongestName=!gvStrLen!  
15:    SET vsLongestName=%%S  
16:   )  
17:  )  
18:    
19:  IF NOT "!vsLongestName!"=="" (  
20:   ECHO Longest name is !vsLongestName! with !viLongestName! characters  
21:  ) ELSE (  
22:   ECHO No names found in file  
23:  )  
24:  ENDLOCAL  

Deconstructing this:
  • @ECHO OFF turns off console “echoing” for each statement evaluated by the interpreter; with this switched off, the batch file will not print anything to the console unless 1) an error occurs, or 2) It is explicitly ECHO(ed) to the console by the batch file.
  • Lines 3 and 4 set variables “iLongestName” (representing the length of the longest name found so far) and “sLongestName” (representing the text of the longest name found so far) to their starting values. All variables – more correctly called “local environment variables”1 - are represented as strings.
    But we can still store a numerical value in an environment variable because the comparison operators available:

    EQU – for testing whether two values are identical
    NEQ – for testing whether two values are non-identical
    LSS – for testing whether one value is “less than” another value
    LEQ – for testing whether one value is “less than OR equal to” another value
    GTR – for testing whether one value is “greater than” another value
    GEQ – for testing whether one value is “greater than OR equal to” another value

    ...can all be used with numbers stored as strings in the same way that:

    var
      aString, bString: String;
    begin
      aString := '1';
      bString := '2';
      if bString > aString then
        ShowMessage(bString + ' is greater than ' + aString)
      else
        ShowMessage(aString + ' is greater than ' + bString);
    end;

    Will work as expected: the message: “2 is greater than 1” would be shown (bString evaluates to “greater then” aString).

  • Lines 5, 6, 7 and 8 write out some names to a text file, c:\temp\names.txt
    Something to note here, the ECHO command usually prints a string to the console, but in this instance we use '>' to “redirect” the string to a file.
    Using '>' overwrites the content of a file with a new string; using '>>' appends a string to the existing content of a file (or creates a new file and adds the string to it if no file of the given name currently exists). 

    The additional spacing on some of these lines (i.e., between the names and the '>' or '>>' characters) is added for visual clarity and has no bearing on the way that they are evaluated by the interpreter.

  • Line 10 sets ENABLEDELAYEDEXPANSION. This is required because we're updating variable values in a loop. Without ENABLEDELAYEDEXPANSION, variable values are fixed when the script is initially parsed (each variable keeps the first value it is assigned); by setting ENABLEDELAYEDEXPANSION variables are instead updated at runtime, so can be modified in the loop. Every declaration of ENABLEDELAYEDEXPANSION must be paired with a closing ENDLOCAL, which signifies the end of a delayed variable expansion block.

  • Line 11 sets up the loop which iterates over all of the lines in thenames.txt file.
    The command line interpreter For command is very versatile, with the ability to walk directories and process all or a subset of the files found. Each file can be parsed line-by-line and tested, for example, for specific End Of Line characters, or broken up into individual tokens for further processing - including evaluating tokens at numbered indexes along each line.
    In this example, For /F tells the interpreter that we're operating on one or more files, and %%S is the syntax for the loop variable that stores each line of each file as it is parsed out ('S' could be any character, I've chosen 'S' as shorthand for “String”).

    IN precedes a parenthesis enclosed set of files (the wildcard character may be used here; individual file names within the parenthesis should be separated by a space) to loop over, and DO precedes a list of commands to perform on them – or in our case, because we're using the/F parameter - to perform on each line of thenames.txt file.

  • Line 12, the first to be processed within the scope of the FOR loop (which is delimited by the opening and closing parenthesis) uses theCALL command to switch processing to b.bat, and passes CALL three parameters, TRUE, :strlen and %%S. These three parameters are presented to the CALL(ed) batch file, which can retrieve them as command line parameters - not unlike the ParamStr(<n>) function in Delphi – and use them as arguments in the named sub-routine.
    The parameters could, of course, be anything (any String values), but they have special meaning in the context of this example.

    The first parameter, TRUE, is used to determine whether to set a FAIL errorcode if a problem occurs in the called batch file. The second parameter is a label that identifies the name of the sub-routine to execute in the called batch file. The third parameter contains a line of text from the names.txt file.

  • Line 13 is a conditional test, using IF, of the “result” global variable set by the sub-routine in b.bat. If the global variable evaluates to “greater than” (GTR) the value currently stored in iLongestName, then processing continues to the two lines below, which respectively assign this value to iLongestName, and the text stored in %%S to sLongestName (i. e ., keeping a running count of the longest name seen so far).

  • When the FOR loop terminates (line 17), execution continues to the test on line 19. sLongestName is compared against an empty string (if the names.txt file contained no entries, sLongestName would never have been modified from its starting value). If it has been assigned a name from the file, the name and its length are printed to the console.
    The comparison is made using the == operator, but could alternatively have used EQU. The test employs a common convention when checking against an “empty” value. A variable cannot be tested against nothing (i.e., it might be assumed that leaving a space after the equality operator would be equivalent to testing against an empty String – but this is not so – a command must follow the equality test, and so a second test value must come ahead of this).
    By surrounding the variable to test with quotes, and adding a pair of consecutive quotes on the other side of the operator, the interpreter will return TRUE if no value has been assigned to the variable (from the perspective of the interpreter the comparison would resolve to: “”==””).

  • A final point about this first code listing: variables referenced outside of a ENABLEDELAYEDEXPANSION/ENDLOCAL block are represented using the %VariableName% syntax (each variable must have a percent sign at the beginning and end); variables inside of a delayed expansion block that are to be updated must instead be represented using exclamation marks as variable name delimiters, e.g., !VariableName!


Now lets examine the code listing for b.bat

1:   @ECHO OFF  
2:    
3:   REM Contains functions used in more than one batch file  
4:   REM Prerequisities: None  
5:   REM NOTE: for all functions, first parameter must be a boolean that indicates whether to issue a 'fail' errorcode if a problem occurs  
6:   REM NOTE: for all functions, second parameter must be a label that indicates which function to call  
7:    
8:   SET cErrOnFail=%1  
9:   SET cFuncName=%2  
10:  SET cUnitTempFile=%TEMP%\Temp%RANDOM%.txt  
11:    
12:  FINDSTR.EXE /n ^%cFuncName% b.bat>%TempOutputFile%  
13:  SET /p cLblExists=<%TempOutputFile%  
14:    
15:  IF NOT DEFINED cLblExists GOTO error0  
16:  GOTO %cFuncName%  
17:    
18:  REM Return the length of a string passed in as the first command line parameter  
19:  REM -param1 = RESERVED  
20:  REM -param2 = RESERVED  
21:  REM -param3 = String to be measured (if more than one word, enclose in double-quotes)  
22:  :strlen  
23:  SET gvStrLen=0  
24:  IF "%~3"=="" GOTO :EOF  
25:    
26:  SET _FName=_%RANDOM%.txt  
27:  SET _FNameFull=%TEMP%\%_FName%  
28:    
29:  ECHO %~3 > %_FNameFull%  
30:  FOR /F "tokens=3" %%F IN ('DIR /-C "%_FnameFull%"^|"FIND.EXE" /I "%_FName%"') DO (  
31:   SET /a gvStrLen=%%F - 3  
32:  )  
33:  DEL %_FNameFull%>nul  
34:  IF ERRORLEVEL 1 GOTO error2  

Many of the lines in this file are comments, ignored by the interpreter, and included to aid callers in understanding how to make use of it. These lines all begin with the REM(ark) statement.

  • The first line of interest is 8: lines 8, 9 and 10 all set “constant” environment variables that will not change for the duration of the execution of the batch file. The variables on lines 8 & 9 are both assigned values from the “parameters” passed in by the calling batch file, a.bat (which are stored in %1 and %2, though the series could run up to %n, where 'n' is the index of the last parameter).

    cerrOnFail is set to TRUE (i.e., a String value containing the text “TRUE”, rather than a Boolean) and determines whether to return an error code if something goes wrong (the code listing for b.bat is part of a longer batch file, and cErrOnFail is not used in this abridged version).

    cfuncName holds the value of the second parameter, which is the name of the sub-routine that the caller wants to execute in b.bat (in this case the :strlen routine).

    cunitTempFile is assigned a file path in the TEMP directory (using the pre-defined %TEMP% environment variable – a complete list of pre-defined environment variables can be viewed by typing SET at the command prompt), a name that begins “Temp”, and is followed by a random number provided by the %RANDOM% environment variable, and “.txt” extension.
    This doesn't guarantee uniqueness, but it's good enough for our purposes. This will be used as a “scratch” file, a temporary dumping place for data that will be used later.

  • Line 12 uses the FINDSTR command to parse b.bat for a token matching the name of the sub-routine we want to execute (“:strlen”). The /n switch causes FINDSTR to return the line number if a match is found, and this is written out to our scratch file using redirection '>'.

  • Line 13 calls SET with the /p switch, which reads in a String from a named file instead of accepting a value typed interactively at the command prompt. Here the “constant” variable cLblExists is assigned the line number written to the scratch file by FINDSTR.

  • Line 15 checks whether the call to SET was successful. If FINDSTR did not locate “:strlen” inside of b.bat, nothing would have been written to the scratch file and variable cLblExists would be unassigned. If cLblExists is unassigned, this test will fail and processing will jump to the position in b.bat indicated by the label error0 using the GOTO command (note that, as with GOTO in Delphi, processing does not return after the code following the label has been executed).

  • Line 16 is the last portion of the “lead in” part of b.bat, before work begins inside of the sub-routine specified by a.bat. The GOTO command causes processing to jump to the sub-routine name stored in cFuncName.

  • Line 22 is the label for the :strlen sub-routine (the “entry point” for the function, as it were).

  • Line 23 initializes a global variable (global because all batch file variables are, but also because it is used to store the routine “result”, and so will be referenced outside of “scope” of :strlen) named gvStrLen to its starting value of 0.

  • Line 24 is a test which evaluates whether the third parameter passed to b.bat (the String to have its length measured) is empty, and if it is, skips to the EndOfFile (:EOF – a pre-defined label), after which the interpreter returns execution to a.bat.

  • Lines 26 & 27 set two variables that store the name and fully qualified name of a scratch file to use for temporary storage.

  • Line 29 writes the passed-in String to the scratch file.

  • Line 30 is a compound statement. The FOR command is used with the /F switch. In this instance it's indicating that we're working with a String (which will be returned by FIND at runtime), although it can also be used to indicate operations on a set of files, as we saw in the FOR loop in a.bat.
    In additional the “tokens” option is specified with a qualifier “3”. This tells FOR that we want it to search for the third token (note that, because we're not specifying the “delims” option, the default token separator - a space or tab - will be used) in the String that will be given to it by FIND.

    The section of the statement following IN (inside of the parenthesis) is processed first. The DIR command is called with the /-C switch and the path to the scratch file as a parameter. When supplied with a file name as a parameter, DIR performs a FindFirstFile on the file path, rather than a directory enumeration. Calling DIR with /-C means the output from the DIR(ectory) listing will print:

    1) the file creation date
    2) the file creation time
    3) the file size in bytes
    4) the file name

    But it will also print a summary “table” with volume and directory information. To remove this, the output from DIR is piped (using the “pipe” | character – which must be escaped in this statement to prevent it from being parsed early, by inserting a carat ^ character immediately in front of it) into FIND. FIND searches through the DIR output for a line containing the name of the scratch file, and returns this line alone, minus the volume/directory information table.

    This line is then passed to the FOR loop, which extracts the third token from the line – the file size in bytes – and stores it in the gvStrLen variable.

    It is this value that is equal to the length of the passed-in String. This works because the scratch file is otherwise empty, has no binary header or other fields, and so is comprised entirely from the bytes representing the text it contains (with each character one byte in length).

  • Line 33 deletes the temporary scratch file from disk (we clean up after ourselves). Notice that the output of the DEL command is directed to the special nul device, which essentially swallows it and prevents it from appearing in the console. Why do this? Deletion of the temporary file is not critical to the outcome of the sub-routine call so we don't worry if it fails to run successfully.

  • Finally, line 34 checks whether an error was signalled by DIR or FIND (i.e., if either command set the environment variable ERRORLEVEL by way of an exit code upon termination), and if so, skips to a label called error2, which is omitted from this listing, but would record the error to a log file before exiting b.bat.

Character escaping

In much the same way that a single quote character must be escaped in a Delphi String, or a backslash character in a C String, any percent character must be escaped inside of a batch file to distinguish it from a variable.


Practical application

So how would these ideas be applied in building a Delphi project? As a worked example I'll discuss the steps involved in building my own project using the batch files I created to automate each stage.

Coding convention

  • My batch files are divided into two categories, “work” files, and “library” files. A work batch file does a job – compiles a Delphi project, uploads a file to a FTP server, creates an installer – the library files are listings of independent sub-routines that are used by the work files.

  • To prevent a “work” batch file from being called more than once (if everything has been structured correctly this shouldn't happen anyway), each file is given a special global variable of the form:

    BUILD_<FILENAME>

    And this variable is assigned the String “ACTIVE” the first time the batch file is called. The variable is tested each time the file is invoked (using the DEFINED command, which tests whether a variable has been previously declared), and if it has, processing skips to the end of the file.

The first step involves setting environment variables containing paths to frequently accessed files and directories, including:

  • The path to the Build directory (where the batch scripts are located)
  • The path to the UNIX tools directory (more on this below)
  • The names of .inc files included (using the $I compiler directive) in more than one project
  • The path to the Windows System(32/64) directory
  • The paths to some scratch files for temporary String storage
  • The path to a log file for recording errors
  • A drive letter mapped to the directory containing all other project folders (using the SUBST command – all of the folders exist on a local hard drive, but mapping a drive letter with SUBST made it easier to work with them)
  • The path to the Delphi folder; the path to the Delphi Source folder

At this point all other batch file global variables/constants are also set to their default starting values. I've gathered this collection of variable assignment commands into a batch script called “Globals.bat” and this is always the first file to run. Every other batch file includes a check which forces Globals.bat to be called before executing any of its own commands if the Globals.bat special global variable is undefined (see box-out above).

This way we can be certain that all of the files and directories any of our other batch files might need to know about will have been already been set and are ready for use.


The second step is to prepare the way for building all of the application executables, which consist of the main program binary, and a collection of helper utilities and DLL files (all are separate Delphi projects).

This stage does things like:

  • Ensure each project has an associated DCC32.CFG file
  • Ensure the paths inside of each DCC32.CFG file point to the correct locations (to make it easier to move the build process from machine to machine, where the Delphi source files may reside in different directories)
  • Update configuration files for shared resources, e.g., I use FastMM4 as my drop-in Memory Manager replacement (available from SourceForge, but supplied as standard with Delphi 2006 and later), and I keep two separate configuration file “profiles” depending upon whether I'm debugging, or building a “release” binary

These various processes are all performed by a batch file called “FileUpdates.bat”.

The Delphi DCC32.CFG file

When you compile an application in the Delphi IDE, the compiler uses the Library paths variable (which is persistently stored in the Registry) to find the location of any unit files not contained within the project folder.

You can inspect the Library paths value by clicking Tools->Environment Options, selecting the Library tab and then clicking the browse button alongside the “Library paths” edit box.

When compiling from the command line with the DCC32 compiler, the Library paths variable is not used. Instead paths to folders containing all:
  • Delphi compiled unit files (.dcu)
  • Delphi source code files (.pas)
  • Resource files (.res)
  • Included files (.inc)
...used by the project that are outside of the project folder must either be passed as arguments to DCC32, or (more practically, because it's likely to be a long list) added to a file named DCC32.CFG which is stored in the project folder*.

Alternatively, the two methods can be mixed – you can enter a set of paths as command line arguments to DCC32 AND include additional paths in a DCC32.CFG file (though the paths passed directly from the command line have a higher precedence).

Any compiler directives must also be specified in this way (as command line arguments or in the DCC32.CFG file).

*It can also reside in the same directory as DCC32.EXE, but this can be tricky since each project is likely to have a different set of dependencies. Creating an all-encompassing DCC32.CFG file with paths to every Delphi unit on disk could be one way around that, if a rather unwieldy solution.

The third step is to build each of my projects using the Delphi command line compiler, DCC32.
While the final distributable - an installer application created using Inno Setup by Jordan Russell – is a single file, it bundles together all of the binaries, help files, license files, ini files and other resources that constitute “my application”. The sequence of commands involved in compiling any of these projects is much the same, so we'll limit our examination of them to the “main” application executable.

The batch file for compiling this executable can be roughly divided into eight sections. In order these are:
  • Setting unit-wide variable values for: the path to the folder containing the icon for the executable; the name for the compiled file (which is distinct from the project name); the path to a scratch file for compiler error logging; the path to an anti-tampering utility which marks the compiled file with a CRC stamp (more on this below).

  • Taking a backup of the DCC32.CFG file in the project folder, and replacing the original with an updated copy created during Step Two.

  • Regenerating the project .res file to include the most recent version of the application icon (and “labelling” the build in the Version Control System used for this application – more on this below).

  • Regenerating all other binary .res files from their counterpart .rc files for resources compiled into the executable using the $I compiler directive (these include additional icons, internationalized text, copies of ini files should the user want to restore default settings etc.).

  • Generating a new serial number for the “trial” version of the application (more on this below).

  • Invoking DCC32 to compile the project.

  • Modifying the compiled executable icon using a third party utility: prior to Delphi 2007, “Vista style” icons (up to 256x256 pixels, with PNG compression) were not supported via the IDE (though later versions are reported to have some problems, resolved finally in XE, see http://bit.ly/v1ozEs). There are several solutions to this problem (see http://bit.ly/uPjrDr), but I plumped for ReplaceVistaIcon from RealWorld Graphics (http://bit.ly/bjLlci).

  • Moving compiled executable and map files out of the project directory, renaming the executable to match its “market” name, and stamping the executable with a CRC value.

My Delphi project is a shareware application with two release versions, a “trial” edition, that expires after a fixed number of uses, and a “retail” version that has no usage limitations.

It's also a little unusual in that it has a “boot” loader (my term), a small standalone executable that's launched first. This checks whether it was started from a desktop short-cut or the console – the “main” executable is then started by the loader with a switch indicating which of these modes to adopt (it can run with or without a GUI).

As an attempt at a double line of defence, I integrated shareware protection components from two vendors, both open source: mxProtector from Max's Components for the loader, and OnGuard from TurboPower for the main executable. Each has a multitude of options for configuring the method of protection, and in either case I went for a “maximum uses” limit, but also enforced a 365-day cut-off (from the date of compilation, and regardless of total uses) with OnGuard for the main executable. Both components encode this information in a string (or “serial number”, if you will) that is stored in a separate text file.

OnGuard also includes a utility (StampEXE) that can take the CRC (a short value calculated from the content of a file - http://bit.ly/MTih2) of a compiled file, and then embed this value within the file. Each time the file is run it performs a self-test to check whether it has been modified in any way.

The hobbled-shareware model is a bit passé these days, but that's the topic for another discussion.

Because I planned to add incremental improvements over time, updating both trial and retail versions on a regular basis, one of my main motivations for scripting the build process was to make all of this a single-click process.

The 365-day cut-off that I'd built into the trail version using the OnGuard component would obviously be a problem for users who downloaded it towards the end of this time-frame. So I wanted to include updates to the serial numbers as a part of the overall build process. That way, with frequent (bi-monthly) builds and web uploads, users would never be exposed to a “stale” trial product that was nearing the end of its shelf life.

Version Control System

When it came to choosing a Version Control System for managing my source code and associated project files, I wanted something that had some integration with the Delphi IDE. There are countless VCS packages to choose from - Borland's StarTeam, Mercurial, CVS, Subversion, Bitkeeper and Git, to name a small number of the more well known examples – but I wanted to stick to the Delphi ecosystem so I settled on the JEDI VCS, hosted at SourceForge.

It doesn't support distributed revision control (http://bit.ly/dtDJ1), which stores a local copy of the repository for each client, allowing many users to work with the code base offline and tools for merging changes, but this was a single user project so that was never going to be an issue. Regardless, the more traditional approach of the JEDI VCS, with a single central database, and file check-in/check-out is notionally simpler, and it's proven to be a very reliable solution (the client is chock full of features, and has all sorts of useful built-in tools for doing things like side-by-side diffs).

One of the features I've come to appreciate most is the Label Manager. Quoting from the supplied Help File:

"Labeling modules [a JEDI VCS term for any file that can be stored in the repository] is recommended as a way to freeze a moment in the development cycle. With a label, you can easily find, get and work with a development state that has been identified as significant in the development cycle."

By creating a new Label and then “stamping” (associating it with) all files used in a build, I can create a “snapshot” of my project at the time of compilation. If a problem is later discovered with that executable, I can go back into the JEDI VCS and recreate the project exactly as it was when the executable was compiled, and verify whether or not the bug has been fixed in later releases.

The JEDI VCS client can be driven from the command line, so it was trivial to add a batch file command for doing this automatically for each build.

Step four creates the application Help files: these are supplied in two formats, Compiled HTML (.chm) for Windows 98 and later, and WinHelp (.hlp) for Windows 95. Again, there are dozens of different editors, commercial and freeware (http://bit.ly/kmjnc2 – several are Delphi applications, Help & Manual is one, Precision Helper is another, and freeware to boot), that can be used to write documentation, but I use Help & Manual from EC Software.
Help & Manual can be launched from the command line with switches for automating different tasks, such as “compiling” written content into one or more help file formats (HTML, PDF, CHM, etc.). I've included the batch file (HelpFile.bat) I use for this step in the companion files archive at the end of the article, but because of the diversity of help file authoring software, it's not likely to be of much use without the necessary modifications to support other clients (if these support command line compiling at all).



The fifth step pulls together all of the files distributed with the application into a single stand-alone executable. I use Jordan Russell's Inno Setup (http://bit.ly/17JDQN) to create an installer that can be downloaded from my website.
All of the logic for identifying which files to include, whether or not compression should be used on included files, which platforms to target and so on is contained within a handful of Inno Setup scripts that are parsed by the Inno compiler. Therefore the batch file (Installers.bat) which launches Inno Setup and directs it to compile these scripts is relatively short and simple. Because two separate installers are created, one for the shareware, one for the retail version, there is some special handling; a FOR loop calls Inno Setup once for each of these versions and updates a text file containing a “build number” (shown on the installer title bar) as it goes.


The sixth and final step takes the finished installers and uploads them by FTP to my website. This batch file, FTPUpload.bat:
  • Sets variables for the FTP host, FTP server path and local path to the installer executables
  • Uses a command line FTP client to upload the installers to the FTP server
Windows includes a command line FTP utility – called “ftp” - but repeated upload tests always resulted in corruption of any binary files sent to the server. A Delphi console application using the Indy socket library was created instead, and this is now used to transmit the installers to the server.


Step six rounds out the cycle. The entire process takes about 10 minutes, but once I've hit RETURN on the first batch file in the sequence, it all runs to completion unattended. I encountered some problems early on in testing that seemed to point to an obscure issue with the command line interpreter (a highly unlikely though not unprecedented scenario! http://bit.ly/eYx8pA), but lengthy investigation revealed my anti-virus software to be the culprit. It was interfering with Inno Setup during compilation (as a result of heuristic scanning of the installer executable as it was being assembled); disabling it during active builds prevented the error from re-occurring.

3rd Party Tools

Although there's a tremendous amount that you can do using only commands available from the interpreter (i.e., supplied with the Windows operating system), constructing sub-routines to perform, for example, complex file parsing can be extremely long-winded.

As a short-cut, or where some functionality beyond the scope of what's available with native commands is required, I've made use of a set of console-only tools that have been ported from the UNIX world.

There are several projects that have a similar ambition, Cygwin and Gow are very popular, but these are either more heavyweight than I needed, or include installers that make modifications to the Windows environment with a view to extending the POSIX capabilities of the platform.

Instead I opted for the SourceForge hosted UnxUtils package (http://bit.ly/aCjSI), a sub-1MB zip file containing a little over 30 common GNU utilities (things like sed, grep and tail).

Several of the sub-routines in my UtilityFunctions.bat file make use of these tools or other small, single-task console applications that I've written myself. These are all trivial, but included in the companion files archive at the end.


1 Local Environment Variables are only valid within a single instance of the command shell. Once the shell is closed, they are disposed of. System Environment Variables are operating system wide, and common to all instances of the command shell.

Companion Files:

Example batch file scripts and additional command line utilities (1.06 MB):

http://www.mediafire.com/?e8a833tahsekhih

No comments:

Post a Comment