<?xml version="1.0" encoding="UTF-8"?>
<!-- $Id: building.xml 18246 2013-02-06 19:48:23Z dizzzz $ --><book><bookinfo><productname>eXist-db – Open Source Native XML Database</productname><title>Building eXist-db</title><date>September 2009</date><author><firstname>Wolfgang M.</firstname><surname>Meier</surname><affiliation><address format="linespecific"><email>wolfgang at exist-db.org</email></address></affiliation></author><author><firstname>Dannes</firstname><surname>Wessels</surname><affiliation><address format="linespecific"><email>dannes at exist-db.org</email></address></affiliation></author><author><firstname>Leif-Jöran</firstname><surname>Olsson</surname><affiliation><address format="linespecific"><email>ljo at exist-db.org</email></address></affiliation></author></bookinfo><chapter><title>Building eXist-db</title><section id="war"><title>Building a WAR from the Standard Distribution</title><para>If you would like to deploy eXist-db into an existing servlet engine (e.g. tomcat), you will need a
                    <filename>.war</filename> file. We no longer distribute a <filename>.war</filename> build, but it is
                easy to create one from the standard distribution by <emphasis>repackaging</emphasis> (not building)
                it:</para><procedure><step><title>Install the standard distribution</title><para>Get a standard distribution from the <ulink url="https://sourceforge.net/projects/exist/files/Stable/2.0/">download
                            page</ulink> on Sourceforge. Install it into a directory of your choice.
                        Make sure you include the <emphasis>source package</emphasis>.</para></step><step><title>Generate the WAR</title><para>Open a command line prompt and change to the directory where you just installed eXist-db.
                        Call</para><synopsis>build.sh dist-war</synopsis><para>or</para><synopsis>build.bat dist-war</synopsis><para>once. This will generate a <filename>.war</filename> archive into directory
                            <filename>dist</filename>.</para></step></procedure><para>Note that generating a WAR archive does only repackage the files in the distribution, so if the
                distribution worked properly for you, the WAR should as well. You don't need to recompile eXist-db or sign
                any jars.</para></section><section id="svn"><title>Building eXist-db from SVN</title><para>Building eXist-db is quite easy and requires 3 simple steps:</para><procedure><step><title>Check out from SVN</title><para>The SVN URL for the development trunk is:</para><synopsis>http://svn.code.sf.net/p/exist/code/</synopsis><para>You need an SVN client to check out the source code. Examples for SVN clients are Tortoise SVN
                        on the windows platform or the command line SVN tools on Unix (see <ulink url="#checkout">below</ulink>). The <ulink url="http://oxygenxml.com">oXygen XML editor</ulink> ships with
                        its own SVN client.</para></step><step><title>Set JAVA_HOME</title><para>Before starting the build, your JAVA_HOME environment variable should be set to point to the
                        root directory of the JAVA JDK (note: JDK, not JRE!). Open a console or a DOS box on windows and
                        type:</para><synopsis>set JAVA_HOME=c:\path\to\jdk</synopsis><para>on windows or</para><synopsis>export JAVA_HOME=/path/to/jdk</synopsis><para>on unix.</para></step><step><title>Build eXist</title><para>Change into the eXist-db directory you just checked out and call</para><synopsis>build.bat</synopsis><para>on windows or on unix:</para><synopsis>build.sh</synopsis></step></procedure></section><section id="build-system"><title>Using the Build System</title><para>You can rebuild all eXist-db packages from the standard distribution or the subversion repository. You
                can even create a new distribution from the one you installed. For example, assume you installed eXist
                using the installer GUI, but you need to have an exist.war package for deployment within a different
                servlet engine. Just call the main build script (<filename>build.sh</filename> or
                    <filename>build.bat</filename>) and pass it the target <option>dist-war</option>:</para><synopsis>build.sh dist-war</synopsis><para>This will create a fresh <filename>.war</filename> archive in the <filename>dist</filename>
                directory.</para><note><para>To be able to use the build system, you need to include the "source" module when selecting
                    installable packages in the installer GUI.</para></note><para>eXist-db relies on Ant for all compile and build tasks. Ant is included in the distribution (directory
                    <filename>tools/ant</filename>). As already mentioned above, there are two shell scripts to start
                Ant: <filename>build.sh</filename> or <filename>build.bat</filename>. The shell scripts will take care
                to create a proper command line to launch Java with the Ant version included in the distribution.</para><para>Calling Ant without further argument will compile the sources and create the main jar-files
                    (<filename>exist.jar</filename>, <filename>exist-optional.jar</filename>,
                    <filename>start.jar</filename>).</para><synopsis>build.sh -p</synopsis><para>lists the available build targets. Passing one of these as argument to ant will call execute the
                corresponding target. The main targets are:</para><table><title>Useful Build Targets</title><tgroup cols="2"><tbody><row><entry>all</entry><entry><para>Compile the sources and create the main jar-files
                                        (<filename>exist.jar</filename>, <filename>exist-optional.jar</filename>,
                                        <filename>start.jar</filename>).</para></entry></row><row><entry>benchmark</entry><entry>Performs the benchmark tests for eXist-db.</entry></row><row><entry>clean</entry><entry>Cleans the files and directories generated by a previous build. Data is preserved.</entry></row><row><entry>clean-all</entry><entry>Cleanup deeper.</entry></row><row><entry>clean-conf</entry><entry>Cleanup config file.</entry></row><row><entry>clean-default-data-dir</entry><entry>Remove contents in default data directory.</entry></row><row><entry>compile</entry><entry>Compiles the source code.</entry></row><row><entry>compile-aspectj</entry><entry>Compiles the aspects.</entry></row><row><entry>dist-tgz</entry><entry>Packages eXist-db into a <filename>.tgz</filename> file.</entry></row><row><entry>dist-war</entry><entry>Packages eXist-db into a <filename>.war</filename> file for deployment within a servlet
                                engine.</entry></row><row><entry>dist-zip</entry><entry>Packages eXist-db into a <filename>.zip</filename> file.</entry></row><row><entry>download-additional-jars</entry><entry>Download optional third-party jar files. These are currently: saxon. These
                                are downloaded into <filename>lib/user/</filename>.
                            </entry></row><row><entry>download-icu4j</entry><entry>Download the optional icu4j library.</entry></row><row><entry>download-xars</entry><entry>Download xars to include in installer.</entry></row><row><entry>download-xqts</entry><entry>Download XQTS file.</entry></row><row><entry>download-xslts</entry><entry>Download XSLTS file.</entry></row><row><entry>extension-modules</entry><entry>Build the extensions directory. Edit the file <filename>extensions/build.properties</filename> enable extensions and modules.</entry></row><row><entry>installer</entry><entry>Creates a new installer package based on <ulink url="http://izpack.org/">IzPack</ulink>.
                            </entry></row><row><entry>installer-exe</entry><entry>Create <filename>.exe</filename> from installer</entry></row><row><entry>jar</entry><entry>The default: compiles the source and creates eXist's main <filename>.jar</filename>
                                files.</entry></row><row><entry>javadocs</entry><entry>Creates the API docs into <filename>webapp/api</filename>.
                            </entry></row>
                        <!--<row>
                            <entry>javadocs-uml</entry>
                            <entry>
                                <para>Creates the API docs along with UML diagrams into
                                        <filename>webapp/api</filename>
                                </para>
                                <note>
                                    <para>This build target takes a long time to run, because it is generating graphics
                                        for every package in eXist-db.</para>
                                </note>
                            </entry>
                        </row>--><row><entry>jdepend</entry><entry>JDepend traverses Java class file directories and generates design quality metrics
                                for each Java package.</entry></row><row><entry>jetty-keygen</entry><entry>Generate keystore for jetty.</entry></row><row><entry>jnlp-clean</entry><entry/></row><row><entry>jnlp-keygen</entry><entry>Generate keystore for signing jars.</entry></row><row><entry>jnlp-pack200</entry><entry>Pack all jar files.</entry></row><row><entry>jnlp-sign-all</entry><entry>This signs all of the jar files, so that the webstart of the <ulink url="java-admin-client.xml">eXist-db client Java application</ulink> will launch without any signing
                                errors.</entry></row><row><entry>jnlp-sign-core</entry><entry>Sign all CORE jar files.</entry></row><row><entry>jnlp-sign-exist</entry><entry>Sign all EXIST jar files.</entry></row><row><entry>jnlp-unsign-all</entry><entry>This unsigns all of the jar files.</entry></row>
                        <!--<row>
                            <entry>pmd</entry>
                            <entry>
                                <para>PMD scans Java source code and looks for potential problems like: <variablelist>
                                        <varlistentry>
                                            <term>Possible bugs</term>
                                            <listitem>empty try/catch/finally/switch statements</listitem>
                                        </varlistentry>
                                        <varlistentry>
                                            <term>Dead code</term>
                                            <listitem>unused local variables, parameters and private methods</listitem>
                                        </varlistentry>
                                        <varlistentry>
                                            <term>Suboptimal code</term>
                                            <listitem>wasteful String/StringBuffer usage</listitem>
                                        </varlistentry>
                                        <varlistentry>
                                            <term>Overcomplicated expressions</term>
                                            <listitem>unnecessary if statements, for loops that could be while
                                                loops</listitem>
                                        </varlistentry>
                                        <varlistentry>
                                            <term>Duplicate code</term>
                                            <listitem>copied/pasted code means copied/pasted bugs</listitem>
                                        </varlistentry>
                                    </variablelist>
                                </para>
                            </entry>
                        </row>--><row><entry>rebuild</entry><entry>Performs a <emphasis role="italic">clean</emphasis> and the <emphasis role="italic">all</emphasis>.
                            </entry></row><row><entry>samples</entry><entry>Builds the examples and packs them into <filename>examples.jar</filename>.
                            </entry></row><row><entry>snapshot-installer</entry><entry>Creates a snapshot installer.</entry></row><row><entry>svn-diff</entry><entry>Makes a patch file.</entry></row><row><entry>svn-download</entry><entry>This downloads a jar file that places the current revision number of the working copy
                                of eXist-db into VERSION.txt. This allows for the tagging of the build with the revision
                                number.</entry></row><row><entry>svn-update</entry><entry>Update project from SubVersion archive.</entry></row><row><entry>test</entry><entry>Runs all tests in the test suite and outputs the HTML result into
                                    <filename>test/junit/html</filename>. This may take several minutes.</entry></row><row><entry>xquery</entry><entry>Regenerate the XQuery parser code with ANTLR (only required if you change
                                    <filename>src/org/exist/xquery/parser/XQuery.g</filename>).</entry></row><row><entry>xars</entry><entry>Scan apps directory and include all .xar files into installer.</entry></row></tbody></tgroup></table><note><para>You might want to set some custom settings (in particular your HTTP proxy when external libraries
                    are required) before running <filename>build.sh</filename> or <filename>build.bat</filename>. See
                        <filename>build.properties</filename>.</para></note></section><section id="svn-repo"><title>Using the Subversion Repository</title><para>Since spring 2006, the entire code base of eXist-db is managed by the <ulink url="http://sourceforge.net/svn/?group_id=17691">Subversion repository at Sourceforge</ulink>.
                Before that we used CVS, which had a few issues, especially due to the separation into a developer's and
                anonymous server. Using Subversion should be much simpler, faster and more reliable.</para><para>Fortunately, you don't need to be a hardcore Java developer to use the SVN version of eXist-db. In
                general, we try to keep the development version in a consistent state. The code should always compile
                and run the test cases, though temporary failures are possible, especially while a larger update is in
                progress.</para><para>The repository root URL is:</para><synopsis>http://svn.code.sf.net/p/exist/code/</synopsis><important><para>Please note you should use https or svn+ssh, not http, if yo are a developer with commit rights.</para></important><para>The current development trunk is at:</para><synopsis>http://svn.code.sf.net/p/exist/code/trunk/eXist</synopsis><para>This is the URL you will normally use for a checkout of the main eXist-db development tree.</para><section id="checkout"><title>Checking Out from the Repository</title><para>There are many Subversion clients to choose from. If you want to actively contribute to eXist-db or
                    otherwise work on eXist's source code, we recommend to use eclipse to check out the sources (see
                    below) and setup a project. Otherwise, you can use any other SVN client. On Linux, I prefer the
                    plain command line. For example, the following command checks out the main development trunk into a
                    directory called eXist-trunk:</para><synopsis>svn co http://svn.code.sf.net/p/exist/code/trunk/eXist eXist-trunk</synopsis><para>A good choice on the Windows platform is <ulink url="http://tortoisesvn.tigris.org/">Tortoise
                        SVN</ulink>. Tortoise is an extension to the windows shell. The following screenshot shows the
                    checkout dialog:</para><screenshot><graphic fileref="resources/tortoise.png"/></screenshot><para>The repository already contains project configuration files for eclipse, so setting up the project
                    should be pretty simple: you normally don't need to change much in order to dive into the source
                    code (though you may need some customizations to build and run directly from within eclipse).</para><para>To access a Subversion repository from within eclipse, you need to install the <ulink url="http://subclipse.tigris.org/">subclipse</ulink> plugin first! Please follow the <ulink url="http://subclipse.tigris.org/install.html">install instructions</ulink> available
                    there.</para><para>Once you installed subclipse, change to the perspective: "SVN Repository Exploring". In the "SVN
                    Repository" tab of the left view, right click into the empty space and select
                    <guimenu>New</guimenu>, then <guimenu>Repository Location</guimenu>. In the appearing dialog, enter
                    the URL of the eXist-db development trunk: </para><screenshot><graphic fileref="resources/subclipse1.png"/></screenshot><para>The URL will now be listed in the "SVN Repository" view. Right click the URL and select
                        <guimenu>Checkout</guimenu>. A progress window appears, saying that eclipse is "getting remote
                    info". Once it finished, you should see the following dialog:</para><screenshot><graphic fileref="resources/subclipse2.png"/></screenshot><para>Just press <guimenu>Finish</guimenu> unless you want to specify a different directory or checkout
                    a certain tag.</para></section></section><section id="webstart"><title>Signing jar files for webstart</title><para>The database can be easily accessed with the Java client, when the client is launched by clicking the
                    <ulink url="http://en.wikipedia.org/wiki/Java_Web_Start">Java Webstart</ulink> "Launch" button on
                the right side of this page. Java webstart explicilty requires that all application code, the 'jar'
                files, are <ulink url="http://java.sun.com/docs/books/tutorial/deployment/jar/signing.html">signed</ulink> using a security certificate. Out of the box, when eXist-db is installed using the
                installer, all relevant jar files have already been signed, so webstart can be used directly. It is
                nessecary however to (re-)sign the jar files yourself if you want to (re-)build the eXist-db java code.
                This chapter explains how to do this.</para><section id="quickstart"><title>Quickstart</title><para>To get started quickly, just execute the following command:</para><synopsis>./build.sh -f build/scripts/jarsigner.xml</synopsis><para>This command sequentially creates the key.store file and signs a selection of jar files:</para><example><title>Jarsigner.xml Output</title><screen>./build.sh -f build/scripts/jarsigner.xml 
Starting Ant...

Buildfile: ..../eXist-db/build/scripts/jarsigner.xml

jnlp-prepare:

jnlp-keygen:
   [genkey] Generating Key for exist

jnlp-sign-exist:
  [signjar] Signing JAR: ..../eXist-db/exist-fluent.jar to ..../eXist-db/exist-fluent.jar as exist
  [signjar] Signing JAR: ..../eXist-db/exist-optional.jar to ..../eXist-db/exist-optional.jar as exist
  [signjar] Signing JAR: ..../eXist-db/exist.jar to ..../eXist-db/exist.jar as exist
  [signjar] Signing JAR: ..../eXist-db/start.jar to ..../eXist-db/start.jar as exist

jnlp-sign-core:
  [signjar] Signing JAR: ..../eXist-db/lib/core/antlr-2.7.7.jar to ..../eXist-db/lib/core/antlr-2.7.7.jar as exist
  [signjar] Signing JAR: ..../eXist-db/lib/core/commons-collections-3.2.1.jar to ..../eXist-db/lib/core/commons-collections-3.2.1.jar as exist
  ....
  [signjar] Signing JAR: ..../eXist-db/lib/core/xmlrpc-server-3.1.3.jar to ..../eXist-db/lib/core/xmlrpc-server-3.1.3.jar as exist
</screen></example><para>Note that this command needs to be executed just once. In subsequent builds newly generated jars
                    are signed automatically. </para></section><section id="ant-targets"><title>Ant targets</title><para> An overview of all available ant targets can be retrieved by executing the following
                    command:</para><synopsis>./build.sh -projecthelp -f build/scripts/jarsigner.xml </synopsis><para>The following ant targets are available for signing jar files:</para><para><variablelist><varlistentry><term>jnlp-all</term><listitem><para>Create keystore file and sign all EXIST and CORE jar files. <emphasis role="bold">(Default task)</emphasis></para></listitem></varlistentry><varlistentry><term>jnlp-keygen</term><listitem><para>Generate a new keystore file if not present.</para></listitem></varlistentry><varlistentry><term>jnlp-clean</term><listitem><para>Delete the keystore file.</para></listitem></varlistentry><varlistentry><term>jnlp-sign-core</term><listitem><para>Sign all CORE jar files in <filename>lib/core</filename>.</para></listitem></varlistentry><varlistentry><term>jnlp-sign-exist</term><listitem><para>Sign all EXIST jar files, e.g. exist.jar, exist-XX.jar and start.jar</para></listitem></varlistentry><varlistentry><term>jnlp-unsign-all</term><listitem><para>Unsign all jar files.</para></listitem></varlistentry></variablelist></para><para>The target <emphasis role="bold">jnlp-keygen</emphasis> generates a new certificate file
                        <filename>key.store</filename>. If this file is present the <emphasis role="bold">EXIST jar
                        files</emphasis> are signed during each following build. The process of signing jar files can be
                    stopped by removing this keystore file.</para><para>The ant script contains two additional targets: <emphasis role="bold">jetty-keygen</emphasis>
                    which generates a keystore file for the jetty server and <emphasis role="bold">jnlp-pack200</emphasis> which processes the signed jar files with <emphasis role="bold">pack200</emphasis> technology. More about the latter in the next section.</para></section><section id="pack200"><title>Reducing jar sizes with pack200 compression</title><para>Pack200 is a compression technology introduced in Java 1.5.0. It has been designed for compressing
                    jar files and works most efficiently on Java class files. Pack200 can reduce the size of a jar file
                    up to 60% resulting into a significant reduction of the amount of downloaded webstart data.</para><para>The webstart jar files can be compressed with pack200 with the following command:
                    <synopsis>./build.sh -f build/scripts/jarsigner.xml jnlp-pack200</synopsis> This command
                    consecutively performs the following steps for all jar files: <itemizedlist><listitem><para>unsign</para></listitem><listitem><para>repack (normalization)</para></listitem><listitem><para>sign</para></listitem><listitem><para>compress with pack200</para></listitem><listitem><para>compress with gzip</para></listitem></itemizedlist></para><para><example><title>pack200 Output</title><screen>
                    ./build.sh -f build/scripts/jarsigner.xml jnlp-pack200
                    Starting Ant...
                    
                    Buildfile: .../eXist-db/build/scripts/jarsigner.xml
                    
                    jnlp-pack200:
                    
                    jnlp-unsign-all:
                    [unsignjar] Unsigning exist-fluent.jar
                    [unsignjar] Unsigning exist-optional.jar
                    [unsignjar] Unsigning exist.jar
                    [unsignjar] Unsigning start.jar
                    [unsignjar] Unsigning antlr-2.7.7.jar
                    ...
                    [unsignjar] Unsigning xmlrpc-server-3.1.3.jar
                    [repack] Normalizing exist-fluent.jar
                    [repack] Normalizing exist-optional.jar
                    [repack] Normalizing exist.jar
                    [repack] Normalizing start.jar
                    [repack] Normalizing antlr-2.7.7.jar
                    ...
                    [repack] Normalizing xmlrpc-server-3.1.3.jar
                    
                    jnlp-prepare:
                    
                    jnlp-keygen:
                    
                    jnlp-sign-exist:
                    [signjar] Signing JAR: .../eXist-db/exist-fluent.jar to .../eXist-db/exist-fluent.jar as exist
                    [signjar] Signing JAR: .../eXist-db/exist-optional.jar to .../eXist-db/exist-optional.jar as exist
                    [signjar] Signing JAR: .../eXist-db/exist.jar to .../eXist-db/exist.jar as exist
                    [signjar] Signing JAR: .../eXist-db/start.jar to .../eXist-db/start.jar as exist
                    
                    jnlp-prepare:
                    
                    jnlp-keygen:
                    
                    jnlp-sign-core:
                    [signjar] Signing JAR: .../eXist-db/lib/core/antlr-2.7.7.jar to .../eXist-db/lib/core/antlr-2.7.7.jar as exist
                    ...
                    [signjar] Signing JAR: .../eXist-db/lib/core/xmlrpc-server-3.1.3.jar to .../eXist-db/lib/core/xmlrpc-server-3.1.3.jar as exist
                    [pack] Packing exist-fluent.jar
                    [pack] Packing exist-optional.jar
                    [pack] Packing exist.jar
                    [pack] Packing start.jar
                    [pack] Packing antlr-2.7.7.jar
                    ...
                    [pack] Packing xmlrpc-server-3.1.3.jar
                    
                    BUILD SUCCESSFUL
                    Total time: 1 minute 0 seconds
                        </screen></example></para><para>The pack200 compression feature works with Java SE 6 update 10 
                    (<ulink url="http://java.sun.com/developer/technicalArticles/javase/java6u10/#usability">Java6u10</ulink>) or
                    newer. Older Java6 versions will only receive the larger jar files.</para></section><section id="references"><title>References</title><para><itemizedlist><listitem><para><ulink url="http://en.wikipedia.org/wiki/Java_Web_Start">Java Web Start</ulink>
                                (Wikipedia)</para></listitem><listitem><para>Java Web Start: <ulink url="http://en.wikipedia.org/wiki/Java_Web_Start#Pack200_compression">Pack200
                                    compression</ulink> (Wikipedia)</para></listitem><listitem><para><ulink url="http://en.wikipedia.org/wiki/Pack200">Pack200</ulink> (Wikipedia)</para></listitem><listitem><para><ulink url="http://wiki.eclipse.org/Pack200">Pack200</ulink> (Eclipse.org)</para></listitem><listitem><para><ulink url="http://java.sun.com/j2se/1.5.0/docs/guide/deployment/deployment-guide/pack200.html">Pack200 and Compression</ulink> for network deployment (Sun, bit out dated)</para></listitem><listitem><para>Introducing Java SE 6 update 10: <ulink url="http://java.sun.com/developer/technicalArticles/javase/java6u10/#usability">Deployment Usability</ulink> (Sun).</para></listitem><listitem><para>Ant tasks: <ulink url="http://code.google.com/p/existdb-contrib/wiki/ASOCAT">ASOCAT</ulink>.</para></listitem></itemizedlist></para></section><section id="notes"><title>Notes</title><para><itemizedlist><listitem><para>Sometimes when the jar files are signed, eXist-db does not start anymore. In this situation
                            unsign the jar files first. Calling target 'jnlp-pack200' does always unsign all jar files as first step.</para></listitem><listitem><para>The pack200 target is not executed automatically. If eXist-db code is changed and
                                recompiled, you might want to execute the jnlp-pack200 target again.</para></listitem></itemizedlist></para></section></section></chapter></book>