replaceregexp workaround/bug

[Dennis Putnam]

Since you folks were so helpful finding my dopey problem with my java command (I still can't believe how dumb I was), I have a little more challenging (I think) problem for you. You may have noticed in that thread (https://coderanch.com/t/567843/tools/Linux-vs-Windows-jar-file) that I am using replaceregexp in the posted XML file.

I have stumbled onto an old bug (https://issues.apache.org/bugzilla/show_bug.cgi?id=34633) that seems to not really have been resolved. If you look at the last entry, that is mine and the crux of the problem is that while replaceregexp works from command line, it does not work in background. I need to automate building my jar file via a web service which means building the jar in background. All that is working except for this bug. Can anyone help me come up with an alternative way to accomplish what I'm trying to do? TIA.

[Peter Johnson]

Rather than replacing in-place, why not have the original file in a different location and use the copy task with a ReplaceRegex filterchain? Or what I usually do is have a source file containing property references and then use copy with the replaceproperty filterchain. The source is unusable without being filtered, but you are essentially always filtering the file anyway.

[Dennis Putnam]

Thanks. That sounds like a possible solution. Can you point me to an example?

[Peter Johnson]

Here is a very simple example. The directory structure is the one used by Maven (actually, I use Maven rather that Ant for compiling, but Ant should do just as well), and I strictly follow the rule that the source files and folders never get updated during a build, and all files created by the build go into the target directory.

Here is the directory structure for the pertinent file:

build.xml
src/main/resources/config.xml

Here is the content of config.xml:

<config>
  <url>http://${host.name}/app</url>
</config>

Here is the copy statement in build.xml, with hard-coded directories:

<copy todor="target/classes">
  <fileset dir="src/main/resources" />
  <filterchain>
    <expandproperties />
  </filterchain>
</copy>

Then during the build I provide a value for the host.name property. Exactly how it is provided doesn't matter, perhaps as a -D value passed to Ant:

ant -Dhost.name=example.com

After the build, here are the pertinent files. Files in black are unchanged, files in blue are created:

build.xml
src/main/resources/config.xml
target/classes/config.xml

And the contents of target/classes/config.xml file:

<config>
  <url>http://example.com/app</url>
</config>

That should give you the gist of the technique.

[Dennis Putnam]

Thanks for the info but I guess I am missing something basic. Here is my current XML file:
<?xml version="1.0" encoding="UTF-8"?> <project name="KCBSEvents" default="jar" basedir="."> <property name="build.properties" value="src/KCBSEvents/resource/build.properties"/> <property name="propertysrc" value="buildsrc.properties" /> <property name="excludelist" value="build.bat,makejar.xml" /> <target name="checkOS"> <condition property="isWindows"> <os family="windows" /> </condition> <condition property="isLinux"> <os family="unix" /> </condition> </target> <target name="if_windows" depends="checkOS" if="isWindows"> <property name="jarfile" value="C:\temp\KCBSEvents.jar" /> <property name="antcontrib" value="H:\html\Applets\ant-contrib" /> </target> <target name="if_linux" depends="checkOS" if="isLinux"> <property name="jarfile" value="/tmp/KCBSEvents.jar" /> <property name="antcontrib" value="/usr/share/ant/lib" /> </target> <target name="setclass" depends="if_linux,if_windows"> <taskdef resource="net/sf/antcontrib/antcontrib.properties"> <classpath> <pathelement location="${antcontrib}" /> </classpath> </taskdef> </target> <loadproperties srcfile="${propertysrc}"> <filterchain> <linecontains> <contains value="=" /> </linecontains> </filterchain> </loadproperties> <target name="incserial" depends="setclass"> <math result="newserialnumber" operand1="${serialnumber}" operation="+" operand2="1" datatype="int" /> <echo message="Created new serial number: ${newserialnumber}" /> <copy file="${propertysrc}" tofile="${build.properties}"> <filterchain> <replaceregex byline="false" pattern="serialnumber=${serialnumber}" replace="serialnumber=${newserialnumber}" /> </filterchain> </copy> <copy tofile="${propertysrc}" file="${build.properties}" /> </target> <target name="jar" description="Compile serialized jar" depends="incserial,if_windows,if_linux"> <echo message="Using destination file ${jarfile}" /> <jar destfile="${jarfile}" basedir="bin" excludes="${excludelist}"> <manifest> <attribute name="Manifest-Version" value="1.0"/> <attribute name="Created-By" value="1.6.0 (Sun Microsystems Inc.)" /> <attribute name="Main-Class" value="KCBSEvents.KCBSEvents" /> </manifest> </jar> </target> </project>
Note the target 'incserial". The <echo> indicates that correct calculation but when I look at the resultant files (${propertysrc} and ${build.properties}) neither is changed. It appears that the copies simply do nothing. What I am trying to do is have an initial file, ${propertysrc}, whose properties are loaded (via <loadproperties>). That file is then copied to ${build.properties} with an incremented serial number. That resulting file is then copied back to ${propertysrc} so the next run will again increment a new serial number.

[Peter Johnson]

Your code works for me. When the build runs, do you see output like this for the incserial target:

incserial:
     [echo] Created new serial number: 3
     [copy] Copying 1 file to D:\temp\ranch\src\KCBSEvents\resource
     [copy] Copying 1 file to D:\temp\ranch

Specifically, do you see both copy statements? If you don't see both of them, then most likely the copy was not performed because the target class had a later timestamp than the source file.

Also, what are the contents of the buildsrc.properties file?

Finally, what you are doing is not what I suggested. In my suggestion, the file at src/KCBSEvents/resource/build.properties would contain this:

serialnumber=${serialnumber}

and the buildsrc.properties file would contain:

serialnumber=2

Then the copy task would like like this:

<copy file="${build.properties}" tofile="???" >
  <filterchain>
    <replaceproperties />
  </filterchain>
</copy>

I didn't provide a tofile value because I cannot tell from your build script where the file ends up (it doesn't go into the JAR file, so what is it used for?). You should not update anything in the src directory, that is usually a good way to really mess things up. Always copy things from the src directory into a target directory, such as bin or output or target or dist. This makes the "clean" task easier to write - just delete the output directory. So again, where does the file fit into the final output?

By the way, is the some reason you are not using the BuildNumber task? http://ant.apache.org/manual/Tasks/buildnumber.html
If you did, then the incserial target would look like this (assuming the properties file goes into the bin directory), and you wouldn't need to use the ant-contrib tasks:

<target name-"incserial" />
  <buildnumber />
  <copy file="${build.properties}" todir="bin" >
    <filterchain>
      <replaceproperties />
    </filterchain>
  </copy>
</target>

The file build.number would contain only this, nothing else:

2

And the src/KCBSEvents/resource/build.properties would the entrire set of properties previous defined in buildsrc.properties, including this line

serialnumber=${build.number}

In addition, you would not need the loadproperties task.

[Dennis Putnam]

Thanks for the reply. No, I am not seeing either copy statement. As for the rest of the code I need to look through it to understand what you are suggesting. A cursory glance seems better than my idea. I'm not sure when I'll be able to do that but I'll get back to you when I do.

[Peter Johnson]

If you need a more complex example, grab the source code for JBoss in Action (use the link in my signature). Then take a look at the ch00/src/main/db/jbia-ds.xml file, and look at file ch00/build-include.xml, specifically the targets g-init-db and g-db-copy. An example of the property values used to can be found in ch00/build.properties.

[Dennis Putnam]

Now that I've had a chance to look at your suggestion, I like it much better than my way. I didn't use buildnumber earlier because I didn't know it existed. However you brought up some other issues that raised questions in my mind. The build.properties file is really misnamed as it contains several parameters used at run time that are specific to the user of that particular copy of the application. Perhaps I should have named it runtime.properties but that was named when I was even greener than I am now. I have that file in src/resources. When I build the jar file that file shows up in resources in the jar. Whether that is normal practice or not I don't know but it works. That is the file that needs to be updated with the serial number I am trying to generate. Using your suggestion, I put the skeleton build properties (with the ${serialnumber} parameter) outside src. From there I think the copyfile task uses that skeleton, updates the serial number and plugs it in. Here is where it gets fuzzy for me. You show the destination as bin. My thinking is that the destination should be src/resources. Since I am still not up to speed on how all this stuff is put together into a jar file, I don't really know what is right. If it goes into bin, how do I access it from my java class (I mean what path would I use)? How much reprogramming would I need to do?

[Peter Johnson]

My use of the bin directory was only because I have no idea of your complete directory structure, and that is typically the default folder for compiled Java classes, and I was trying to get away from updating things in the src folder (that is always a horrible idea). For my builds I always use Maven, and in Maven properties files are placed in src/main/resources, and when they are "processed", they get copied to target/classes. In addition, all Java source files are located at src/main/java and get compiledi nto target/classes. When Maven builds the JAR file, it includes all of the files found in target/classes, or if you are building a WAR, Maven takes everything in target/classes and puts it into WEB-INF/classes in the WAR file.

Based on your description of where the properties file ends up (I assume it will be resources/runtime.properties in the JAR, using the new name), I would start the file off at:

src/main/resource/resource/runtime.properties

The build would place the resulting file at:

target/classes/resource/runtime.properties

Finally, can I assume that you are using the getResourceAsStream method to load the properties file:

Class.getResourceAsStream("resource/runtime.properties");

[Dennis Putnam]

As a noob I guess I need to make sure we are talking apples and apples. You obviously can't guess what I have and I don't want to misinterpret your valuable advice. Therefore this is the directory tree that I have via Eclipse. The tree is on a share so both Linux and Eclipse can see the same directories.



Well for some reason this forum will not display the image so here is the link:

http://www.freeimagehosting.net/qqtba

[Peter Johnson]

Based on the basedir for the <jar> task on line 48 of your build.xml file, I assume that all of the .java source files are compiled to the bin directory. You should do the same thing with the properties file: use the <copy> task to copy the properties to the bin directory, using the <replaceproperties> filterchain like I mentioned.

But you still didn't tell me how your code reads the properties file. Without that, I can only make guesses. But my recommendation to use getResourceAsStream still stands.

[Dennis Putnam]

Sorry, you are correct I do use getResourceAsStream. If I copy the properties file to bin, what path do I then use in getResourceAsStream?
This is the method I use to get the properties:

private static Registration instance=null;
private static boolean windowsOS=false;
private static boolean regFlag=false;
private static boolean chkFlag=false;
private static String name="";
private static String memberNumber="";
private static String serialNumber="";
private static String magic="";
private static String keystring="";
private static String propertiesFile="resource/build.properties";
private static Properties prop=new Properties();

.
.
.
.

public boolean isRegistered() {
		
	BufferedReader br=null;
	String data="";
	String sep="/";
	if (!chkFlag) {
		chkFlag=true;
		if (windowsOS) {
			sep="\\";
		}
		InputStream in=this.getClass().getResourceAsStream(propertiesFile);
		if (in==null) {
			regFlag=false;
			return(false);}
		try {
			prop.load(in);
		}
		catch (Exception e) {
			CountDownLatch lock=new CountDownLatch(1);
			MessageScreen.showWindow(Helpers.missingProperties(),lock);
			try {
				lock.await();
			} catch (InterruptedException e1) {
				// TODO Auto-generated catch block
				e1.printStackTrace();
			}
			System.exit(-1);
		}
		try {
			in.close();
		} catch (IOException e1) {
			// TODO Auto-generated catch block
			e1.printStackTrace();
		}
		keystring=prop.getProperty("keystring");
		serialNumber=prop.getProperty("serialnumber");
		if (keystring.equals("")) { return(false); }
		name=prop.getProperty("name");
		memberNumber=prop.getProperty("membernumber");
		magic=prop.getProperty("magicnumber");
		String file="";
		for (int i=0; i<magic.length(); i+=2) {
			file+=(char) Integer.parseInt(magic.substring(i,i+2));
		}
		String filename=System.getProperty("user.home")+sep+"."+file;
		try {
			br=new BufferedReader(new InputStreamReader(new DataInputStream(new FileInputStream(filename))));
		}
		catch (Exception e) {
			CountDownLatch lock=new CountDownLatch(1);
			MessageScreen.showWindow(Helpers.missingRegistration(),lock);
			try {
				lock.await();
			} catch (InterruptedException e1) {
				// TODO Auto-generated catch block
				e1.printStackTrace();
			}
			System.exit(-2);
		}
		try {
			data=br.readLine();
		}
		catch (Exception e) {
			CountDownLatch lock=new CountDownLatch(1);
			MessageScreen.showWindow(Helpers.corruptRegistration(),lock);
			try {
				lock.await();
			} catch (InterruptedException e1) {
				// TODO Auto-generated catch block
				e1.printStackTrace();
			}
			System.exit(-3);
		}
		try {
			br.close();
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
		MessageDigest sha1=null;
		try {
			sha1 = MessageDigest.getInstance("SHA1");
		} catch (NoSuchAlgorithmException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
		try {
			sha1.update((keystring+" "+serialNumber).getBytes("UTF-8"));
		} catch (UnsupportedEncodingException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
		byte[] hash=sha1.digest();
		if (new String(hash).equals(data)) {
			regFlag=true;
		}
	}
	return(regFlag);
}

.
.
.
.

[Peter Johnson]

Combining lines 10 and 28, you are using:

InputStream in=this.getClass().getResourceAsStream("resource/build.properties");

I would then use this copy task (I'll use hard-coded locations, you can substitute with properties as you see fit):

<copy todir="bin">
  <fileset dir="src/KCBSEvents">
    <include name="**/*.properties"/>
  </fileset>
  <filterchain>
    <replaceproperties />
  </filterchain>
</copy>

That should copy src/KCBSEvents/resource/build.properties to bin/resource/build.properties, and replace the property values within the file.

[Dennis Putnam]

I am finally getting back to this project. Unfortunately I was away long enough to be a little fuzzy on the thread so I may have forgotten a key point or two. I think I did what you suggested but I am getting an error:

filterchain doesn't support the nested "replaceproperties" element.

Here is my current code:
<?xml version="1.0" encoding="UTF-8"?> <project name="KCBSEvents" default="jar" basedir="."> <property name="build.properties" value="resource/build.properties"/> <property name="KCBSDir" value="src/KCBSEvents" /> <property name="excludelist" value="build.bat,makejar.xml" /> <target name="checkOS"> <condition property="isWindows"> <os family="windows" /> </condition> <condition property="isLinux"> <os family="unix" /> </condition> </target> <target name="if_windows" depends="checkOS" if="isWindows"> <property name="jarfile" value="C:\temp\KCBSEvents.jar" /> <property name="antcontrib" value="H:\html\Applets\ant-contrib" /> </target> <target name="if_linux" depends="checkOS" if="isLinux"> <property name="jarfile" value="/tmp/KCBSEvents.jar" /> <property name="antcontrib" value="/usr/share/ant/lib" /> </target> <target name="setclass" depends="if_linux,if_windows"> <taskdef resource="net/sf/antcontrib/antcontrib.properties"> <classpath> <pathelement location="${antcontrib}" /> </classpath> </taskdef> </target> <target name="incserial" depends="setclass"> <buildnumber /> <copy todir="bin"> <fileset dir="${KCBSDir}"> <include name="${build.properties}" /> </fileset> <filterchain> <replaceproperties /> </filterchain> </copy> </target> <target name="jar" description="Compile serialized jar" depends="incserial,if_windows,if_linux"> <echo message="Using destination file ${jarfile}" /> <jar destfile="${jarfile}" basedir="bin" excludes="${excludelist}"> <manifest> <attribute name="Manifest-Version" value="1.0"/> <attribute name="Created-By" value="1.6.0 (Sun Microsystems Inc.)" /> <attribute name="Main-Class" value="KCBSEvents.KCBSEvents" /> </manifest> </jar> </target> </project>