logback SiftingAppender generated log files cannot be deleted untill stop the tomcat server

[Mayuran Satchi]

We are using logback "SiftingAppender" for generating the log files based on the date and other information such as cycle(Each date can have multiple cycles).

Following are the sample logging file names

20150515_1_Job1.log
20150515_2_Job1.log

For the above scenario we are using the following logback configuration.

<appender name="BATCH_LOGGER_APPENDER" class="ch.qos.logback.classic.sift.SiftingAppender"> <discriminator> <key>batchLoggerFileName</key> <defaultValue>unknown</defaultValue> </discriminator> <sift> <appender class="ch.qos.logback.core.FileAppender"> <file>${batchlog.dir}/${batchLoggerFileName}.log</file> <layout class="ch.qos.logback.classic.PatternLayout"> <pattern>%d{yyyy-MM-dd_HH:mm:ss.SSS} | %-5level | [%thread] | %msg%n%rEx</pattern> </layout> </appender> </sift> </appender> <logger name="BATCH_LOGGER" level="INFO" additivity="false"> <appender-ref ref="BATCH_LOGGER_APPENDER"></appender-ref> </logger>

Following is the java code for logging the details to the specific logs files.

private static final Logger BATCH_LOGGER = LoggerFactory
        .getLogger("BATCH_LOGGER");

public void info(JobInfo jobInfo, String message) {
    MDC.put("batchLoggerFileName", jobInfo.getJobId());
    BATCH_LOGGER.info(message);
    MDC.remove("batchLoggerFileName");
}

We have some other jobs that do the housekeeping of the old log files with the retention of 2 days(Two day old files will be moved to another location).

But even after the job completes, we are not able to move the file. We suspect that logback is holding the resource and not letting the file to move or delete.

All the jobs are deployed in a single war file in apache tomcat 8 server.

Can anyone suggest how to enforce logback to release the resource after logging is completed?

Regards,
Mayuran

[Srikkanth Mohanasundaram]

Hi Mayuran,

I haven't tried this out myself but from the documentation, there are a couple of things which you might want to try out

1. Set a timeout.
2. Or send an instruction from the code that this appender is no longer needed.

..., it may be easier to pinpoint a location in your code after which a nested appender is no longer needed. If such a location exists, even approximately, log from that location using the FINALIZE_SESSION marker.

 // will cause the nested appender reach end-of-life. It will
    // linger for a few seconds.
    logger.info(FINALIZE_SESSION_MARKER, "About to end the job");

Thanks
Srikkanth

[Mayuran Satchi]

Hi Srikkanth,

Thanks for your suggestion.
I have tried your suggestion and still i was not able to delete the file while the server is running.

The following is the java class i used to log the messages and immediately i am setting the "FINALIZE_SESSION" marker.

public class BatchLoggerImpl implements BatchLogger {
	private static final Logger BATCH_LOGGER = LoggerFactory
			.getLogger("BATCH_LOGGER");

	@Override
	public void info(JobInfo jobInfo, String message) {
		MDC.put("batchLoggerFileName", jobInfo.getJobId());
		BATCH_LOGGER
				.info(ch.qos.logback.classic.ClassicConstants.FINALIZE_SESSION_MARKER,
						message);
	}
}

The following is the logback configuration.

<appender name="BATCH_LOGGER_APPENDER" class="ch.qos.logback.classic.sift.SiftingAppender"> <discriminator> <key>batchLoggerFileName</key> <defaultValue>unknown</defaultValue> </discriminator> <timeout>30 seconds</timeout> <sift> <appender name="FILE-${batchLoggerFileName}" class="ch.qos.logback.core.FileAppender"> <file>${batchlog.dir}/${batchLoggerFileName}.log</file> <layout class="ch.qos.logback.classic.PatternLayout"> <pattern>%d{yyyy-MM-dd_HH:mm:ss.SSS} | %-5level | [%thread] | %msg%n%rEx</pattern> </layout> </appender> </sift> </appender>

Since i was not able to delete the files after all, i have extended the "SiftingAppender" and overwritten the "append" method to forcefully call the "removeStaleComponents" to remove any resources used by the appender. I am not sure whether its the correct way to do or not, but i am able to delete the log files without any problem.

Following is the extended "SiftingAppender" class definition.

public class CustomSiftingAppender extends SiftingAppender {

	@Override
	protected void append(ILoggingEvent event) {
		super.append(event);
		super.getAppenderTracker().removeStaleComponents(System.currentTimeMillis() + 
				AppenderTracker.LINGERING_TIMEOUT + 1);
	}

}

Regards,
Mayuran