• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Ron McLeod
  • Paul Clapham
  • Jeanne Boyarsky
  • Liutauras Vilda
Sheriffs:
  • Rob Spoor
  • Bear Bibeault
  • Tim Cooke
Saloon Keepers:
  • Tim Moores
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Piet Souris
Bartenders:
  • Frits Walraven
  • Himai Minh

TimeZone and/or DST settings

 
Ranch Hand
Posts: 62
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello. If you have a minute, I have a servlet running a query against a Domino db and returning XML stream of results. A user obbserved that just after midnight, date values returned from records are displayed minus one day. For example, record contains 03/15/2021, but 03/14/2021 is displayed. Sometime after, the problem resolves itself and everything is cool during normal bus. hours.

I am trying to find the problem in the chain. I checked the Timezone on the server OS (Windows) and it's good (EST) and DST is set. I checked Domino server and that's good with Timezone (=5, which is EST and set to use DST). I believe Domino is set by default to use OS timezone.

Is there a place to view Tomcat's Timezone and DST settings, maybe in config somewhere? I can't find anything in server.xml, web.xml... I think Tomcat defaults to OS stuff as well but I want to be sure. Thank you so much.

PS..any other suggestions to what may be causing this day behind thing would be appreciated.

 
Saloon Keeper
Posts: 23689
161
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I'm on shaky ground with Windows these days, but my Linux machines have their hardware clocks set to UTC, not EDT.

Now that's not what I see when I log in as a user, because I can override this at the OS level and at the user level.

Tomcat does not have internal time facilities. As a general rule, webapps should work with Java's UTC-based functions (which is essentially from java.util.Date right up through Calendar and more recent developments. If you have users in different timezones, then they'd basically have to keep session information indicating their preferred zone (which doesn't need to be the zone that they're actually in!) and that information could be used to inform date/time conversions.

You might notice, for example, the the JavaRanch official timezone is US Mountain Time. Although it looks like we are localizing things like posting times on messages. Offhand that suggests that either it's deducing the timezone from the source IP address (which isn't always accurate) or from the location set in people's user profiles.
 
Thomas Griffith
Ranch Hand
Posts: 62
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks. Does Date.util and Calendar use the OS settings?

It's just really weird. All the Tomcat app does is lift two fields, action_date and action_title, from Lotus Notes and delivers in an xml stream. But somewhere in here, action_date is set back a day just after midnight then resolves. I am unsure yet if "resolution" is at 1 am but I suspect it is.
 
Tim Holloway
Saloon Keeper
Posts: 23689
161
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The java.util.Date class is a wrapper around a long integer value that represents the number of milliseconds since Midnight, Sunday Jan 1, 1970UTC. When used properly, it never reflects local timezones.

Calendar, on the other hand, takes a basic UTC Date object and localizes it, but in regard to timezone and preferred calendar, handling not only the common Western calendar, but also others such as year of the Hadjj, and the Jewish calendar thanks to its various localized subclasses.

So for maximum accuracy, you'd store items in your database in UTC format, retrieve them as date objects (such as java.sql.Date. localize them via Calendar functions, and render them via DateFormat.

You might want to check your Domino export settings.
 
Marshal
Posts: 26595
81
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I think it's very unlikely that Tomcat would use a timezone setup which differs from what any other Java application running on that same machine uses. I do think it's possible that your web app contains some coding which doesn't handle DST correctly -- I notice that the dates you reported are shortly after the start of DST in North America and Europe. I know nothing of your code of course, but perhaps it's retrieving the Domino data as string data and parsing it to a Java date type. Perhaps there's a setting in its connection to Domino which doesn't handle DST correctly, too.

I know I'm just speculating but you did invite speculation. You might consider writing a small test application which connects to Domino and gets that data, using the same code as in the servlet. Then try running that app from the command line on the same machine where Tomcat is running and see what you can observe. It's worth knowing whether the problem exists in the whole interval between midnight and 1 am or whether it goes away earlier than that.

There are barriers to that, sure. Maybe you aren't authorized to run other code on that machine. Maybe there's Domino connection security you have to deal with. And for sure you have to run this test application between midnight and 1 am. You could start by running it on your own machine to see if the same problem exists there, though.
 
Thomas Griffith
Ranch Hand
Posts: 62
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
yeah, what the servlet source looks like it's doing is reading the lotus.domino.DateTime, converting to java.util.Date (via toJaveDate() method in Domino), then applying SimpleDateFormat (mm/dd/yyyy) for the text rendering. There's no real

I still don't see how dates in the past, contained in the records, are all set back one day from what I think is 12 AM to 1 AM...then all are fine...despite any timezone and DST stuff. I have to do some more testing on this at midnight. I'm not sure if this is isolated to searches
 
Tim Holloway
Saloon Keeper
Posts: 23689
161
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Paul Clapham wrote:I think it's very unlikely that Tomcat would use a timezone setup which differs from what any other Java application running on that same machine uses.



It doesn't. There's no separate Date/Time functionality for JEE, just the same classes that ordinary Java uses. Further, Tomcat cannot meddle internally with those classes, since the core Java sandbox doesn't permit overriding the class definitions.

You can set the TimeZone for Tomcat's JVM, but only by the usual mechanisms that any other JVM would have - the OS timezone setting, the TZ environment setting for a given user/shell instance, and the -Duser.timezone option on the JVM's "java" command line. Which a quick look at the catalina.sh/catalina.bat scripts should dispel.

I looked at the toJavaDate() JavaScript function. It returns "the number of milliseconds relative to January 1, 1970 at 12:00 AM UTC". Hmmm. Where have I seen that said before?

So the webapp can safely convert that value to a long integer and construct a valid (UTC-based) java.util.Date object with no extraneous twiddling. But, again, to output that as a localized time/date value, you'd have to submit that Date object to Calendar services.
 
Paul Clapham
Marshal
Posts: 26595
81
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tim Holloway wrote:

Paul Clapham wrote:I think it's very unlikely that Tomcat would use a timezone setup which differs from what any other Java application running on that same machine uses.



It doesn't.



As I thought. However it's possible to set the time zone for a Java application on the command line, so it's possible that some well-intentioned person might have changed the command which starts Tomcat to have it use a time zone with the correct difference from UTC but without a DST change. (Such a time zone would cause the errors which Thomas has described.) Very unlikely, but perhaps not too difficult to look into?
 
Tim Holloway
Saloon Keeper
Posts: 23689
161
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Again, I don't have Windows handy, but the easy way ti see if Tomcat's timezone is being overridden in Linux at least, is to login as the Tomcat user and type the "date" command, which will display not only current date/time, but timezone in effect.

But the fact that the problem happens right after midnight but "soon corrects itself" makes me suspect some sloppy timestamp-handling logic within the webapp itself. In fact, probably brute-forcing something that should have been done with Java standard chronometric functions.

A really obscure case where legitimate logic might get thrown is on a day with a "leap second" in it. But those only happen a few times a year, and in many years, not at all.
 
Thomas Griffith
Ranch Hand
Posts: 62
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Ok, I woke up last night and conducted more tests. I realized it's not a Tomcat issue and that the JVM has it's own DateTime thing going on. I wrote a procedure in Domino to grab a Notes DateTime value from a record, then called toJaveDate(). This is a Domino method which converts the Notes DateTime to java.util.Date.

Between midnight and 1 AM, the Notes DateTime would be 03/25/2021, and the java.util.Date would be 03/24/2021. So based on what was said regarding 1/1/1970, this kinda indicates that maybe daylight savings time isn't being identified by the JVM?

I checked Windows time setting and it does adjust to Daylight Savings Time. Could this be something which could be set for an in-place JVM?


 
Tim Holloway
Saloon Keeper
Posts: 23689
161
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Show me the code that you used to convert the java.util.Date to a string.
 
Thomas Griffith
Ranch Hand
Posts: 62
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The java.util.Date (blah) is a day back between midnight and 1 AM even before the formatter.

 
Tim Holloway
Saloon Keeper
Posts: 23689
161
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Could you please not double-space your code? It's hard to read. Thanks.

You might want to try using this date format string: "yyyy.MM.dd G 'at' HH:mm:ss z".

Also, if you could provide me with an offending value of blah. And do a System.out.println(dt);

You do realize that if it was simply not honoring DST, that your time would be off by 1 hour all day long?
 
Thomas Griffith
Ranch Hand
Posts: 62
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi. Thanks. i fixed the code. Yeah, it is off by an hour all day long, but the app is only using the MM.dd/yyyy...so the date is off between midnight and 1 AM.

blah reads the Lotus Notes date field (stored as only MM/dd/yyyy), then apparently appends the current JVM time when converted to java.util.Date (blah)...which looks to be an hour behind. For example, the Lotus Notes ActionDate field reads 01/12/2021...the System.out.println's from the code...

I ran this a few minutes ago at 01:26:47 PM EST...

dt = 01/08/2021
blah = Fri Jan 08 12:26.47 EST 2021
dateText = 01/08/2021

Last night, I ran code at 12:17:38  AM EST

dt = 01/08/2021
blah = Thu Jan 07 23:17.38 EST 2021
dateText = 01/07/2021




 
Tim Holloway
Saloon Keeper
Posts: 23689
161
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
You realize that 23:17 is 11:17 PM and therefore NOT after midnight, I hope.

I have to conclude that either you read the time as being earlier than it was or that Lotus has a problem.
 
Tim Holloway
Saloon Keeper
Posts: 23689
161
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Whoops! Did you notice? That says EST, not EDT!

Unles you're in Indiana, it's likely that your Tomcat server is running a time system with Daylight Savings switched off.

 
Paul Clapham
Marshal
Posts: 26595
81
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
So it's the whole dam server which has DST turned off, not just Tomcat!
 
Thomas Griffith
Ranch Hand
Posts: 62
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi. I don't think that's it. I ran that program in Lotus Notes and it was using it's own JVM. The Tomcat logs have DST times. So I coded a quick command line java program to run directly in Windows on the Tomcat server. just prints out the current system MM/dd/yyyy HH:mm:ss z. however, when I move the java file to the tomcat server c drive, try to run it in the command window, it says Error: Could not find or load mainclass PrintTime...ClassnotFoundException

This had me thinking. When I installed the Tomcat JVM, i unzipped to a directory and pointed Tomcatw.exe (windows service) to the JVM folder. The Tomcat service runs, no oroblem. Does the JVM have to be "installed" in some other manner to run java programs and perhaps, this might be behind the issue with not recognizing system
 
Tim Holloway
Saloon Keeper
Posts: 23689
161
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
JAVA_HOME is not part of official Java. It's a convention used by many - but not all - Java applications to indicate that the application should use a specific Java installation, since Java - unlike Internet Explorer does not operate in Highlander Mode (There can Be Only One).

As an example, the Eclipse IDE does not use JAVA_HOME. it references a config file to find the JVM it will run under.

Tomcat does use JAVA_HOME. It also supports JRE_HOME. Failing that, it will look on your shell instance's PATH for a Java executable and deduce a JAVA_HOME from there. There's likely to be some further tweaks for Tomcat when deployed via the Windows Service manager, but what I listed is what the standard Tomcat startup scripts use.

I hope I'm not reading that you unzipped Tomcat into a JRE/JDK directory. That would horribly pollute both Java and Tomcat. They must be completely separate nor should one contain the other.
 
Thomas Griffith
Ranch Hand
Posts: 62
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
yeah, I fixed it by setting PATH to %JAVA_HME%\bin. However, now I get Error: could not find or load mainclass HelloWorldDST. I have HelloWorldDST.java sitting right in the c: root and see it via dir command. Oh, crap, I moved the source file over, not the class. Hold on...
 
Thomas Griffith
Ranch Hand
Posts: 62
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Ok. After fixing path env. variable, I have the standalone java program running on the Tomcat server to print JVM system date time...


06/07/2021 01:37:56 EDT

Then added...


In daylight time? - true

I don't know where the no DST is coming from. I've tried now with programs on both Notes side and JVM side...and I guess  invoking java.util.Date in Tomcat instantiates from JVM. When i run the Lotus Notes program on the Domino server, it looks good. however, when I run locally with client, it's an hour behind. This isolates it to something within Notes, I think, and I'll start going at it from that angle. i'd be surprised if the code is invoking from clients. Thank you so much again.
 
Tim Holloway
Saloon Keeper
Posts: 23689
161
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Yes, as I said before, Tomcat does not supply time/date/calendar services. Webapps (and things like Tomcat's internal loggers) get their services straight from the JVM. Under Linux.Unit, the JVM gets timezone/DST information from an apallingly complex database called tzdata which is installed as a component of the operating system. I don't know what Windows uses, but if it's like their networking stack, probably ripped off from the same source.

The tzdata database is so complex because it tracks a multiitude of locales throughout history. So it knows, for example, that parts of Indiana go DST and parts do not, that Arizona doesn't do DST, but the Navaho Nation does, but the Zuni reservation within the Navaho Nation does not, that in some parts of the world timezones fit on 15-minute boundaries, that there's a kink in the International Date Line, and that's even before you get to the parts where Western calendars skipped 11 days - at different times in different countries.

And amazingly, the original tzdata set was, so i understand, set up and maintained for a long time by an astrologer and only comparatively recently became a public trust.

According to what I understood, Lotus should be keeping timestamps in UTC, which does not observe DST, You did make it sound like there was a third system besides the Lotus server and the Tomcat server operating as a client? That's where I'd look. It may have DST turned off and/or not have its hardware clock set to UTC.
 
Thomas Griffith
Ranch Hand
Posts: 62
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The Tomcat servlet contains the Lotus Notes Java api (NCSO.jar) in it's lib, so the servlet can access Lotus Notes objects and methods, etc. So when the code converts the Notes DateTime to the java.util.Date, it is using the Lotus Notes toJavaDate() method...



I have a feeling that since toJavaDate(), a native Lotus Notes method, is called outside of the Lotus Domino server context, and instead inside the Tomcat servlet container, it has lost a link to DST (although for some reason retains the current date and time). When I run a Lotus Notes agent with this code, it's good running scheduled on the server but when I run the agent in local Lotus Notes, it likewise has no DST. It's the only thing I can think of and I'm considering re-coding the servlet to check if DST. If true, add one hour to the resultant java.util.Date blah. I hate to do that but through all these tests, there doesn't seem to be anything concrete to explain no DST for blah. Like you said, maybe it's the UTC thing.
 
Tim Holloway
Saloon Keeper
Posts: 23689
161
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I just did a swing through the Lotus documentation and it's not entirely clear. It looks like a Lotus DateTime object includes a timezone.

One thing I did notice, however, is that Domino has its own internal timezone setting irrespective of the OS or current shell settings. So there's something to look at, and in particular how it handles DST.

You might also want to try this:


In other words, instead of plucking the datetime out by brute force, see if this method doesn't provide the necessary corrections.
 
Thomas Griffith
Ranch Hand
Posts: 62
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
ok, thank you. I think the reason it';s not using the getDateTimeValue() as opposed to casting DateTime on a generic getItemValue() is in case it runs into a text value in the ActionDate field. But I'm going to try your suggestion and see how it goes. UPDATE - still the same, no DST using getDateTimeValue().

One thing which I've noticed while working on this is the Notes methods inside the Tomcat web app (dumping NCSO.jar into Tomcat lib). This is a legacy app and I think it's using CORBA for the Lotus Notes integration. Does this require only the Tomcat server with Lotus Notes or do the client pc's seniding requests to Tomcat require Lotus Notes as well? The Lotus Notes server and Tomcat instance are on the same server and the client accesses Louts Notes on the server via URL. I've tested with Lotus Notes client closed on my pc and it's ok but then I'm not sure if CORBA gets into local client registry objects or whatever. I would think only the Lotus Notes server where the Tomcat instance is running but now I'm not sure.
 
Tim Holloway
Saloon Keeper
Posts: 23689
161
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
That was something I was wondering about - were you always getting data straight from the Notes server, or were you ever accessing the Notes database files directly? That last could explain much.

CORBA is basically just Java RMI but more obsolete so I wouldn't worry about it messing up anything. Of course, a more modern access mechanism would use SOAP or JSON. And at this point, even SOAP is pretty obsolete. Then again, I see that current HCL Notes also supports WebDAV.

Actually, there are so many ways to interact with Domino, I'd be hard put to know what approach to take myself. Except not CORBA, of course.

 
Thomas Griffith
Ranch Hand
Posts: 62
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks so much. Ok, inside Lotus Notes, I ran this set of code in a java agent and ran locally...this time, I replaced the Notes method toJavaDate() with basically a mashing together of the date from Notes field (stored as MM/dd/yyyy) and the current system time...date_system comes out as EDT but date_final moves an hour behind on EST....







 
Tim Holloway
Saloon Keeper
Posts: 23689
161
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The one thing I would have liked to have seen is the absolute binary value of that timestamp, but while java.util.Date can return that, I don't think that the Lotus DateTime can.

I should point out, however, that if you use the stock java DateFormat and it says that the time is Standard and not DST, then the JVM running the DateFormat itself isn't operating in DST mode.
 
Thomas Griffith
Ranch Hand
Posts: 62
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
My sample size was looking at Notes records with MM/dd/yyyy in January and February. I expanded the number of records into May and June, those in those warm months returned EDT while January and February returned EST.

I also entered two hard coded dates in a standalone java program...02/25/2020 and 08/25/2020...the former returned EST and the latter EDT.

So I guess the JVM looks at that past date (from Lotus Notes, or whatever the source is, hard-coded, etc...), and adjusts one hour, one way or the other, if EDT/EST is opposite EDT/EST from now.

I think? Would I have to code the above conditional to check each date from Lotus Notes with respect to now and adjust? This seems like a lot to just only need the MM/dd/yyyy. It also seems weird that this doesn't appear to be a bigger issue.
 
Thomas Griffith
Ranch Hand
Posts: 62
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
This is basically what I am going to do. Since i need only MM/dd/yyyy, I am going to "rebuild" the Lotus Notes field date with a time of 06:00:00 and current EST/EDT, then if any EST/EDT one hour adjustment is made, it'll fall back to 05:00:00 or move ahead to 07:00:00, within the same day. I'm still confused as to how to handle EDT/EST if I needed the time elements but this should do it for MM/dd/yyyy, although kind of verbose...



 
 
Tim Holloway
Saloon Keeper
Posts: 23689
161
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I'm note sure if you've tried these, but give them a shot:
 
Thomas Griffith
Ranch Hand
Posts: 62
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
thanks. I think I tried getLocalTIme() and I think it adjusted an hour back for winter dates but I'll try both again tonight after midnight. GMT should not readjust anything as I think about this as it's using the same principle, it is five hours ahead and set the date.
 
reply
    Bookmark Topic Watch Topic
  • New Topic