• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Date range

 
Ranch Hand
Posts: 86
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
How to store all date strings from the given date range such as from "12/02/2002" to "01/17/2003"?
 
Greenhorn
Posts: 12
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Whats your problem? To store, or to create those Strings?
An easy way for storing is to serialize the Vector containing the Strings through an ObjectOutputStream.
To create the Strings you have to invent some nested loops.
 
Amy Howard
Ranch Hand
Posts: 86
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
It seems that I need two steps for this problem.
(1) get all dates (string type) between d1 and d2.
(2) store dates in vector or array (which one is better?)
Can you please provide me your solution with code in details?
Thanks for your help!!!
 
Ranch Hand
Posts: 3061
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Amy Howard:
It seems that I need two steps for this problem.
(1) get all dates (string type) between d1 and d2.
(2) store dates in vector or array (which one is better?)
Can you please provide me your solution with code in details?
Thanks for your help!!!


I don't think many people here are in the habit of doing programming projects for other people, especially ones that look suspiciously like homework problems. Remember that we voluntarily answer questions here. I'm sure most of us are busy with our own lives and our own programming projects.
With that said, please show that you have put some effort into writing this program yourself. When you have specific questions about specific problems, there are many people here that will be willing to help. However, don't expect any one to provide the complete code for your solution. If that's what you really want, I think you have come to the wrong place.
Layne
 
Amy Howard
Ranch Hand
Posts: 86
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
One of my application function is to enable the users to search for a specific error from all log files within the given date range that the user selects. I tried to use regular expression search.
RE pattern = new RE(date);
pattern.match(line);
But I don't know how to get all dates between the date range.
 
mister krabs
Posts: 13974
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Create a Calendar object with the starting date and then add that to your array (or ArrayList). Then use the Calendar add method and see if you are past the end date. If not, add it to your array. As it says on the shampoo bottle: lather, rinse, repeat.
 
Layne Lund
Ranch Hand
Posts: 3061
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
First, I would create two Date objects to represent the range. Then read an error from the log file into a String and convert it to a Date object as well. I think Date provides some comparison methods so you can easily test if the Date from the log file is between the Dates that represent your range. For the details, you should check the API docs.
HTH
Layne
 
Thomas Paul
mister krabs
Posts: 13974
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Layne Lund:
First, I would create two Date objects to represent the range. Then read an error from the log file into a String and convert it to a Date object as well. I think Date provides some comparison methods so you can easily test if the Date from the log file is between the Dates that represent your range. For the details, you should check the API docs.

This is abetter solution for your particular problem but you should use the Calendar class since it has the convenience comparison methods, before and after.
 
Amy Howard
Ranch Hand
Posts: 86
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I agree with Tom's suggestion to use the Calendar class to add all dates to an ArrayList.
I was confused by Layne's suggestion. Here is an example for the search line:
"[12/23/02 14:54:27:374 EST] 682b6963 WebGroup I SRVE0091I: [Servlet LOG]: Error 20098: InvokerServlet: init"
The user wants to search all error "Error 20098" from 12/20/02" to "12/25/02".
How can convert the search string "Error 20098" to date object?
I don't know whether the regular expression can do search "Error 20098" and the date (exp. "12/23/02") using one combined string. But I can do search "Error 20098" from the result getting from the search on the dates, right?
 
Amy Howard
Ranch Hand
Posts: 86
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Based on Tom's suggestion, I wrote the following code for test. But the output is not correct dates. Do you know what the problem?
***************
output
***************
01/25/2003
01/26/2003
01/27/2003
01/28/2003
01/29/2003
01/30/2003
01/31/2003
02/01/2003
02/02/2003
02/03/2003

public static void main(String[] args)
{
Calendar d1 = new GregorianCalendar(2002,12,25);
Calendar d2 = new GregorianCalendar(2003,01,03);
int dif = (int)((d2.getTime().getTime() - d1.getTime().getTime()) / (1000*60*60*24));
String[] dates = new String[dif + 1];
SimpleDateFormat formatter = new SimpleDateFormat("MM/dd/yyyy");
String s = null;
for(int i = 0; i < dif + 1; i++){
dates[i] = formatter.format(d1.getTime());
System.out.println(dates[i]);
d1.add(Calendar.DATE, 1);
}
}
 
Amy Howard
Ranch Hand
Posts: 86
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Ahh! Actually, the Calendar class presents Jan as 0 and Dec. as 11.
So I need to rewrite the month for each date.
 
Thomas Paul
mister krabs
Posts: 13974
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
By the way, I wrote a class to calculate the timespan between two dates. You can find it in this thread:
https://coderanch.com/t/393160/java/java/calculate-number-days-between-two
To use it for your purpose you could do this:
Calendar d1 = new GregorianCalendar(2003,0,03);
Calendar d2 = new GregorianCalendar(2002,11,25);
TimeSpan ts = TimeSpan.subtract(d1.getTime(), d2.getTime());
String[] dates = new String[ts.getDays()+1];
 
Amy Howard
Ranch Hand
Posts: 86
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Since my application enables the use to search all log files across different servers. Some log files are very large. It will become a performance issue to do a search for each date if the date range is big. Can someone give me a good idea what the appropiate search range should be?
Thanks a lot!
[ January 30, 2003: Message edited by: Amy Howard ]
 
Thomas Paul
mister krabs
Posts: 13974
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Calendar d1 = new GregorianCalendar(2002,11,25);
Calendar d2 = new GregorianCalendar(2003,0,03);
Calendar d3 = new GregorianCalendar(2002,11,26);
Let's say I have the three dates above and I want to know if d3 is between d1 and d2:
if (d3.before(d1) || d3.after(d2)) {}
else {
// do stuff
}
 
Amy Howard
Ranch Hand
Posts: 86
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Please advice me how to create an object of class GregorianCalendar using Date dt1 and Date dt2.
public class DateSpan {

Calendar d1;
Calendar d2;

DateSpan(Date dt1,Date dt2){

d1 = new GregorianCalendar(dt1); //error
d2 = new GregorianCalendar(dt2); //error
}

public String[] getDates(){
int dif = (int)((d2.getTime().getTime() - d1.getTime().getTime()) / (1000*60*60*24));
String[] dates = new String[dif + 1];
SimpleDateFormat formatter = new SimpleDateFormat("MM/dd/yyyy");
String s = null;
for(int i = 0; i < dif + 1; i++){
dates[i] = formatter.format(d1.getTime());
d1.add(Calendar.DATE, 1);
}
return dates;
}
}
 
Thomas Paul
mister krabs
Posts: 13974
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
[12/29/02 14:54:25:421 EST] 682b6963 WebGroup I SRVE0091I: [Servlet LOG]: JSP 1.1 Processor: init
|--------| --> isn't this d3?
 
Thomas Paul
mister krabs
Posts: 13974
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Amy Howard:

d1 = new GregorianCalendar(dt1); //error
d2 = new GregorianCalendar(dt2); //error


d1 = new GregorianCalendar();
d1.setTime(dt1);
 
Amy Howard
Ranch Hand
Posts: 86
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Those dates stored in the log file rather than in any database table. We want to do regular expression search on each lines to find the line has the date within the date range:
RE pattern = new RE(date);
pattern.match(line);
I hope this helps to clear my question.
 
Thomas Paul
mister krabs
Posts: 13974
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
It might be faster to parse it as a Date because once the input date is greater than the end range you know you don't have to read any more records in that log.
 
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
[Damn - I've got to remember to refresh my views of threads when I post after an external interruption. ]
Please advice me how to create an object of class GregorianCalendar using Date dt1 and Date dt2.
Calendar c1 = new GregorianCalendar();
c1.setTime(dt1);
However I personally would stay away from Calendars unless I really need them. They're overweight and have confusing APIs. Date + SimpleDateFormat is sufficient here. E.g. to see if 12/26 is between 12/20 and 12/25:

DateFormat format = new SimpleDateFormat("MM/dd/yyyy");

Date d1 = format.parse("12/20/2002");
Date d2 = format.parse("12/25/2002");
Date d3 = format.parse("12/26/2002");

if ((d1.compareTo(d3) <= 0) && (d2.compareTo(d3) >= 0)) {
// do stuff
}
[/code]
[ January 30, 2003: Message edited by: Jim Yingst ]
 
Jim Yingst
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
You know, it's possible that searching with regex's isn't really the way to go here. If the log file's not too huge, then you might want to just read the entire file and store the data from each log entry in a List of custom LogEntry objects. Or maybe a TreeSet which sorts by date, or a HashMap which uses error codes as the key. Or some combination of the above. Then you can search the data structure for the data you want. E.g. with a TreeSet sorted by date/time, you can get a subset of all entries with dates in a certain range - then check the error code (if any) for each one. Or with a HashMap using error code as a key, just look up the error code, and retrieve some sort of Collection (say another TreeSet sorted by date/time) containing all LogEntry objects with the appropriate error code.
There are a lot of different ways to organize this - you may want to consider more alternatives. Relevant considerations are - how big are the log file(s)? How many files? Will the data all fit in memory? Will there be a lot of different searches run, or just a few? How comfortable are you with regexes? Or collections? Etc. Enjoy...
 
Amy Howard
Ranch Hand
Posts: 86
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I'm working for a big company. We have 7 groups, and each group has his own servers in different envirnoments.The log file search will across all those servers. I used a LogEntry object to store all match entry data.
 
Jim Yingst
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
So, what would happen if you write a program right now to just read all the files and create, say, one big LinkedList with LogEntry objects for each entry? Would the total memory usage be acceptable? (No, having one big LinkedList is probably not the optimal design here - I just want to get an idea of the memory usage.) If the data can fit comfortably in memory, I'd probably pursue the options I listed last post; if not, back to regex searches.
Another possibility - read all the log file entries into a database, and use that do do arbitrarily complex searches. That should keep memory usage down nicely. You've got to invest time in setting up and maintaining the database though.
 
Sheriff
Posts: 6450
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
If I were doing this I might take an approach similar to if I were doing it in Perl. Something like this pseudo-code:

Or something like that anyway. You might have to tweak it, but it should be close.
[ January 30, 2003: Message edited by: Jason Menard ]
 
Jason Menard
Sheriff
Posts: 6450
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I was just thinking that another regex you could use beside pattern (1) which I used above, would be pattern (2). Pattern (2) breaks out the components of the date for use by group(), which would eliminate the need for the split().
(1) "^\[(\d{2}/\d{2}/\d{2}) .*Error " + error + ":"
(2) "^\[((\d{2})/(\d{2})/(\d{2})) .*Error " + error + ":"
group(1) would return the entire date string
group(2) would return the month
group(3) would return the day
group(4) would return the year
My guess though is that using the split() on pattern (1) would be less intensive than using pattern (2) and group() simply because pattern (1) is less complicated, and the split is only called on successful matches.
 
Amy Howard
Ranch Hand
Posts: 86
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks a lot for your effort on this issue. Your suggestions are very helpful!
One more condition, all log files may have different date formats.
[ February 01, 2003: Message edited by: Amy Howard ]
[ February 01, 2003: Message edited by: Amy Howard ]
 
Thomas Paul
mister krabs
Posts: 13974
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
It seems to me that it would be faster to just read in the files and scan the lines. The dates are always in a fixed position so it shouldn't be a big deal. I don't think you need a regex for this at all.
 
Jason Menard
Sheriff
Posts: 6450
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Thomas Paul:
It seems to me that it would be faster to just read in the files and scan the lines. The dates are always in a fixed position so it shouldn't be a big deal. I don't think you need a regex for this at all.


The regex is primarily to choose the lines that match the error code one is looking for, but has the added benefit of easily extracting the date as well.
IMHO, this is one of those things where Perl would probably excel over Java since it was pretty much designed to do text processing like this. Not that Java isn't perfectly capable of doing the same thing. It's probably just the old Perl hacker in me surfacing a bit. Either way, it's about damn time Java incorporated regexes into the JDK. Hell, even Javascript has had them for awhile. :roll:
Yet another regex for the same thing as pattern (1):
"^\[([\d{2}/?]{3}) .*Error " + error + ":"
O'Reilly has a great book on regexes that I recommend anyone interested check out.
 
Thomas Paul
mister krabs
Posts: 13974
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Jason Menard:
Either way, it's about damn time Java incorporated regexes into the JDK.


Your wish is my command:
http://java.sun.com/j2se/1.4/docs/api/java/util/regex/package-summary.html
 
Jim Yingst
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Cool, eh? Let's try this:
Hey Thomas - it's about damn time Java incorporated a foreach keyword and parameterized types into the JDK!
Waiting patiently...
To be fair, I interpreted Jason's comment as "I'm glad regexes were finally incorporated in 1.4, but they were long overdue." Could be mistaken though.
[ January 30, 2003: Message edited by: Jim Yingst ]
 
Thomas Paul
mister krabs
Posts: 13974
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Jim Yingst:
Cool, eh? Let's try this:
Hey Thomas - it's about damn time Java incorporated a foreach keyword and parameterized types into the JDK!
Waiting patiently...


Ummm.... are you in a hurry?
http://java.sun.com/features/2002/03/totiger.html

Side note: I have no idea how the heck they implemented the foreach in C# but it is up to 10X slower than writing your own for loop. Yech!

Originally posted by Jim Yingst:

To be fair, I interpreted Jason's comment as "I'm glad regexes were finally incorporated in 1.4, but they were long overdue." Could be mistaken though.


I did too but I couldn't resist mis-understanding him.
[ January 30, 2003: Message edited by: Thomas Paul ]
[ January 30, 2003: Message edited by: Thomas Paul ]
 
Jason Menard
Sheriff
Posts: 6450
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Thomas Paul:
I did too but I couldn't resist mis-understanding him.



Now if only Java had assertions...
 
Jim Yingst
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hey Jason, quit throwing him easy ones.
Ummm.... are you in a hurry?
Yes, dammit. I'm already making plans for foreach; I want it now.
I have no idea how the heck they implemented the foreach in C# but it is up to 10X slower than writing your own for loop. Yech!
Oh. Well then, I want it just as soon as they've made sure it's not as crappy as Microsoft's version. Though I assume MS should be able to fix that soon - there's no reason it should be slower. Early JDKs has their share of lame bugs too, so I guess I shouldn't judge too harshly.
[ January 31, 2003: Message edited by: Jim Yingst ]
 
Thomas Paul
mister krabs
Posts: 13974
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Jim Yingst:
Yes, dammit. I'm already making plans for foreach; I want it now.


How come no one tells me these cool conversations are going on???
 
Amy Howard
Ranch Hand
Posts: 86
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I tried to create a date range RE string. For example:
From "12/25/2002" to "02/03/2003"
String RE1="12/(25|26|27|28|29|30|31)/2002";
String RE2="01/d+/2003";
String RE3="02/(01|02|03)/2003";
String RE=RE1 + "|" + RE2 + "|" + RE3;
How to write a function to dynamic create the RE string based the given start date and end date?
 
Jim Yingst
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
[TP]: How come no one tells me these cool conversations are going on???
Didn't want to distract you while you work on my request.
[AH]: How to write a function to dynamic create the RE string based the given start date and end date?
Errmmm... well, it's going to get complex I think. Are you sure you really want to do this? For what it's worth Jason's Perl-like pseudocode is pretty much the same sort of solution I was envisioning (except for using that darn GregorianCalendar), and it seems like it would be much simpler.
Note also that you've commented that different log files may have different date formats. This means that all the work you put into creating a regex for one format may not translate well to a different date format. You could find yourself attempting to maintain a lot of different hypercomplex regexes. Added incentive to keep them as simple as possible, as Jason's example does.
But if you really want to make a regext that finds only dates in a particular range, I'd say there are three basic cases to start with (in increasing complexity):
(1) Start and end date have the same year and month.
(2) Start and end date have the same year but different month.
(3) Start and end date have different year and month.
Within each of these cases, you will need to be able to construct subpatterns that match a single number in a given range. Let's say 5 to 17 for example. The simplest way is to list all these numbers as alternates:
(05|06|07|08|09|10|11|12|13|14|15|16|17)
However this may not be as efficient as you'd like. A more complex alternative is something like
(0[5-9]|1[0-7])
Or for a range like 5 to 31:
(0[5-9]|[1-2][0-9]|3[0-1])
Of course [1-2] could be replaced by [12], and [0-9] could be \d - I just used hyphens for consistency. Later examples will use \d where appropriate.
Here's a medium-complexity example of a complete regex for the date range 02/13/02 - 05/04/02 (which falls under case (2) of the three I listed initially, since year is constant):
((02/(1[3-9]|[2-3]\d)|0[3-4]/\d\d|05/([0-1]\d|2[0-4]))/02)
Here's commented Java code that makes this pattern:

I kept all groups as simple capturing groups for readability; you can make some non-capuring of course, or add other groups. Also I left off the error code stuff for simplicity; you can tack it back on later.
Of course this code just makes a nice constant pattern. Generating it dynamically for an arbitrary date range is left as an excercise for the reader, if she's enough of a masochist.
For all that, I'm not even sure this expression will be any faster than a solution based on Jason's pseudocode. And this is certainly harder to program. I'd implement Jason's solution first (it's halfway done already) and see how fast it is. Then try running another search with a statically-created pattern like the one above, to see how fast it is. If you can establish that a pattern like the above really is faster (and that you actually need this increased speed), then write code to generate such a pattern dynamically.
Or if you just like puzzles and programming challenges, go ahead and write a dynamic generator anyway. But don't say you weren't warned.
[ January 31, 2003: Message edited by: Jim Yingst ]
 
Amy Howard
Ranch Hand
Posts: 86
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

For all that, I'm not even sure this expression will be any faster than a solution based on Jason's pseudocode. And this is certainly harder to program. I'd implement Jason's solution first (it's halfway done already) and see how fast it is. Then try running another search with a statically-created pattern like the one above, to see how fast it is.


Jim, thanks for your help and suggestion. I'm going to take your advice to try Jason's solution first.
What are the regex for the following date formats?
31 Jan 2003
2002-11-13
2002/11/08
 
Jim Yingst
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Um, they're all just as complex as the example I showed, but follow very similar principles. Just rearrange the order a bit, or replace / with - or a space. The only thing that's significantly different is the one with month names - a range like Feb-May would be specified as
(Feb|Mar|Apr|May)
for example. (Or substitute whatever actual abbreviations your format uses.)
 
Never trust an airline that limits their passengers to one carry on iguana. Put this tiny ad in your shoe:
a bit of art, as a gift, the permaculture playing cards
https://gardener-gift.com
reply
    Bookmark Topic Watch Topic
  • New Topic