This week's book giveaway is in the Artificial Intelligence and Machine Learning forum.
We're giving away four copies of Transfer Learning for Natural Language Processing (MEAP) and have Paul Azunre on-line!
See this thread for details.
Win a copy of Transfer Learning for Natural Language Processing (MEAP) this week in the Artificial Intelligence and Machine Learning forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Tim Cooke
  • Paul Clapham
  • Devaka Cooray
  • Bear Bibeault
Sheriffs:
  • Junilu Lacar
  • Knute Snortum
  • Liutauras Vilda
Saloon Keepers:
  • Ron McLeod
  • Stephan van Hulst
  • Tim Moores
  • Tim Holloway
  • Piet Souris
Bartenders:
  • salvin francis
  • Carey Brown
  • Frits Walraven

Data type for timestamp?

 
Ranch Hand
Posts: 539
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I came across an OOP problem that deals with transactions. For a transaction, the time is stored in the following format:

XXXX-XX-XXT00:00:00

What's the best way to read, store and compare this type of thing if it's an input? I have never done any programming with timestamps.

I looked at this thread which talks about using SimpleDateFormat, but it seems to have a set format, but I have that "T" in the middle. Would I just have to store it as a string, then extract and compare? Or have some other type?

I might post a new thread for the design of this problem once I get to it, but for now I wanted to get some initial ideas. I might potentially have to compare timestamps for different transactions, or find how many transactions occured in a day etc.

Any ideas would be greatly appreciated!
 
Ranch Hand
Posts: 128
Hibernate Eclipse IDE Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You might want to look at the following new Date API example
 
Marshal
Posts: 68917
275
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Prasanna Raman wrote:. . . the time is stored in the following format:

XXXX-XX-XXT00:00:00 . . .

No, it isn't. It is stored as a LONG INTEGER maybe with an INTEGER to store nanoseconds too. It is only displayed in that format. You can download a timestamp from a database; I think it will correspond to java.sql/java.sql.Timestamp. You can use date formats with and without the T, but you are probably better off turning your object into one of the newer classes immediately. (Look for the section about legacy code.)
 
Prasanna Raman
Ranch Hand
Posts: 539
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello Campbell,

Thank you. I meant I will be receiving input in that format, and I have to compare it, find transactions in a 24 hour period etc.
 
Prasanna Raman
Ranch Hand
Posts: 539
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I was able to use LocalDateTime to parse the input "String". Should I have the method argument itself receive a String, then parse it into LocalDateTime and then do computations with it?

Is that the best way?
 
Campbell Ritchie
Marshal
Posts: 68917
275
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Prasanna Raman wrote:I was able to use LocalDateTime . . .

Well done

Should I have the method argument itself receive a String . . .

Don't know. Try it and see what happens. Maybe you should overload that method taking Strings, multiple ints, etc. Just a suggestion: don't know.
 
Marshal
Posts: 25452
65
Eclipse IDE Firefox Browser MySQL Database
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Well, if your application needs to do computations with dates and times, then yes, you should certainly use LocalDateTime and similar classes. Don't attempt to do date arithmetic with dates in text format, for example.

And if your application is getting date and time information from outside the application, then it may well be coming in as text. So yes, you should certainly convert that text to an LocalDateTime object. And yes, it's usually convenient to write a method to do that.

It's quite common for an application to get a collection of data from an external source and convert the data into Java objects which it will work with.
 
Prasanna Raman
Ranch Hand
Posts: 539
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thank you all. What's the best way to compare different times to see if it's within a let's say 12 or a 24 hour window?

Is there any existing API that makes it easier to do this type of thing?
 
Saloon Keeper
Posts: 11899
253
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Duration works well for this.

You can simply use it like this:
 
Prasanna Raman
Ranch Hand
Posts: 539
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thank you Stephan! That will work nicely for me.

I have input like this,

customerId, time, amount
customerId, time, amount
customerId, time, amount
....

Let's say the above is a list. What's the best way to check if for a given customer id, the amount spent exceeds a certain value for a given time period?

My method takes a List of the above values, and it also knows the spend limit.

I was thinking I could use a Map to store the unique customerIds and values as transactions and then do the calculation.

Is there a better way (or a more OO) to do this?

Any ideas would be welcome!
 
Stephan van Hulst
Saloon Keeper
Posts: 11899
253
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I don't know what your input represents. You need to be very precise in your wording.

Your input is named time, but you're talking about a time period. A time period is an amount of time between two points in time. From the name of your input I can not determine whether it represents an instant in time or an amount of time.

What does the amount input represent? An amount of time? An amount of money? Something else? What is it spent on? How do you determine that you've exceeded the limit?

How do you get the input? From where?
 
Prasanna Raman
Ranch Hand
Posts: 539
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello Stephan,

The input is like this,

CustomerId, time (2020-01-01T00:00:00), $$$

The input is a list of values like this, and I have a set 'max spend' limit.

I have a function that formats the above input and passes it to another function that takes the same list (but with time in LocalDateTime).

That other function is supposed to find all customer IDs that have spent more than the allocated amount in any 12 hour period (the time argument is in chronological order in the list).

I hope this is clearer. Otherwise, I'll try to paste some code here.
 
Saloon Keeper
Posts: 21975
150
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Databases often support multiple forms of time and date object with varying degrees of precision. Some are only accurate to days, some to seconds, some to milliseconds, and so forth, There is no standard, so you have to select from what's available for the database that you are using.

The actual (true) format for a timestamp in a database is not usually related to the form that you code in SQL or get back from SQL utilities. A form like "yyyy-dd-mm hh:mm:ss.iii" in text form takes up a lot of space when compared to something like a 4 or even 8 byte binary stamp value. And since databases are expected to be able to store lots of data, they prefer the more compact binary form internally.

Then you get into the problem of reading and writing such fields in Java. Java's primary date/time objects are granular to milliseconds and microseconds. The java.util.Date and java.sql.Date are accurate to milliseconds. What this means is that if you do fractional-second operations on a java.sql.Date and write it out to a database field that's only accurate to seconds, the data will be correct in the appliction, but get permanently mangled when it's stored and retrieved from the database. Likewise, if you pull a microsecond-accurate timestamp from a database into a java.sql.Date, it's going to get chopped to milliseconds.

So working with timestamps and databases in Java requires some careful planning. And that's even before timezones and Summer Time get factored in.
 
Prasanna Raman
Ranch Hand
Posts: 539
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello Tim,

Thank you for that, but thankfully I am not working with any databases here, just plain Java

My problem is using the input in the given format and then finding the solution from that.
 
Stephan van Hulst
Saloon Keeper
Posts: 11899
253
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Prasanna Raman wrote:That other function is supposed to find all customer IDs that have spent more than the allocated amount in any 12 hour period (the time argument is in chronological order in the list).


Okay, so you need a sliding window?

First, create a strongly typed representation of your input:

Now, given a stream of expenses, you will want to collect them into windows of a preset duration. A window might look like this:
<br /> Collect your expenses: <br />
Finally, for each customer in the map, iterate over the time windows and if the sum of the monetary amounts of the expenses in a single window exceed your maximum, you'll know too much has been spent.
 
Prasanna Raman
Ranch Hand
Posts: 539
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:

Prasanna Raman wrote:That other function is supposed to find all customer IDs that have spent more than the allocated amount in any 12 hour period (the time argument is in chronological order in the list).


Okay, so you need a sliding window?

First, create a strongly typed representation of your input:



Is the requireThat a customer function that I should be creating?
 
Prasanna Raman
Ranch Hand
Posts: 539
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Also, I see that you haven't used the word 'get' in front of getter methods. Is that convention or good practice now?
 
Prasanna Raman
Ranch Hand
Posts: 539
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Honestly, the TimeWindow class looks a bit overwhelming to me! Is that a standard format for creating something like a sliding window?

I am thinking if I need the whole thing or just the method that calculates if it's within duration.
 
Stephan van Hulst
Saloon Keeper
Posts: 11899
253
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Prasanna Raman wrote:Is the requireThat a customer function that I should be creating?


It's something I created for my own projects. You can easily replace it by just writing:

Prasanna Raman wrote:Also, I see that you haven't used the word 'get' in front of getter methods. Is that convention or good practice now?


It's personal taste. I really dislike the convention to put the get- prefix in front of simple accessors. I only use get- when getting the result actually involves some complex or time-consuming calculation, or when I'm writing a Bean. Ditching the get- prefix also makes method handles nicer to use, because it looks like you're referring to a property rather than a function.

Prasanna Raman wrote:Honestly, the TimeWindow class looks a bit overwhelming to me! Is that a standard format for creating something like a sliding window?


I wouldn't say so. It's just something I came up with yesterday, and it should work. Maybe it's overwhelming because it uses generics, but how it works is really not too difficult. The addEventBySlidingWindow() method works by dropping events from one end of the window to make room to add events on the other end of the window. For the T type parameter you can use either Instant or LocalDateTime or ZonedDateTime. For the E parameter you would use whatever it is you want to associate with your timestamps, in this case Expense (or from your other thread, Transaction).
 
Prasanna Raman
Ranch Hand
Posts: 539
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thank you. I'll try and break it down to understand. Can you please explain how the for loop works in the last function that you have written (groupingTimeWindows)?
 
Stephan van Hulst
Saloon Keeper
Posts: 11899
253
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

The result is a map containing a list of windows for each customer ID. Each window for a single customer represents a 12 hour period in which the customer made expenses. Note that the time windows overlap: If I made expenses at 7:00, 12:00, 17:00 and 20:00, it will contain 2 time windows: One containing the expenses at 7:00, 12:00 and 17:00, and one containing the expenses at 12:00, 17:00 and 20:00. The window slid once to drop 7:00 to make room for 20:00.


This piece of code simply ensures that when you encounter a new customer, the result will contain a new list for that customer, containing a single empty window.


The last window is retrieved from the list, and the expense is added to it. If the event fitted in the window without having to slide the window, the original window is returned. Otherwise it creates a new window. If a new window was created, it needs to be added to the list of windows.
 
Prasanna Raman
Ranch Hand
Posts: 539
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thank you. And what does this line do?



And also, in canAddEvent, why do you compare the first entry as well as the last entry with the current event's time? If the first entry
and the current event itself is within max duration, then won't the last entry also automatically be within max duration of this transaction?
 
Prasanna Raman
Ranch Hand
Posts: 539
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Another question: if the source of my input can assure that the input will be in chronological order always, will I still need the complete TimeWindow code? Would I be able to just use some of it?
 
Stephan van Hulst
Saloon Keeper
Posts: 11899
253
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Prasanna Raman wrote:Thank you. And what does this line do?


It removes events from one end of the window to make room for events on the other end of the window.

And also, in canAddEvent, why do you compare the first entry as well as the last entry with the current event's time? If the first entry
and the current event itself is within max duration, then won't the last entry also automatically be within max duration of this transaction?


If the first event in the time window is at 9:00, and the last event in the time window is at 20:00, you can not add 7:00 to the time window because 7:00 - 20:00 exceeds the max window size of 12 hours. So you need to compare the time to add to the last time in the window.

Similarly, you can not add 22:00 to the time window because 9:00 - 22:00 exceeds the max window size of 12 hours. So you also need to compare the time to add to the first time in the window.

Another question: if the source of my input can assure that the input will be in chronological order always, will I still need the complete TimeWindow code? Would I be able to just use some of it?


Then you don't need to check if the time to add will be within the time window defined by the last time. You also don't need to determine the removeAction, because you can always use pollFirstEntry.

These changes are so small that it's not worth giving up the guarantee that the time window will function correctly even when the times are not added in chronological order.
 
Prasanna Raman
Ranch Hand
Posts: 539
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Ah, ok! So in that case I probably won't need a TreeMap either, correct? Actually it can even be a List, can't it? Doesn't need to be a Map at all. Can just store a list of events.

I initially assumed that you had written only for chronological order, that's why I was confused about all the checks for time. Now I understand why.

I am trying to decide between using this as is vs simplifying it given I know that input will be chronological.

Even given that, you don't think we can strip off too many things from this class to warrant a change?
 
Prasanna Raman
Ranch Hand
Posts: 539
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Also, for computing the final result, I am unable to do it without some ugly casting and loop within loop. What's a better way to do this?

 
Stephan van Hulst
Saloon Keeper
Posts: 11899
253
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
If you want, you don't need the class at all. You can also just write a procedural algorithm that rejects expenses:

Disclaimer: I wrote this in a text editor without compiling it, so I don't know if it works.
 
Stephan van Hulst
Saloon Keeper
Posts: 11899
253
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Prasanna Raman wrote:Also, for computing the final result, I am unable to do it without some ugly casting and loop within loop.


It's because you're using raw types. TimeWindow should be TimeWindow<LocalDateTime, Transaction>. NavigableMap should be NavigableMap<String, List<Transaction>>. Object should be String.

If you're on Java 10+, you can use var instead.
 
Prasanna Raman
Ranch Hand
Posts: 539
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thank you. I really like the class way though, but thank you for showing an alternative way.

For my compute method, I have 3 for loops. Could you take a look please and let me know how/if it could be simplified?
 
Stephan van Hulst
Saloon Keeper
Posts: 11899
253
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
From your code I gather that you want to determine—given a list of time windows for a single customer—whether any single time window contains a higher amount of spent money than is allowed.

Your three loops approach really is optimal in terms of time complexity. You could make it a little bit prettier like this:

Note that 'pretty' is in the eye of the beholder. While few here will disagree with my choice to elide the map and the transactionList variables, many will not like that I've removed braces for the second and third for-loop. This is my personal preference. It's important that you stick to what is the norm for your team.

You could also do it like this:

While this code is more declarative, note that the time complexity hasn't gone down, as evidenced by the 3 'nested' stream() invocations (on lines 2, 8 and 9).
 
Prasanna Raman
Ranch Hand
Posts: 539
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thank you again! For some reason, the containsExcessiveSpending method doesn't return any values at all even though I know it should.

Even the version that I had written produced the same output which makes me think TimeWindow may not be working as expected. What would be the easiest way to test it? I've spent quite a bit of time on this, but I am not sure where the bug is.
 
Stephan van Hulst
Saloon Keeper
Posts: 11899
253
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
There are two good ways: debugging and unit testing. The goal is to find out which piece of code doesn't work as intended, so step through the code a line at a time to find out whether the line has the expected result.

Do you know how to set breakpoints in your IDE?
 
Prasanna Raman
Ranch Hand
Posts: 539
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I've identified the problem to be in the loop method, but don't know what it is

I think I am doing something in the computations, but despite having spent more a couple of hours with it I can't figure it out!

Please point me in the right direction!
 
Prasanna Raman
Ranch Hand
Posts: 539
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I think I've identified it. I am not setting the sum back to the computed sum, like



Now, how do I modify the stream version to match this?
 
Stephan van Hulst
Saloon Keeper
Posts: 11899
253
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You don't need to. The reduction operation automatically does this. Have you tried if it works as-is?
 
Prasanna Raman
Ranch Hand
Posts: 539
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Yes, just tried and it works! Thank you. How do you generated Javadocs like the one you had for the method earlier?

Do you just use the IDE and then edit the values?
 
Stephan van Hulst
Saloon Keeper
Posts: 11899
253
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I wrote everything by hand.
 
Do not meddle in the affairs of dragons - for you are crunchy and good with ketchup. Crunchy tiny ad:
Two software engineers solve most of the world's problems in one K&R sized book
https://coderanch.com/wiki/718759/books/Building-World-Backyard-Paul-Wheaton
    Bookmark Topic Watch Topic
  • New Topic