• Post Reply Bookmark Topic Watch Topic
  • New Topic

How would you build this algorithm?  RSS feed

 
Michael Giara
Greenhorn
Posts: 25
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello!

I'm using java to run through a list of records stored in a CSV file, put there by an Arduino.

Here's an example of the records:


I've  been using the Arduino to keep time, but I've found that its internal clock is not accurate enough, and the particular components I have attached to the Arduino won't allow me to add a Real Time Clock.

I'm wondering how you might elegantly go through these records and find the net time for a particular ID
Say "SP was driving car 111115-005 at 11:36"
"He was also driving it at 11:37" (add 1 minute)
"He was also driving it at 11:38" (add 1 minute)
"He was also driving it at 11:39" (add 1 minute)
"He was also driving it at 11:40" (add 1 minute)
"He was driving a different car (111113-003) at 11:40 and 11:41 so the net drive time for car 111115-005 is 4 minutes and now start counting for 111113-003"

However say at 12:15 PM he starts driving 111115-005 again, so we'll need to find the previous net time for that ID and add onto it.

Accuracy to within a minute or two is fine.
The best way I can explain the reason this will be more accurate than the Arduino even if it's off by a few minutes is by showing you the code:


This seemed like it would work, but I don't know how long it takes to run the checkID method
Based on my testing it's 20-100ms, which again doesn't seem like a lot, but over the course of hours, I'm losing 6-12% of time (checked against Google Stopwatch)
I'm thinking if I just have the Arduino yell out more than once per second which ID it sees, and the data logger stores time received for each record based on the system clock, my accuracy with the new method will be based on the internet-synchronized system clock.

I'm more than happy with pseudo-code, and I've taken a beginner's Data Structures course in Java so feel free to throw lingo at me.
 
Stefan Evans
Bartender
Posts: 1837
10
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Considering analysing the existing data:
Dump the raw data into a database.
Run a query filtering/grouping by CarId would probably give you some useful feedback.


Are you committed to the "add 1 second" approach though?
It appears your problem is you "add 1 second" for every "second + a little bit of processing" of actual time.
Instead of this could you store an actual timestamp?
If I am interpreting this correctly, what you are more interested in is:
- When did a person start using X
- when did a person stop using X
- how long did they use it for?

 
Michael Giara
Greenhorn
Posts: 25
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stefan Evans wrote:Considering analysing the existing data:
Dump the raw data into a database.
Run a query filtering/grouping by CarId would probably give you some useful feedback.


Are you committed to the "add 1 second" approach though?

Nope
It appears your problem is you "add 1 second" for every "second + a little bit of processing" of actual time.

That's correct!  When I initially tested, I calculated the "little bit of processing time" to be approx 40ms, but it's now looking to be more around 160ms (a huge jump)
Instead of this could you store an actual timestamp?

My logging software does this!  If you mean can the Arduino create the time stamp and send it as a part of the record, I don't think the Arduino will be able to unless someone replies to this.

If I am interpreting this correctly, what you are more interested in is:
- When did a person start using X
- when did a person stop using X
- how long did they use it for?

That's more or less how I figured it could be done in my head but it seems like a silly way to go about reaching that third bullet point.
I just figured there'd be a more elegant way to get that "how long" value.
 
Stefan Evans
Bartender
Posts: 1837
10
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
>That's more or less how I figured it could be done in my head but it seems like a silly way to go about reaching that third bullet point.
What strikes you as silly about it?

startTime =X
stopTime = Y
duration = Y - X;

 
Randall Twede
Ranch Hand
Posts: 4696
8
Java Scala
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Sorry I can't help but I am curious. what is an arduino? for that matter what is CSV?
 
Michael Giara
Greenhorn
Posts: 25
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Randall Twede wrote:Sorry I can't help but I am curious. what is an arduino? for that matter what is CSV?

No problem at all.

An Arduino is a programmable microcontroller about the size of a deck of cards.
It has a lot of I/O pins to which peripherals can be connected (sensors, motors, lights, etc)
One would program the Arduino to interact with the peripherals, taking input and sending output.

A CSV is simply a text document. It stands for Comma Separated Value.  You would store data in it with a delimiter.
 
Michael Giara
Greenhorn
Posts: 25
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stefan Evans wrote:>That's more or less how I figured it could be done in my head but it seems like a silly way to go about reaching that third bullet point.
What strikes you as silly about it?

startTime =X
stopTime = Y
duration = Y - X;


I suppose it's not terribly silly.

I just hoped I could make it more intelligent.

I would want to send just barely more than 1 record per minute so that a minute would me much less likely to be skipped, but 2 records for the same "minute" should only count as 1 minute.
Also I didn't think it'd be terribly difficult to make it intelligent enough to decipher "A tick mark for 1:14, a tick mark for 1:15, ..., a tick mark for 1:45, no tick mark for 1:46 (maybe there was an error sending the data), a tick mark for 1:47, 1:48, ... 1:52"
I thought it would be fun for it to know not to exclude 1:46, because while it wasn't explicitly stated, it's implicitly implied that there should be a record for that time.
 
Randall Twede
Ranch Hand
Posts: 4696
8
Java Scala
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I don't know if this will help, but if you are concerned about losing data if it's coming in too fast, there is a solution, at least in windows. If the environment will allow, you. Can increase the priority of your program. Your program will then get more processor time and not wait long for the processors attention. I had to do this once. I can maybe finnd the windows call if you need it.
 
Don't get me started about those stupid light bulbs.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!