I am trying to read data from web sites.
I seem to have an error rate of about 0.1%.
Is this typical? What is the method/procedure to correct this type of error? Should multiple reads be done until a consistent value is obtained?
I'm reading Stock prices from Yahoo and TSP fund prices from thrift savings plan, Thrift Saving Plan Share Prices.
The error is that the historic stock, or fund, price changes since the last time I read it.
For example, I have previously read 19 Sep 2005 G Fund as "11.0100", and that value has previously passed a unit test of the type stock.getprice(date). However, that unit test just failed.
This may not be the best example, but in other cases, I have downloaded different prices for historical dates.
Generally speaking, the error is of the type that a different value is obtained for a fund at some point in the past. But, I always get a value, not just gibberish or garbled text.
That's at least 500% more code than I think is reasonable to ask of people to try and make sense of :-)
Are the two files that give different results at different times identical? At a quick glance, it seems that you're scraping a web page, and web pages differ over time. If they' re not identical, then examining the differences will give you an idea of how to generalize your code so that it can deal with said differences.