I have a big data table (1M rows). One column is serialNumber, one column is dateTime, one column is elapsedTime. Another much smaller table also has these columns, but the dateTime is earlier by an small amount (5-20 minutes) and the elapsedTime is a much more accurate value. The small table also has lots of extra rows associated with other events at other dateTimes (I do not have an 'event' column). I want to update the big table with the accurate elapsedTimes by matching the serialNumbers and selecting the largest dateTime less than or equal to the big table's dateTime.
This will be part of a bigger script that is run daily, updating each serialNumber each day.
Right now, my best solution is to compromise on the accuracy and just select the average elapsedTime for the entire day, and using an exact match for the dateTime column. This will introduce an inaccuracy when the small table includes multiple events at multiple times within a given day, but "within 12 hours" is still much more accurate than the big table is presently.
There is another solutions using a pair of nested foreachrows loops, but that will be very slow.
If you have an idea, please just suggest it, don't worry about writing the code (yet).
Thanks,
Jonathan