cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Choose Language Hide Translation Bar
user8421
Level II

How do I join two tables based on timestamps that don't exactly match?

Hi everyone!

 

I am having an issue where I have Data Table 1 as shown below with a timestamp column and various columns of data

 

TimestampColumn AColumn B
02/08/2021 11:45 AMA1B1
02/08/2021 1:39 PMA2B2
02/08/2021 11:37 PMA3B3
02/13/2021 12:01 AMA4B4
02/13/2021 4:00 PMA5B5
02/13/2021 10:22 PMA6B6
02/18/2021 9:00 AMA7B7
02/18/2021 12:01 PMA8B8
02/21/2021 9:15 AMA9B9
02/21/2021 11:50 AMA10B10
02/21/2021 3:00 PMA11B11

 

 

And Data Table 2 also with a timestamp column and corresponding columns of data

 

TimestampColumn CColumn D
02/08/2021 11:24 AMC1D1
02/13/2021 12:07 AMC2D2
02/18/2021 10:29 AMC3D3
02/21/2021 11:16 AMC4D4

 

 

I would like to join these two data tables together where I preserve the timestamp column of Data Table 1, but join Columns C and D from Data Table 2 into Data Table 1 where the data is applied starting from the time indicated in the timestamp in Data Table 2.  The resulting column would look like this:

 

TimestampColumn AColumn BColumn CColumn D
02/08/2021 11:45 AMA1B1C1D1
02/08/2021 1:39 PMA2B2C1D1
02/08/2021 11:37 PMA3B3C1D1
02/13/2021 12:01 AMA4B4C1D1
02/13/2021 4:00 PMA5B5C2D2
02/13/2021 10:22 PMA6B6C2D2
02/18/2021 9:00 AMA7B7C2D2
02/18/2021 12:01 PMA8B8C3D3
02/21/2021 9:15 AMA9B9C3D3
02/21/2021 11:50 AMA10B10C4D4
02/21/2021 3:00 PMA11B11C4D4

 

Thanks in advance for any help!

5 REPLIES 5
jthi
Super User

Re: How do I join two tables based on timestamps that don't exactly match?

Maybe this could help or at least give idea what you could do:

"Fuzzymerge" - join tables with slightly different timestamps 

-Jarmo
txnelson
Super User

Re: How do I join two tables based on timestamps that don't exactly match?

Here is a script that will duplicate the matching you have in your example.

match.PNG

The actual working part of the script is less than 20 lines.  For completeness, I have included the JSL that creates your original example data tables at the beginning of the script.  That allows you to just cut and paste the script and it should run without error.

Names Default To Here( 1 );
dt = New Table( "Base",
	Add Rows( 11 ),
	New Column( "Timestamp",
		Numeric,
		"Continuous",
		Format( "m/d/y h:m", 19 ),
		Input Format( "m/d/y h:m" ),
		Set Values(
			[3695629500, 3695636340, 3695672220, 3696019260, 3696076800, 3696099720, 3696483600, 3696494460, 3696743700,
			3696753000, 3696764400]
		)
	),
	New Column( "Column A",
		Character( 16 ),
		"Nominal",
		Set Values( {"A1", "A2", "A3", "A4", "A5", "A6", "A7", "A8", "A9", "A10", "A11"} )
	),
	New Column( "Column B",
		Character( 16 ),
		"Nominal",
		Set Values( {"B1", "B2", "B3", "B4", "B5", "B6", "B7", "B8", "B9", "B10", "B11"} )
	)
);
dt2 = New Table( "Match",
	Add Rows( 4 ),
	New Column( "Timestamp",
		Numeric,
		"Continuous",
		Format( "m/d/y h:m", 19 ),
		Input Format( "m/d/y h:m" ),
		Set Values( [3695628240, 3696019620, 3696488940, 3696750960] )
	),
	New Column( "Column C", Character( 16 ), "Nominal", Set Values( {"C1", "C2", "C3", "C4"} ) ),
	New Column( "Column D", Character( 16 ), "Nominal", Set Values( {"D1", "D2", "D3", "D4"} ) )
);

// Here is the actual matching code
dt << New Column("TheRow");
dt2 << New Column("TheRow", formula(Row()));

// Find the row in the Match data table that is to be paired with the row
// in the Base data table
For( i = 1, i <= N Rows( dt ), i++,
	dt:theRow[i] = Max( dt2 << get rows where( dt2:timestamp <= dt:Timestamp[i] ) )
);
// Update data tables
Data Table( "Base" ) << Update(
	With( Data Table( "Match" ) ),
	Match Columns( :TheRow = :TheRow )
);
// Delete column: TheRow
Data Table( "Base" ) << Delete Columns( :TheRow );
// Delete column: TheRow
Data Table( "Match" ) << Delete Columns( :TheRow );
Jim

Re: How do I join two tables based on timestamps that don't exactly match?

In your small example it looks like you are just matching based on date, not time. Could that possibly work instead?

 

EDIT: You can safely ignore this comment. I had looked too quickly at the example.

Dan Obermiller
txnelson
Super User

Re: How do I join two tables based on timestamps that don't exactly match?

@Dan_Obermiller 

2 of the rows do not match based upon date.  The table matching as shown seems to be based upon prior date closest to target date.

Jim

Re: How do I join two tables based on timestamps that don't exactly match?

Of course you are right @txnelson. I just looked too quickly. I have edited my original reply.

Dan Obermiller