Home > Software design >  SAS: Fuzzy Joins
SAS: Fuzzy Joins

Time:12-04

I have the following SQL query I am running in SAS:

proc sql;
create table my_table as
select a.*, b.* 
from table_a a
inner join table_b b
on (a.date_1 between b.date_2 and b.date_3 and a.id1 = b.id1)
or a.id2 = b.id2;
quit;

My Question: I am trying to replace "a.id1 = b.id1" with "a.id1 FUZZY EQUAL a.id1" (https://www.ibm.com/docs/en/psfa/7.2.1?topic=functions-fuzzy-string-search), and have this an "explicit pass-through" (https://www.lexjansen.com/mwsug/2013/RF/MWSUG-2013-RF02.pdf):

 proc sql;
connect to netezza(server = &abc database = &abc user =&abc password = &abc  bunkload = yes);
    create table my_table as
    select a.*, b.* 
    from table_a a
    inner join table_b b
    on (a.date_1 between b.date_2 and b.date_3 where le_dst(a.id1, b.id1) = 1 )
    or a.id2 = b.id2;
    quit;

But I am new to SAS and do not know how to do this properly (the tables are on netezza) .

Can someone please show me how to do this? Are there any other common "fuzzy join" functions that are well suited for this kind of problem?

Thanks!

Note1: The tables look like this:

> table_a

    id1 id2     date_1
1 123 A  11 2010-01-31
2 123BB  12 2010-01-31
3  12 5  14 2015-01-31
4 12--5  13 2018-01-31

> table_b

     id1 id2     date_2     date_3
1   0123 111 2009-01-31 2011-01-31
2   1233 112 2010-01-31 2010-01-31
3 125  .  14 2010-01-31 2020-01-31
4   125_ 113 2010-01-31 2020-01-31

Note 2: R Code used to create these tables for this example (in my original problem, the dates appear in "factor" variable type within R):

table_a = data.frame(id1 = c("123 A", "123BB", "12 5", "12--5"), id2 = c("11", "12", "14", "13"),
date_1 = c("2010-01-31","2010-01-31", "2015-01-31", "2018-01-31" ))

table_a$id1 = as.factor(table_a$id1)
table_a$id2 = as.factor(table_a$id2)
table_a$date_1 = as.factor(table_a$date_1)

table_b = data.frame(id1 = c("0123", "1233", "125  .", "125_"), id2 = c("111", "112", "14", "113"),
date_2 = c("2009-01-31","2010-01-31", "2010-01-31", "2010-01-31" ),
date_3 = c("2011-01-31","2010-01-31", "2020-01-31", "2020-01-31" ))


table_b$id1 = as.factor(table_b$id1)
table_b$id2 = as.factor(table_b$id2)
table_b$date_2 = as.factor(table_b$date_2)
table_b$date_3 = as.factor(table_b$date_3)

CodePudding user response:

Push the SQL into the remote database.

proc sql;
  connect to netezza .... ;
  create table sastable as 
    select * from connection to netezza
      (
select a.*, b.* 
  from table_a a
  inner join table_b b
    on (a.date_1 between b.date_2 and b.date_3)
      and (le_dst(a.id1, b.id1) = 1 or a.id2 = b.id2)
     )
  ;
quit;
  • Related