Migration recently encountered some problems, for the total design update delete hive is not suitable for this kind of operation should do
Such as:
V_begin_time such as variables, the script to set the
1. Oracle SQL: delete from cli_other_distinct c where c. obile in (select mobile from cli_user);
I moved to hive:
Initial version: the delete from dmid cli_other_distinct where mobile in (select mobile from base. Ods_khd_cli_user_di where ymd=${v_date1}) a day from dmid. Delete cli_other_distinct base. Ods_khd_cli_user_di contain table data but not the operation, the efficiency of slow table can be locked
The second version of the hive:
Insert the overwrite table dmid. Cli_other_distinct
Select
T. *
The from dmid. Cli_other_distinct t
Left the join
(select mobile from base. Ods_khd_cli_user_di where ymd='${v_date1}') s
On t.m obile=s.m obile
Where s.m obile is null;
For a full table refresh efficiency is slow,
Because every base. Ods_khd_cli_user_di phone number in the table uncertainty in dmid. Cli_other_distinct which partition can only, so the main table of all matching in the main table data about 3000 w article, efficiency is very slow all update

With bosses to ask you, the latest do migration is a little bit dizzy 5555555555!