Having issue with exporting each line item from an SQL server database into it's own XML file.
All the questions I've read on site all have various knowledge points that don't solve the answer for me and I am having one or two pointed gaps in knowledge letting me down. Basically for each item (~350 rows in a table called BPAProcess which exists across 6 databases, each database is basically a different version they're independent of each other. so ideally I can execute this query 6 times manually once on each db to extract all the rows from this table) I want to take XML for each row and save it locally, the xml contents of the cell are already suitable for xml structure they just need to export. So far I've tried creating a loop mechanism to loop over a temp table I created then for each row item extract it into an SQL command to save the details out. I have gotten this far where BCP is failing because it can't see the temporary table ive created, after reading up on this I think its going to fail once i resolve this issue in some other design because there is a character limit of 8k on fiel output but these files are ~300k long each.
USE [BP6.8.1]
GO
-- Save XML records to a file:
DECLARE @fileName VARCHAR(175)
DECLARE @filePath VARCHAR(175)
DECLARE @sqlStr VARCHAR(1000)
DECLARE @sqlCmd VARCHAR(1000)
DECLARE @MaxRowsCount INT
DECLARE @Iter INT
select ROW_NUMBER() OVER (order by processid) row_num, name, processxml into #Process from BPAProcess
SET @MaxRowsCount = (SELECT MAX(row_num) FROM #Process)
SET @Iter = 1
WHILE @Iter <= @MaxRowsCount
BEGIN
SET @fileName = (select name from #Process where row_num = @Iter)
SET @filePath = 'C:\Temp\sql queries\' @fileName '.xml'
SET @sqlStr = 'select processxml from #Process where row_num =' cast(@Iter as varchar(3))
SET @sqlCmd = 'bcp "' @sqlStr '" queryout "' @filePath '" -w -T'
--EXEC xp_cmdshell @sqlCmd
SET @Iter = @Iter 19090
-- 19090 just to execute first iteration only
END
Drop TABLE #Process
I feel what I need is some way to loop the #Process table and for each item at the row_num then export it out via stdout/similar or create a looping mechanism to do a bpc command with the sql pointing at the xml cell I want but I don't know how to make bpc see the table I'm creating.
Few caveats, this is not my DB its a db used by an application and changing anything is not an option. The task is to take each individual XML and store it on a file drive saved as [name].xml where name is in the table. I know this isn't (from reading comments around everywhere) the correct way to use SQL, admittedly im not an SQL developer nor do we have one to hand but Ive been tasked with exporting this code and the manual way on the GUI will take several weeks as it's a long a laborious process wheras this would be much faster. The XML contents are quite long like i said 300k->500k in some instances.
Any help is appreciated, and if that help is 'this is not appropriate for SQL to be executing' that would be fine I could go explore it in C# or some other language potentially if this really isn't the way it should be done.
CodePudding user response：
When you call someone rude, should you expect an actual answer? A developer with any significant experience (you) should be able to find possible solutions and evaluate their usability. As an example, searching "sql server export one row to file" finds first entry using a cursor.
Quite frankly you seem to have significant experience in different languages so I question why you chose a TSQL based solution rather than one involving whatever language that is your strong suit. But that's a very different issue. No matter - RBAR is still RBAR regardless of language / development platform.
Your code converted to a simple cursor is below. There are some things you assume and I again suggest you use a language that you are proficient in.
declare @sql varchar(500);
declare @name varchar(20);
declare c1 cursor FAST_FORWARD for
select name from dbo.mytable order by name;
open c1;
fetch next from c1 into @name;
while @@fetch_status = 0
begin
--print @name;
set @sql = 'select processxml from dbo.mytable where name = ''' @name '''';
set @sql = 'bcp "' @sql '" queryout "' 'C:\Temp\sql queries\' @name '.xml " -w T'
print @sql;
fetch next from c1 into @name;
end;
close c1;
deallocate c1;
You might need to adjust this to connect to the correct instance and database using a specific login. The documentation has cursor examples - always a good starting point. Perhaps the first thing you should try is to use BCP from a command prompt to export that column in a single specific row to a specific file to validate your assumptions and expectations first.
With just a little work you could add some additional logic to run this for multiple databases within a given instance. Doing that would require handling name collisions in the set of files generated.
Lastly, your code as is but without the reliance on the temp table with the BCP command. Note the small adjustment because of the simplified table in the fiddle.
--select ROW_NUMBER() OVER (order by processid) row_num, name, processxml
select ROW_NUMBER() OVER (order by name) row_num, name
into #Process
from dbo.mytable
SET @MaxRowsCount = (SELECT MAX(row_num) FROM #Process)
SET @Iter = 1
WHILE @Iter <= @MaxRowsCount
BEGIN
SET @fileName = (select name from #Process where row_num = @Iter)
SET @filePath = 'C:\Temp\sql queries\' @fileName '.xml'
SET @sqlStr = 'select processxml from dbo.mytable where row_num =' cast(@Iter as varchar(3))
SET @sqlCmd = 'bcp "' @sqlStr '" queryout "' @filePath '" -w -T'
print @sqlCmd;
--EXEC xp_cmdshell @sqlCmd
SET @Iter = @Iter 1
END
Drop TABLE #Process
fiddle to demonstrate both.
CodePudding user response：
This is really not a job for T-SQL, it is not a generalized scripting language, it is only meant for querying.
Instead, use Powershell to extract the XML as separate rows, then output them into files
Invoke-Sqlcmd
-ServerInstance "YourServer"
-Database "YourDB" -Username "user" -Password "pass"
-Query "select name, processxml from dbo.mytable;"
| % {
Out-File -FilePath ("Downloads\temp\" $_.name ".xml") -InputObject $_.processxml
}
You could also use any other client app to do the same, far more easily than in T-SQL.