In Databricks you can export more than 1000 rows of a table by doing the following:
And downloading the full results.
On Azure Synapse, you get no such option to download the full results:
It defaults to exporting only the first 1000 rows. How can we download all rows of a table to a csv from Azure Synapse without writing more code? I checked and downloading to json is also only 1000 rows.
CodePudding user response:
By default the display()
in Pyspark shows the first 1000 rows only. In databricks it is possible to download more than 1000 rows with display() by re-executing it.
Unfortunately, synapse notebook does not have that feature now. You can raise a feature request for that
The above code is referred from this link by FrancisRomstad.