I want to copy blobs from one storage account (ADLS Gen2) to another storage account (same type). I have even WORKING code with SAS token authorization:

import com.microsoft.azure.storage.{CloudStorageAccount,StorageException,StorageCredentials,StorageCredentialsToken}
import com.microsoft.azure.storage.blob.{CloudBlobClient,CloudBlockBlob,CloudBlobDirectory,CloudBlobContainer,ListBlobItem}

val srcStorageAccountName = "<src-account-name>"
val destStorageAccountName = "<dest-account-name>"


// SAS authorization
val srcSas = "<src-sas-token>"
val srcCloudStorageAccount= CloudStorageAccount.parse(s"DefaultEndpointsProtocol=https;AccountName=${srcStorageAccountName};SharedAccessSignature=${srcSas};EndpointSuffix=core.windows.net")

val destSas = "<dest-sas-token>"
val destCloudStorageAccount= CloudStorageAccount.parse(s"DefaultEndpointsProtocol=https;AccountName=${destStorageAccountName};SharedAccessSignature=${destSas};EndpointSuffix=core.windows.net")

val destBlobClient = destCloudStorageAccount.createCloudBlobClient()
val srcBlobClient = srcCloudStorageAccount.createCloudBlobClient()

try {
  val srcPath = "<src-path>"
  val destPath = "<dest-path>"
  val srcContainerName = "<src-container>"
  val destContainerName = "<dest-container>"
  val srcContainer = srcBlobClient.getContainerReference(srcContainerName)
  val destContainer = destBlobClient.getContainerReference(destContainerName)
  val blobs = srcContainer.listBlobs(srcPath)
  import scala.jdk.CollectionConverters._
  copyBlobList(blobs.asScala, destContainer, srcContainer, srcContainerName, srcPath, destPath)
} catch {
  case e: StorageException =>
    e.printStackTrace()
}

def copyBlobList(blobs: Iterable[ListBlobItem], destContainer: CloudBlobContainer, srcContainer: CloudBlobContainer, srcContainerName: String, srcPath: String, destPath: String): Unit = {
  for (blob <- blobs) {
    blob match {
      case blockBlob: CloudBlockBlob => copySingleBlob(blockBlob, destContainer, srcContainer, srcContainerName, srcPath, destPath)
      case blobDirectory: CloudBlobDirectory => copyBlobDirectory(blobDirectory, destContainer, srcContainer, srcContainerName, srcPath, destPath)
      case _ => println(s"Unknown blob type")
    }
  }
}

def copySingleBlob(blob: CloudBlockBlob, destContainer: CloudBlobContainer, srcContainer: CloudBlobContainer, srcContainerName: String, srcPath: String, destPath: String): Unit = {
  val srcBlob = srcContainer.getBlockBlobReference(blob.getUri.getPath.replaceFirst(srcContainerName, "").substring(1).replace(srcPath, destPath))
  val destBlob = destContainer.getBlockBlobReference(blob.getUri.getPath.replaceFirst(srcContainerName, "").substring(1).replace(srcPath, destPath))
  destBlob.startCopy(srcBlob)
}

def copyBlobDirectory(blobDirectory: CloudBlobDirectory, destContainer: CloudBlobContainer, srcContainer: CloudBlobContainer, srcContainerName: String, srcPath: String, destPath: String): Unit = {
  val blobsFromDir = srcContainer.listBlobs(blobDirectory.getUri.getPath.replace(srcContainerName, ""))
  import scala.jdk.CollectionConverters._
  copyBlobList(blobsFromDir.asScala, destContainer, srcContainer, srcContainerName, srcPath, destPath)
}

The issue here is that any other autorization results in exception: com.microsoft.azure.storage.StorageException: Public access is not permitted on this storage account.

I have tried:

// Account key authorization
val destKey = "<dest-account-key>"
val srcKey =  "<src-account-key>"

val srcCloudStorageAccount = CloudStorageAccount.parse(s"DefaultEndpointsProtocol=https;AccountName=${srcStorageAccountName};AccountKey=${srcKey};EndpointSuffix=core.windows.net")
val destCloudStorageAccount = CloudStorageAccount.parse(s"DefaultEndpointsProtocol=https;AccountName=${destStorageAccountName};AccountKey=${destKey};EndpointSuffix=core.windows.net")

// Access token authorization
val srcTokenCredentials = new StorageCredentialsToken(srcStorageAccountName, srcToken)
val srcCloudStorageAccount = new CloudStorageAccount(srcTokenCredentials, true)

val destTokenCredentials = new StorageCredentialsToken(destStorageAccountName, destToken)
val destCloudStorageAccount = new CloudStorageAccount(destTokenCredentials, true)

Both of them allow to copy blobs inside same storage account but fails on copy action between two different storage accounts.

Question: What I don't really get is why SAS authorization allows to copy between two storage acounts while Account Key autorization and Access Token authorization fails with this "Public access not permitted" exception. Any ideas?

Thanks

CodePudding user response：

• By default, the storage account is not configured to have public access. You can set/check its values from either portal, CLI, Powershell, or Template. If you are granting public access to your storage account, a user with enough permissions can have access to your storage account.
• But by default, A SAS is provided certain permissions while generating it such as read/write, services, resource type, Start and expiry date/time, Allowed IP addresses, etc which gives access to certain parts of your storage account.
• In your case you might provide an access key but due to default permissions with containers and blobs, you are required to provide a SAS token that can be generated by the Access key.
• Providing clients with SAS is recommended that providing with an Access key where the user can have access to all the storage accounts. while SAS gives you restricted access(only a few parts of your storage account with limited time) to your storage account and can also easily revoke the permissions if you don't want to give access anymore by rotating the keys.

REFERENCES:

1. public read access for containers and blobs - MSFT Document
2. Grant limited access to Azure Storage resources using SAS - MSFT Document