Sign in
Microsoft.com
 
Microsoft
 
 
 

locked
Unable to access Azure Blob storage from HDInsight cluster RRS feed

 > 

  • Question

  • Question
    Sign in to vote
    0
    Sign in to vote

    Hi,

    I have spun up a HDInsight Spark cluster and am trying to access blob storage on the cluster as follows, but getting an exception:

    hdfs dfs -ls wasbs://deploy@nisumstorageaccount2.blob.core.windows.net/

    20/06/16 19:28:39 ERROR azure.AzureNativeFileSystemStore: Service returned StorageException when checking existence of container deploy in account nisumstorageaccount2.blob.core.windows.net
    com.microsoft.azure.storage.StorageException: 
            at com.microsoft.azure.storage.StorageException.translateException(StorageException.java:87)
            at com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry(ExecutionEngine.java:209)
            at com.microsoft.azure.storage.blob.CloudBlobContainer.exists(CloudBlobContainer.java:769)
            at com.microsoft.azure.storage.blob.CloudBlobContainer.exists(CloudBlobContainer.java:756)
            at org.apache.hadoop.fs.azure.StorageInterfaceImpl$CloudBlobContainerWrapperImpl.exists(StorageInterfaceImpl.java:233)
            at org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.connectUsingAnonymousCredentials(AzureNativeFileSystemStore.java:863)
            at org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.createAzureStorageSession(AzureNativeFileSystemStore.java:1088)
            at org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.initialize(AzureNativeFileSystemStore.java:543)
            at org.apache.hadoop.fs.azure.NativeAzureFileSystem.initialize(NativeAzureFileSystem.java:1358)
            at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3303)
            at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:124)
            at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3352)
            at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3320)
            at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:479)
            at org.apache.hadoop.fs.Path.getFileSystem(Path.java:361)
            at org.apache.hadoop.fs.shell.PathData.expandAsGlob(PathData.java:352)
            at org.apache.hadoop.fs.shell.Command.expandArgument(Command.java:250)
            at org.apache.hadoop.fs.shell.Command.expandArguments(Command.java:233)
            at org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:104)
            at org.apache.hadoop.fs.shell.Command.run(Command.java:177)
            at org.apache.hadoop.fs.FsShell.run(FsShell.java:328)
            at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
            at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
            at org.apache.hadoop.fs.FsShell.main(FsShell.java:391)
    Caused by: java.net.UnknownHostException: nisumstorageaccount2.blob.core.windows.net
            at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184)
            at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
            at java.net.Socket.connect(Socket.java:607)
            at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:681)
            at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
            at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
            at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
            at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264)
            at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)
            at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
            at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1162)
            at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1056)
            at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
            at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1570)
            at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
            at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
            at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:352)
            at com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry(ExecutionEngine.java:115)
            ... 22 more
    ls: org.apache.hadoop.fs.azure.AzureException: No credentials found for account nisumstorageaccount2.blob.core.windows.net in the configuration, and its container deploy is not accessible using anonymous credentials. Please check if the container exists first. If it is not publicly available, you have to provide account credentials.

    Please help.

    Thanks.

    Tuesday, June 16, 2020 8:13 PM

Answers