none
[DEPP LEARING VM AZUR] unable to Access to GPU through jupyter notebook RRS feed

  • Question

  • The problem is as follow: something  odd occurred on the VM Azur for deep learning. It seems like Jupyter notebook is not detecting the GPU.  

    It is supposed to be preconfigured but still only the CPU is detected.

    PS: I want to use the GPU in tensorflow. So, when running this command "import tensorflow as tf; tf.test.gpu_device_name()"  comm I got this output " which means no gpu is being detected


    • Edited by Moni93 Tuesday, April 24, 2018 12:12 PM
    Tuesday, April 24, 2018 10:05 AM

All replies

  • Hi,

    The GPUs are exposed to Jupyter, and TensorFlow should be able to find them. Are the GPUs available when you run nvidia-smi at a terminal?

    Tuesday, April 24, 2018 4:21 PM
    Owner
  • I myself don't understand why it is not detecting GPU.  This is the output of nvidia-smi:

    +-----------------------------------------------------------------------------+
    | NVIDIA-SMI 384.111                Driver Version: 384.111                   |
    |-------------------------------+----------------------+----------------------+
    | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
    | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
    |===============================+======================+======================|
    |   0  Tesla K80           Off  | 00009C0A:00:00.0 Off |                    0 |
    | N/A   57C    P0    59W / 149W |      0MiB / 11439MiB |      0%      Default |
    +-------------------------------+----------------------+----------------------+
    |   1  Tesla K80           Off  | 0000E76C:00:00.0 Off |                    0 |
    | N/A   39C    P0    73W / 149W |      0MiB / 11439MiB |      0%      Default |
    +-------------------------------+----------------------+----------------------+

    +-----------------------------------------------------------------------------+
    | Processes:                                                       GPU Memory |
    |  GPU       PID   Type   Process name                             Usage      |
    |=============================================================================|
    |  No running processes found                                                 |
    +-----------------------------------------------------------------------------+
    • Edited by Moni93 Wednesday, April 25, 2018 8:58 AM
    Wednesday, April 25, 2018 8:58 AM
  • Thanks. Can you confirm that you have the tensorflow-gpu Python package installed and not tensorflow? The latter only has CPU support, so it will print the same empty output you listed earlier. We include tensorflow-gpu on the DSVM, but perhaps you were upgrading TensorFlow or installing another package that installed it.

    Also, is this on Windows or Ubuntu?

    Wednesday, April 25, 2018 11:07 PM
    Owner
  • This is the operation system 
    Linux DeepLearningVM 4.13.0-1011-azure #14-Ubuntu SMP Thu Feb 15 16:15:39 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

    I tried to check if tensorflow-gpu is detected by jupyter notebook, i executed these lines of code :
    import pip
    l = next(str(i) for i in pip.get_installed_distributions() if 'tensorflow-gpu' in str(i))
    print(l)

    i got this result:    'AttributeError: module 'pip' has no attribute 'get_installed_distributions'

    So I tried to upgrade pip:
    pip install virtualenv --upgrade

    Got this result and nothing else:

    Collecting virtualenv  Using cached https://files.pythonhosted.org/packages/ed/ea/e20b5cbebf45d3096e8138ab74eda139595d827677f38e9dd543e6015bdf/virtualenv-15.2.0-py2.py3-none-any.whlInstalling collected packages: virtualenv

    But still it doesn't solve the error 'module 'pip' has no attribute 'get_installed_distributions'.

    So i tried  to install tensorflow-gpu through terminal (because whenusing the VM there is no visual interactive interface that can allow me use anaconda navigator to easily install packages) , I did this command:

    conda install -c anaconda tensorflow-gpu

    And it gave me this error.

      environment variables:
                     CIO_TEST=<not set>
                   CONDA_ROOT=/anaconda
                        CPATH=/opt/intel/mkl/include:
              LD_LIBRARY_PATH=/opt/intel/tbb/lib/intel64/gcc4.7:/opt/intel/lib/intel64:/opt/intel/mk
                              l/lib/intel64:/dsvm/tools/cntk/cntk/lib:/dsvm/tools/cntk/cntk/dependen
                              cies/lib::/usr/local/cuda/lib64
                 LIBRARY_PATH=/opt/intel/tbb/lib/intel64/gcc4.7:/opt/intel/lib/intel64:/opt/intel/mk
                              l/lib/intel64:/usr/local/cuda/lib64:
          MIC_LD_LIBRARY_PATH=/opt/intel/compilers_and_libraries_2017.3.196/linux/compiler/lib/mic:
                      NLSPATH=/opt/intel/mkl/lib/intel64_lin/locale/%l_%t/%N:
                    NODE_PATH=/usr/lib/node_modules
                         PATH=/home/user_deep/bin:/home/user_deep/.local/bin:/anaconda/envs/py35/bin
                              :/dsvm/tools/cntk/cntk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/u
                              sr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/opt/caffe/bin
                              :/usr/local/cuda/bin:/dsvm/tools/spark/spark-2.2.0/bin:/opt/mssql-
                              tools/bin:/dsvm/tools/torch/bin
                   PYTHONPATH=/opt/caffe/python:/opt/caffe2/build:
           REQUESTS_CA_BUNDLE=<not set>
                SSL_CERT_FILE=<not set>


         active environment : None
           user config file : /home/user_deep/.condarc
     populated config files :
              conda version : 4.4.11
        conda-build version : 3.0.23
             python version : 2.7.14.final.0
           base environment : /anaconda  (writable)
               channel URLs : https://conda.anaconda.org/anaconda/linux-64
                              https://conda.anaconda.org/anaconda/noarch
                              https://repo.continuum.io/pkgs/main/linux-64
                              https://repo.continuum.io/pkgs/main/noarch
                              https://repo.continuum.io/pkgs/free/linux-64
                              https://repo.continuum.io/pkgs/free/noarch
                              https://repo.continuum.io/pkgs/r/linux-64
                              https://repo.continuum.io/pkgs/r/noarch
                              https://repo.continuum.io/pkgs/pro/linux-64
                              https://repo.continuum.io/pkgs/pro/noarch
              package cache : /anaconda/pkgs
                              /home/user_deep/.conda/pkgs
           envs directories : /anaconda/envs
                              /home/user_deep/.conda/envs
                   platform : linux-64
                 user-agent : conda/4.4.11 requests/2.18.1 CPython/2.7.14 Linux/4.13.0-1011-azure ubuntu/16.04 glibc/2.23
                    UID:GID : 1003:1003
                 netrc file : None
               offline mode : False

    An unexpected error has occurred. Conda has prepared the above report.
    If submitted, this report will be used by core maintainers to improve
    future releases of conda.


    I don't know what is wrong with this VM .





    • Edited by Moni93 Thursday, April 26, 2018 10:06 AM
    Thursday, April 26, 2018 10:05 AM
  • Any help ????
    Friday, May 4, 2018 12:12 PM
  • Hi,

    It sounds like you've installed some software on the VM, and something has gone wrong. The pip command you ran will work correctly on a new VM. You can also list the installed packages with conda list -n py35, where you'll see tensorflow-gpu listed. 

    So if you're still having trouble, you might try creating a new VM. 

    Also note that we use Anaconda, not virtualenv.

    Friday, May 4, 2018 8:35 PM
    Owner
  • When i execute conda list -n py35 I can obviously see that tensorflow-gpu is detected. But within Jupyter notebook, running this code below( which is supposed to detected wheter the GPU is detected by tensorflow) I get this message "TensorFlow successfully installed. The installed version of TensorFlow does not include GPU support. An exception has occurred, use %tb to see the full traceback."


    Code used to test

    import tensorflow as tf
    print("TensorFlow successfully installed.")
    if tf.test.is_built_with_cuda():
        print("The installed version of TensorFlow includes GPU support.")
    else:
        print("The installed version of TensorFlow does not include GPU support.")
    sys.exit(0)

    Recap

    Creating a new VM is not an option since it's going to cost additional charges. I wanted to find a solution within the same VM.



    • Edited by Moni93 Monday, May 14, 2018 12:15 PM
    Monday, May 14, 2018 12:13 PM
  • Are you using the Python 3 Jupyter kernel? This uses the py35 conda environment. Also, please confirm that you do not have the tensorflow package installed as well.
    Monday, May 14, 2018 6:31 PM
    Owner
  • I am using Python [conda env:py35] Kernel.  

    Tensorflow is working fine. Only the GPU is not detected.

    Tuesday, May 15, 2018 1:58 PM