Ceci est une ancienne révision du document !
Table des matières
Linux Compiler Tensorflow avec Bazel
L'objectif est de compiler Tensorflow avec les options AVX2 et FMA !
Ressources
Installation de Bazel
Bazel est le compilateur de Google.
# Bazel sudo apt install apt-transport-https curl gnupg curl -fsSL https://bazel.build/bazel-release.pub.gpg | gpg --dearmor > bazel.gpg sudo mv bazel.gpg /etc/apt/trusted.gpg.d/ echo "deb [arch=amd64] https://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list sudo apt update sudo apt install bazel-4.2.1
Sans GPU sur Debian
Désinstaller les packages python protobuf et tensorflow si ils sont installés.
Installation du nécessaire:
python3 -m pip install numpy wheel python3 -m pip install keras_preprocessing --no-deps
Récupération des sources de tensorflow:
cd /votre/dossier/src git clone https://github.com/tensorflow/tensorflow ./configure
Mes réponses
Make sure you are running at least bazel 4.2.1 Please specify the location of python. [Default is /usr/bin/python3]: Found possible Python library paths: /usr/lib/python3/dist-packages /usr/local/lib/python3.9/dist-packages Please input the desired Python library path to use. Default is [/usr/lib/python3/dist-packages] Do you wish to build TensorFlow with ROCm support? [y/N]: n No ROCm support will be enabled for TensorFlow. Do you wish to build TensorFlow with CUDA support? [y/N]: n No CUDA support will be enabled for TensorFlow. Do you wish to download a fresh release of clang? (Experimental) [y/N]: n Clang will not be downloaded. Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -Wno-sign-compare]: -march=native Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: n Not configuring the WORKSPACE for Android builds.
Avant de recommencer
bazel clean
Compilation de tensorflow
sudo ln -s /usr/bin/python3 /usr/bin/python bazel build -j 4 //tensorflow/tools/pip_package:build_pip_package ./bazel-bin/tensorflow/tools/pip_package/build_pip_package /media/data/tensorflow_pkg
Target tensorflow/tools/pip_package:build_pip_package up-to-date:
bazel-bin/tensorflow/tools/pip_package/build_pip_package
INFO: Elapsed time: 5685.443s, Critical Path: 1098.87s
INFO: 3809 processes: 507 internal, 3302 local.
INFO: Build completed successfully, 3809 total actions
soit 1.58 heures !
===Consommation mémoire===
Sur un i5 à 8 threads et 8 Go de Ram:
* -j 4 pour limiter la consommation de mémoire
====Conseils Geek====
Au lieu de répondre aux questions de ./configure, éditer le fichier .tf_configure.bazelrc du dossier tensorflow (fichier caché, faire Ctrl+h).
<code bash>
build –action_env PYTHON_BIN_PATH=“/usr/bin/python3”
build –action_env PYTHON_LIB_PATH=“/usr/lib/python3/dist-packages”
build –python_path=“/usr/bin/python3”
build:opt –copt=-march=native
build:opt –copt=-Wno-sign-compare
build:opt –host_copt=-march=native
build:opt –define with_default_optimizations=true
test –flaky_test_attempts=3
test –test_size_filters=small,medium
test:v1 –test_tag_filters=-benchmark-test,-no_oss,-gpu,-oss_serial
test:v1 –build_tag_filters=-benchmark-test,-no_oss,-gpu
test:v2 –test_tag_filters=-benchmark-test,-no_oss,-gpu,-oss_serial,-v1only
test:v2 –build_tag_filters=-benchmark-test,-no_oss,-gpu,-v1only
</code>
====Installation====
cd /media/data/3D/projets/grande_echelle_move
source mon_env/bin/activate
python3 -m pip uninstall tensorflow
python3 -m pip install /media/data/src/tensorflow-2.8.0-cp38-cp38-linux_x86_64.whl
=====Avec GPU sur Ubuntu qui ne marche pas !=====
Ubuntu Mate 20.04, python 3.8, venv
===Cuda===
* Installation de CUDA et Compilation de OpenCV avec CUDA
===CudNN===
sudo apt install libcudnn8 libcudnn8-dev
===TensorRT===
sudo apt install libnvinfer8 libnvinfer8-dev
====Tensorflow====
<code bash>
# Dépendances
python3 -m pip install numpy wheel
python3 -m pip install keras_preprocessing –no-deps
# Récupération des sources de tensorflow
mkdir src
cd src/
git clone https://github.com/tensorflow/tensorflow.git
cd tensorflow
./configure
</code>
===Le plus abcons: Configurer la compilation===
* Configurer la compilation
* non à ROCm (non à Rocky)
* non à TensorRT sinon ça plante
* non à clang
* non à download a fresh release of clang ça plante
* oui à GCC
* Pour 1060 GTX –> 6.1, Pour GTX 850m –> 5.0
Comment compiler avec AVX2 et FMA: répondre -march=native
===Reset de la config=== bazel clean ====Compilation Création du package pip Installation==== Parce que en plus bazel exige la bonne version: sudo apt update && sudo apt install bazel-4.2.1 bazel build tensorflow/tools/pip_package:build_pip_package
Après 5 heures de travail d'un I7 à fond!
INFO: Build completed successfully, 32129 total actions
./bazel-bin/tensorflow/tools/pip_package/build_pip_package /media/data/tensorflow_pkg python3 -m pip install /media/data/src/tensorflow-2.8.0-cp38-cp38-linux_x86_64.whl
Test
- Ne marche pas !